Intel(R) Resource Director Technology Cache Pseudo-Locking enabling
Subsystems (2)
v7 cover Intel(R) Resource Director Technology Cache Pseudo-Locking enabling
Dear Maintainers,
The Cache Pseudo-Locking enabling series that was recently merged to the x86/cache branch of tip was found to conflict with the new kernfs support for mounting with fs_context.
In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
In an attempt to simplify this series it is named v7 and is a rebase of x86/cache on top of the new no-op hooks. The
patches were taken from x86/cache and contains all signatures (Signed-off-by:, Acked-by:, Cc:, Link:, etc) as they were merged. Only one of these, "[PATCH V7 36/41] x86/intel_rdt: Create character device exposing pseudo-locked region" needed to be amended and the link within the commit description thus does not point to the actual content of the patch. You will find that the current content of the x86/cache branch is exactly the same as what will be produced by this patch series.
Please let me know if there is a way in which I can modify the series to be easier to manage.
The intention is for the first patch from this series to be placed on a separate, minimal, never rebase branch. x86/cache can include that branch and together with the rest of the series obtain Cache Pseudo-Locking support. The mount series can also include the minimal branch with the single commit to complete the fs_context support to kernfs.
No changes below. It is verbatim from previous submission (except for diffstat at the end that reflects v7).
Changes since v4:
- All occurrences of seq_puts(f, one_char_string) replaced with seq_putc(f, one_char).
- Do not use unlikely() after debugfs_file_get().
- No error checking of debugfs directory and file creation return values. Specifically, do not let failures in debugging affect core functionality.
- Always include debug functionality. CONFIG_INTEL_RDT_DEBUGFS has been removed in this series.
Changes since v3:
- Rebase series on top of tip::x86/cache with HEAD:
commit de73f38f768021610bd305cf74ef3702fcf6a1eb (tip/x86/cache)
Author: Vikas Shivappa <[email protected]>
Date: Fri Apr 20 15:36:21 2018 -0700
x86/intel_rdt/mba_sc: Feedback loop to dynamically update mem bandwidth
- The final patch from the v3 submission is not included in this series. The large contiguous allocation work it depends on is actively discussed and patches now at v2:
http://lkml.kernel.org/r/[email protected]
At this time it seems that the large contiguous allocation API may change in future versions so I plan to resubmit the final patch when that is finalized. Until then we are limited to Cache Pseudo-Locked regions of 4MB.
- rdtgroup_cbm_overlaps() now returns bool instead of int.
- rdtgroup_mode_test_exclusive() now returns bool instead of int.
- Respect the tabular formatting in rdt_cbm_parse_data struct declaration.
- In rdt_bit_usage_show() use test to exit loop earlier and thus spare an indentation level of code that follows.
- Follow recommendations of recent additions to checkpatch.pl:
-- Prefer 'help' over '---help---' for new Kconfig help texts. -- Include SPDX-License-Identifier tag in new files.
The last patch of this series depends on the series:
"[RFC PATCH 0/3] Interface for higher order contiguous allocations" submitted at:
http://lkml.kernel.org/r/[email protected]
A new version of this was submitted recently and currently being discussed at:
http://lkml.kernel.org/r/[email protected]
Without this upstream MM work (and patch 39/39 of this series) it would just not be possible to create pseudo-locked regions larger than 4MB. To simplify this work we could temporarily drop the last patch of this series until the upstream MM work is complete.
Changes since v2:
- Introduce resource group "modes" and a new resctrl file "mode" associated with each resource group that exposes the associated resource group's mode. A resource group's mode is used by the system administrator to enable or disable resource sharing between resource groups. A resource group in "shareable" mode allows its allocations to be shared with other resource groups. This is the default mode and reflects existing behavior. A resource group in "exclusive" mode does not allow any sharing of its allocated resources. When a schemata is written to any resource group it is not allowed to overlap with allocations of any resource group that is in "exclusive" mode. A resource group's allocations are not allowed to overlap at the time it is set to be "exclusive". Cache pseudo-locking builds on "exclusive" mode and is supported using two new modes: "pseudo-locksetup" lets the user indicate that this resource group will be used by a pseudo-locked region. A subsequent write of a schemata to the "schemata" file will create the corresponding pseudo-locked region and the mode will then automatically change to "pseudo-locked".
- A resource group's mode can only be changed to "pseudo-locksetup" if the platform has been verified to support cache pseudo-locking and the resource group is unused. Unused means that, no monitoring is in progress, and no tasks or cpus are assigned to the resource group. Once a resource group enters "pseudo-locksetup" it becomes "locked down" such that no new tasks or cpus can be assigned to it. Neither can any new monitoring be started.
- Each resource group obtains a new "size" file that mirrors the schemata file to display the size in bytes of each allocation. There is a difference in the implementation from the review feedback. In the review feedback an example of output was:
L2:0=128K;1=256K;
L3:0=1M;1=2M; Within the kernel I could find many examples of support for user _input_ with mem suffixes. This is broadly supported with lib/cmdline.c:memparse(). I was not able to find as clear support or usage of such flexible _output_ of size. My conclusion was that the output of size tends to always be using the same unit. I also found that printing the size in one unit, in this case bytes, does simplify validation.
- A new "bit_usage" file within the info/<resource> sub-directories contain annotated bitmaps of how the resources are used.
- Cache pseudo-locked regions are now associated 1:1 with a resource group.
- Do not make any changes to capacity bitmask (CBM) associated with the default class-of-service (CLOS). If a pseudo-locked region is requested its cache region has to be unused at the time of request.
- Second mutex removed.
- Tabular fashion respected when making struct changes.
- Lifetime of pseudo-locked region (by extension the resource group it belongs to) connected to mmap region.
- Do not call preempt_disable() and local_irq_save(). Only local_irq_disable().
- Improve comments in pseudo-locking loop to explain why prefetcher needs disabling.
- Ensure that possibility of pseudo-locked region success takes into account all levels of cache in the hierarchy, not just the level at which it is requested.
- Preloading of code was suggested in review to improve pseudo-locking success. We have since been able to connect a hardware debugger to our target platform and with current locking flow we are able to lock 100% of kernel memory into the cache of an Intel(R) Celeron(R) Processor J3455.
- Above testing with hardware debugger revealed that speculative execution of the loop loads data beyond the end of the buffer. Add a read barrier to the locking loops to prevent this speculation.
- The name of the debugfs file used to trigger measurements was changed from "measure_trigger" to "pseudo_lock_measure".
Changes since v1:
- Enable allocation of contiguous regions larger than what SLAB allocators can support. This removes the 4MB Cache Pseudo-Locking limitation documented in v1 submission.
This depends on "mm: drop hotplug lock from lru_add_drain_all", now in v4.16-rc1 as 9852a7212324fd25f896932f4f4607ce47b0a22f.
- Convert to debugfs_file_get() and -put() from the now obsolete debugfs_use_file_start() and debugfs_use_file_finish() calls.
- Rebase on top of, and take into account, recent L2 CDP enabling.
- Simplify tracing output to print cache hits and miss counts on same line.
Dear Maintainers,
Cache Allocation Technology (CAT), part of Intel(R) Resource Director Technology (Intel(R) RDT), enables a user to specify the amount of cache space into which an application can fill. Cache pseudo-locking builds on the fact that a CPU can still read and write data pre-allocated outside its current allocated area on cache hit. With cache pseudo-locking data can be preloaded into a reserved portion of cache that no application can fill, and from that point on will only serve cache hits. The cache pseudo-locked memory is made accessible to user space where an application can map it into its virtual address space and thus have a region of memory with reduced average read latency.
The cache pseudo-locking approach relies on generation-specific behavior of processors. It may provide benefits on certain processor generations, but is not guaranteed to be supported in the future. It is not a guarantee that data will remain in the cache. It is not a guarantee that data will remain in certain levels or certain regions of the cache. Rather, cache pseudo-locking increases the probability that data will remain in a certain level of the cache via carefully configuring the CAT feature and carefully controlling application behavior.
Known limitations:
Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict pseudo-locked memory from the cache. Power management C-states may still shrink or power off cache causing eviction of cache pseudo-locked memory. We utilize PM QoS to prevent entering deeper C-states on cores associated with cache pseudo-locked regions at the time they (the pseudo-locked regions) are created.
Known software limitation (FIXED IN V2):
Cache pseudo-locked regions are currently limited to 4MB, even on platforms that support larger cache sizes. Work is in progress to support larger regions.
Graphs visualizing the benefits of cache pseudo-locking on an Intel(R) NUC NUC6CAYS (it has an Intel(R) Celeron(R) Processor J3455) with the default 2GB DDR3L-1600 memory are available. In these tests the patches from this series were applied on the x86/cache branch of tip.git at the time the HEAD was:
commit 87943db7dfb0c5ee5aa74a9ac06346fadd9695c8 (tip/x86/cache)
Author: Reinette Chatre <[email protected]> Date: Fri Oct 20 02:16:59 2017 -0700
x86/intel_rdt: Fix potential deadlock during resctrl mount
DISCLAIMER: Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Performance varies depending on system configuration.
- https://github.com/rchatre/data/blob/master/cache_pseudo_locking/rfc_v1/perfcount.png
Above shows the few L2 cache misses possible with cache pseudo-locking
on the Intel(R) NUC with default configuration. Each test, which is repeated 100 times, pseudo-locks schemata shown and then measure from
the kernel via precision counters the number of cache misses when accessing the memory afterwards. This test is run on an idle system as well as a system with significant noise (using stress-ng) from a neighboring core associated with the same cache. This plot shows us that:
(1) the number of cache misses remain consistent irrespective of the size of region being pseudo-locked, and (2) the number of cache misses for a pseudo-locked region remains low when traversing memory regions ranging in size from 256KB (4096 cache lines) to 896KB (14336 cache lines).
- https://github.com/rchatre/data/blob/master/cache_pseudo_locking/rfc_v1/userspace_malloc_with_load.png
Above shows the read latency experienced by an application running with
default CAT CLOS after it allocated 256KB memory with malloc() (and usingmlockall()). In this example the application reads randomly (to not trigger
hardware prefetcher) from its entire allocated region at 2 second intervals
while there is a noisy neighbor present. Each individual access is 32 bytes
in size and the latency of each access is measured using the rdtsc
instruction. In this visualization we can observe two groupings of data,
the group with lower latency indicating cache hits, and the group with
higher latency indicating cache misses. We can see a significant portion
of memory reads experience larger latencies.
- https://github.com/rchatre/data/blob/master/cache_pseudo_locking/rfc_v1/userspace_psl_with_load.png
Above plots a similar test as the previous, but instead of the application
reading from a 256KB malloc() region it reads from a 256KB pseudo-locked
region that was mmap()'ed into its address space. When comparing these
latencies to that of regular malloc() latencies we do see a significant
improvement in latencies experienced.
https://github.com/rchatre/data/blob/master/cache_pseudo_locking/rfc_v1/userspace_malloc_and_cat_with_load_clos0_fixed.png
Applications that are sensitive to latencies may use existing CAT technology to isolate the sensitive application. In this plot we show an application running with a dedicated CAT CLOS double the size (512KB) of the memory being tested (256KB). A dedicated CLOS with CBM 0x0f is created and the default CLOS changed to CBM 0xf0. We see in this plot that even though the application runs within a dedicated portion of cache it still experiences significant latency accessing its memory (when compared to pseudo-locking).
Your feedback about this proposal for enabling of Cache Pseudo-Locking will be greatly appreciated.
Regards,
Reinette
Ingo Molnar (1):
x86/intel_rdt: Simplify index type
Reinette Chatre (40):
x86/intel_rdt: Provide pseudo-locking hooks within rdt_mount x86/intel_rdt: Document new mode, size, and bit_usage x86/intel_rdt: Introduce RDT resource group mode x86/intel_rdt: Associate mode with each RDT resource group x86/intel_rdt: Introduce resource group's mode resctrl file x86/intel_rdt: Introduce test to determine if closid is in use x86/intel_rdt: Make useful functions available internally x86/intel_rdt: Initialize new resource group with sane defaults x86/intel_rdt: Introduce new "exclusive" mode x86/intel_rdt: Enable setting of exclusive mode x86/intel_rdt: Making CBM name and type more explicit x86/intel_rdt: Support flexible data to parsing callbacks x86/intel_rdt: Ensure requested schemata respects mode x86/intel_rdt: Introduce "bit_usage" to display cache allocations
details
x86/intel_rdt: Display resource groups' allocations' size in bytes x86/intel_rdt: Documentation for Cache Pseudo-Locking x86/intel_rdt: Introduce the Cache Pseudo-Locking modes x86/intel_rdt: Respect read and write access x86/intel_rdt: Add utility to test if tasks assigned to resource group x86/intel_rdt: Add utility to restrict/restore access to resctrl files x86/intel_rdt: Protect against resource group changes during locking x86/intel_rdt: Utilities to restrict/restore access to specific files x86/intel_rdt: Add check to determine if monitoring in progress x86/intel_rdt: Introduce pseudo-locked region x86/intel_rdt: Support enter/exit of locksetup mode x86/intel_rdt: Enable entering of pseudo-locksetup mode x86/intel_rdt: Split resource group removal in two x86/intel_rdt: Add utilities to test pseudo-locked region possibility x86/intel_rdt: Discover supported platforms via prefetch disable bits x86/intel_rdt: Pseudo-lock region creation/removal core x86/intel_rdt: Support creation/removal of pseudo-locked region x86/intel_rdt: Resctrl files reflect pseudo-locked information x86/intel_rdt: Ensure RDT cleanup on exit
x86/intel_rdt: Create resctrl debug area
x86/intel_rdt: Create debugfs files for pseudo-locking testing x86/intel_rdt: Create character device exposing pseudo-locked region x86/intel_rdt: More precise L2 hit/miss measurements x86/intel_rdt: Support L3 cache performance event of Broadwell x86/intel_rdt: Limit C-states dynamically when pseudo-locking active x86/intel_rdt: Fix passing of value to 32-bit register
Documentation/x86/intel_rdt_ui.txt | 377 ++++- arch/x86/kernel/cpu/Makefile | 4 +- arch/x86/kernel/cpu/intel_rdt.c | 11 + arch/x86/kernel/cpu/intel_rdt.h | 142 +- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 129 +- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 1505 +++++++++++++++++ .../kernel/cpu/intel_rdt_pseudo_lock_event.h | 43 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 798 ++++++++- 8 files changed, 2936 insertions(+), 73 deletions(-) create mode 100644 arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c create mode 100644 arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h
Signature · 1 line
2.17.0
PATCH v7 1/41 x86/intel_rdt: Provide pseudo-locking hooks within rdt_mount
Stephen Rothwell reported that the Cache Pseudo-Locking enabling and the kernfs support for mounting with fs_context are conflicting.
In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +18 -1
@@ -1236,6 +1236,15 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,struct rdtgroup *prgrp, struct kernfs_node **mon_data_kn); +/* +* Define the hooks for Cache Pseudo-Locking to use within rdt_mount(). +* These are no-ops provided for the new kernfs changes to use as a +* baseline in preparation for a conflict-free merge between it +* (kernfs changes) and the Cache Pseudo-Locking enabling. +*/ +#define rdt_pseudo_lock_init () 0 +#define rdt_pseudo_lock_release () do { } while (0) +static struct dentry *rdt_mount (struct file_system_type *fs_type,int flags,const char *unused_dev_name,void *data)
@@ -1289,10 +1298,16 @@ static struct dentry *rdt_mount(struct file_system_type *fs_type, rdtgroup_default.mon.mon_data_kn = kn_mondata; } + ret =rdt_pseudo_lock_init (); +if (ret) { + dentry =ERR_PTR (ret); + goto out_mondata; + } + dentry =kernfs_mount (fs_type, flags, rdt_root,RDTGROUP_SUPER_MAGIC ,NULL );if (IS_ERR (dentry)) - goto out_mondata; + goto out_psl;if (rdt_alloc_capable)static_branch_enable_cpuslocked (&rdt_alloc_enable_key);
@@ -1310,6 +1325,8 @@ static struct dentry *rdt_mount(struct file_system_type *fs_type, goto out; +out_psl: +rdt_pseudo_lock_release (); out_mondata:if (rdt_mon_capable)kernfs_remove (kn_mondata);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Provide pseudo-locking hooks within rdt_mount
x86/intel_rdt: Provide pseudo-locking hooks within rdt_mount
Commit-ID: 32206ab36553be8714d9253ce33f4085681d369c Gitweb: https://git.kernel.org/tip/32206ab36553be8714d9253ce33f4085681d369c Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:52 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 12:53:19 +0200
x86/intel_rdt: Provide pseudo-locking hooks within rdt_mount
Stephen Rothwell reported that the Cache Pseudo-Locking enabling and the kernfs support for mounting with fs_context are conflicting.
In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
--- arch/x86/kernel/cpu/intel_rdt.h | 9 +++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 10 +++++++++- 2 files changed, 18 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt.h +9 -0
@@ -468,4 +468,13 @@ void cqm_handle_limbo(struct work_struct *work);bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free); +/* +* Define the hooks for Cache Pseudo-Locking to use within rdt_mount(). +* These are no-ops provided for the new kernfs changes to use as a +* baseline in preparation for a conflict-free merge between it +* (kernfs changes) and the Cache Pseudo-Locking enabling. +*/ +static inline int rdt_pseudo_lock_init (void ) {return 0 ; } +static inline void rdt_pseudo_lock_release (void ) { } +#endif /* _ASM_X86_INTEL_RDT_H */
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +9 -1
@@ -1289,10 +1289,16 @@ static struct dentry *rdt_mount(struct file_system_type *fs_type, rdtgroup_default.mon.mon_data_kn = kn_mondata; } + ret =rdt_pseudo_lock_init (); +if (ret) { + dentry =ERR_PTR (ret); + goto out_mondata; + } + dentry =kernfs_mount (fs_type, flags, rdt_root,RDTGROUP_SUPER_MAGIC ,NULL );if (IS_ERR (dentry)) - goto out_mondata; + goto out_psl;if (rdt_alloc_capable)static_branch_enable_cpuslocked (&rdt_alloc_enable_key);
@@ -1310,6 +1316,8 @@ static struct dentry *rdt_mount(struct file_system_type *fs_type, goto out; +out_psl: +rdt_pseudo_lock_release (); out_mondata:if (rdt_mon_capable)kernfs_remove (kn_mondata);
PATCH v7 2/41 x86/intel_rdt: Document new mode, size, and bit_usage
In reply to Reinette ChatreBy default resource groups allow sharing of their cache allocations. There is nothing that prevents a resource group from configuring a cache allocation that overlaps with that of an existing resource group.
To enable resource groups to specify that their cache allocations cannot be shared a resource group "mode" is introduced to support two possible modes:
"shareable" and "exclusive". A "shareable" resource group allows sharing of its cache allocations, an "exclusive" resource group does not. A new resctrl file "mode" associated with each resource group is used to communicate its (the associated resource group's) mode setting and allow the mode to be changed. The new "mode" file as well as two other resctrl files, "bit_usage" and "size", are introduced in this series.
Add documentation for the three new resctrl files as well as one example demonstrating their use.
--- Documentation/x86/intel_rdt_ui.txt | 99 +++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 2 deletions(-)
Documentation/x86/intel_rdt_ui.txt +97 -2
@@ -65,6 +65,27 @@ related to allocation: some platforms support devices that have their own settings for cache use which can over-ride these bits. +"bit_usage": Annotated capacity bitmasks showing how all + instances of the resource are used. The legend is: + "0" - Corresponding region is unused. When the system's + resources have been allocated and a "0" is found + in "bit_usage" it is a sign that resources are + wasted. + "H" - Corresponding region is used by hardware only + but available for software use. If a resource + has bits set in "shareable_bits" but not all + of these bits appear in the resource groups' + schematas then the bits appearing in + "shareable_bits" but no resource group will + be marked as "H". + "X" - Corresponding region is available for sharing and + used by hardware and software. These are the + bits that appear in "shareable_bits" as + well as a resource group's allocation. + "S" - Corresponding region is used by software + and available for sharing. + "E" - Corresponding region is used exclusively by + one resource group. No sharing allowed. Memory bandwitdh(MB) subdirectory contains the following files with respect to allocation:
@@ -163,6 +184,16 @@ When control is enabled all CTRL_MON groups will also contain: A list of all the resources available to this group. Each resource has its own line and format - see below for details. +"size": + Mirrors the display of the "schemata" file to display the size in + bytes of each allocation instead of the bits representing the + allocation. + +"mode": + The "mode" of the resource group dictates the sharing of its + allocations. A "shareable" resource group allows sharing of its + allocations while an "exclusive" resource group does not. + When monitoring is enabled all MON groups will also contain: "mon_data":
@@ -502,7 +533,71 @@ siblings and only the real time threads are scheduled on the cores 4-7. # echo F0 > p0/cpus -4) Locking between applications +Example 4 +--------- + +The resource groups in previous examples were all in the default "shareable" +mode allowing sharing of their cache allocations. If one resource group +configures a cache allocation then nothing prevents another resource group +to overlap with that allocation. + +In this example a new exclusive resource group will be created on a L2 CAT +system with two L2 cache instances that can be configured with an 8-bit +capacity bitmask. The new exclusive resource group will be configured to use +25% of each cache instance. + +# mount -t resctrl resctrl /sys/fs/resctrl/ +# cd /sys/fs/resctrl + +First, we observe that the default group is configured to allocate to all L2 +cache: + +# cat schemata +L2:0=ff;1=ff + +We could attempt to create the new resource group at this point, but it will +fail because of the overlap with the schemata of the default group: +# mkdir p0 +# echo 'L2:0=0x3;1=0x3' > p0/schemata +# cat p0/mode +shareable +# echo exclusive > p0/mode +-sh: echo: write error: Invalid argument +# cat info/last_cmd_status +schemata overlaps + +To ensure that there is no overlap with another resource group the default +resource group's schemata has to change, making it possible for the new +resource group to become exclusive. +# echo 'L2:0=0xfc;1=0xfc' > schemata +# echo exclusive > p0/mode +# grep . p0/* +p0/cpus:0 +p0/mode:exclusive +p0/schemata:L2:0=03;1=03 +p0/size:L2:0=262144;1=262144 + +A new resource group will on creation not overlap with an exclusive resource +group: +# mkdir p1 +# grep . p1/* +p1/cpus:0 +p1/mode:shareable +p1/schemata:L2:0=fc;1=fc +p1/size:L2:0=786432;1=786432 + +The bit_usage will reflect how the cache is used: +# cat info/L2/bit_usage +0=SSSSSSEE;1=SSSSSSEE + +A resource group cannot be forced to overlap with an exclusive resource group: +# echo 'L2:0=0x1;1=0x1' > p1/schemata +-sh: echo: write error: Invalid argument +# cat info/last_cmd_status +overlaps with exclusive group + +Locking between applications +---------------------------- Certain operations on the resctrl filesystem, composed of read/writes to/from multiple files, must be atomic.
@@ -510,7 +605,7 @@ to/from multiple files, must be atomic. As an example, the allocation of an exclusive reservation of L3 cache involves: - 1. Read the cbmmasks from each directory + 1. Read the cbmmasks from each directory or the per-resource "bit_usage" 2. Find a contiguous set of bits in the global CBM bitmask that is clear in any of the directory cbmmasks 3. Create a new directory
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Document new mode, size, and bit_usage
x86/intel_rdt: Document new mode, size, and bit_usage
Commit-ID: cba1aab84fb815675038f9f932c1dd0a5519c63d Gitweb: https://git.kernel.org/tip/cba1aab84fb815675038f9f932c1dd0a5519c63d Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:53 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:40 +0200
x86/intel_rdt: Document new mode, size, and bit_usage
By default resource groups allow sharing of their cache allocations. There is nothing that prevents a resource group from configuring a cache allocation that overlaps with that of an existing resource group.
To enable resource groups to specify that their cache allocations cannot be shared a resource group "mode" is introduced to support two possible modes:
"shareable" and "exclusive". A "shareable" resource group allows sharing of its cache allocations, an "exclusive" resource group does not. A new resctrl file "mode" associated with each resource group is used to communicate its (the associated resource group's) mode setting and allow the mode to be changed. The new "mode" file as well as two other resctrl files, "bit_usage" and "size", are introduced in this series.
Add documentation for the three new resctrl files as well as one example demonstrating their use.
--- Documentation/x86/intel_rdt_ui.txt | 99 +++++++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 2 deletions(-)
Documentation/x86/intel_rdt_ui.txt +97 -2
@@ -65,6 +65,27 @@ related to allocation: some platforms support devices that have their own settings for cache use which can over-ride these bits. +"bit_usage": Annotated capacity bitmasks showing how all + instances of the resource are used. The legend is: + "0" - Corresponding region is unused. When the system's + resources have been allocated and a "0" is found + in "bit_usage" it is a sign that resources are + wasted. + "H" - Corresponding region is used by hardware only + but available for software use. If a resource + has bits set in "shareable_bits" but not all + of these bits appear in the resource groups' + schematas then the bits appearing in + "shareable_bits" but no resource group will + be marked as "H". + "X" - Corresponding region is available for sharing and + used by hardware and software. These are the + bits that appear in "shareable_bits" as + well as a resource group's allocation. + "S" - Corresponding region is used by software + and available for sharing. + "E" - Corresponding region is used exclusively by + one resource group. No sharing allowed. Memory bandwitdh(MB) subdirectory contains the following files with respect to allocation:
@@ -163,6 +184,16 @@ When control is enabled all CTRL_MON groups will also contain: A list of all the resources available to this group. Each resource has its own line and format - see below for details. +"size": + Mirrors the display of the "schemata" file to display the size in + bytes of each allocation instead of the bits representing the + allocation. + +"mode": + The "mode" of the resource group dictates the sharing of its + allocations. A "shareable" resource group allows sharing of its + allocations while an "exclusive" resource group does not. + When monitoring is enabled all MON groups will also contain: "mon_data":
@@ -502,7 +533,71 @@ siblings and only the real time threads are scheduled on the cores 4-7. # echo F0 > p0/cpus -4) Locking between applications +Example 4 +--------- + +The resource groups in previous examples were all in the default "shareable" +mode allowing sharing of their cache allocations. If one resource group +configures a cache allocation then nothing prevents another resource group +to overlap with that allocation. + +In this example a new exclusive resource group will be created on a L2 CAT +system with two L2 cache instances that can be configured with an 8-bit +capacity bitmask. The new exclusive resource group will be configured to use +25% of each cache instance. + +# mount -t resctrl resctrl /sys/fs/resctrl/ +# cd /sys/fs/resctrl + +First, we observe that the default group is configured to allocate to all L2 +cache: + +# cat schemata +L2:0=ff;1=ff + +We could attempt to create the new resource group at this point, but it will +fail because of the overlap with the schemata of the default group: +# mkdir p0 +# echo 'L2:0=0x3;1=0x3' > p0/schemata +# cat p0/mode +shareable +# echo exclusive > p0/mode +-sh: echo: write error: Invalid argument +# cat info/last_cmd_status +schemata overlaps + +To ensure that there is no overlap with another resource group the default +resource group's schemata has to change, making it possible for the new +resource group to become exclusive. +# echo 'L2:0=0xfc;1=0xfc' > schemata +# echo exclusive > p0/mode +# grep . p0/* +p0/cpus:0 +p0/mode:exclusive +p0/schemata:L2:0=03;1=03 +p0/size:L2:0=262144;1=262144 + +A new resource group will on creation not overlap with an exclusive resource +group: +# mkdir p1 +# grep . p1/* +p1/cpus:0 +p1/mode:shareable +p1/schemata:L2:0=fc;1=fc +p1/size:L2:0=786432;1=786432 + +The bit_usage will reflect how the cache is used: +# cat info/L2/bit_usage +0=SSSSSSEE;1=SSSSSSEE + +A resource group cannot be forced to overlap with an exclusive resource group: +# echo 'L2:0=0x1;1=0x1' > p1/schemata +-sh: echo: write error: Invalid argument +# cat info/last_cmd_status +overlaps with exclusive group + +Locking between applications +---------------------------- Certain operations on the resctrl filesystem, composed of read/writes to/from multiple files, must be atomic.
@@ -510,7 +605,7 @@ to/from multiple files, must be atomic. As an example, the allocation of an exclusive reservation of L3 cache involves: - 1. Read the cbmmasks from each directory + 1. Read the cbmmasks from each directory or the per-resource "bit_usage" 2. Find a contiguous set of bits in the global CBM bitmask that is clear in any of the directory cbmmasks 3. Create a new directory
PATCH v7 3/41 x86/intel_rdt: Introduce RDT resource group mode
In reply to Reinette ChatreAt this time there are no constraints on how bitmasks represented by schemata can be associated with closids represented by resource groups. A bitmask of one class of service can without any objections overlap with the bitmask of another class of service.
The concept of "mode" is introduced in preparation for support of control over whether cache regions can be shared between classes of service. At this time the only mode reflects the current cache allocations where all can potentially be shared.
--- arch/x86/kernel/cpu/intel_rdt.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +16 -0
@@ -80,6 +80,22 @@ enum rdt_group_type { RDT_NUM_GROUP, }; +/** +* enum rdtgrp_mode - Mode of a RDT resource group +* @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations +* +* The mode of a resource group enables control over the allowed overlap +* between allocations associated with different resource groups (classes +* of service). User is able to modify the mode of a resource group by +* writing to the "mode" resctrl file associated with the resource group. +*/ +enum rdtgrp_mode { +RDT_MODE_SHAREABLE =0 , + +/* Must be last */ +RDT_NUM_MODES , +}; +/** * struct mongroup - store mon group's data in resctrl fs. * @mon_data_kn kernlfs node for the mon_data directory
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce RDT resource group mode
x86/intel_rdt: Introduce RDT resource group mode
Commit-ID: eb956a636f90bd0297309a7eae6db388ae5bfc43 Gitweb: https://git.kernel.org/tip/eb956a636f90bd0297309a7eae6db388ae5bfc43 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:54 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:40 +0200
x86/intel_rdt: Introduce RDT resource group mode
At this time there are no constraints on how bitmasks represented by schemata can be associated with closids represented by resource groups. A bitmask of one class of service can without any objections overlap with the bitmask of another class of service.
The concept of "mode" is introduced in preparation for support of control over whether cache regions can be shared between classes of service. At this time the only mode reflects the current cache allocations where all can potentially be shared.
--- arch/x86/kernel/cpu/intel_rdt.h | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +16 -0
@@ -80,6 +80,22 @@ enum rdt_group_type { RDT_NUM_GROUP, }; +/** +* enum rdtgrp_mode - Mode of a RDT resource group +* @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations +* +* The mode of a resource group enables control over the allowed overlap +* between allocations associated with different resource groups (classes +* of service). User is able to modify the mode of a resource group by +* writing to the "mode" resctrl file associated with the resource group. +*/ +enum rdtgrp_mode { +RDT_MODE_SHAREABLE =0 , + +/* Must be last */ +RDT_NUM_MODES , +}; +/** * struct mongroup - store mon group's data in resctrl fs. * @mon_data_kn kernlfs node for the mon_data directory
PATCH v7 4/41 x86/intel_rdt: Associate mode with each RDT resource group
In reply to Reinette ChatreEach RDT resource group is associated with a mode that will reflect the level of sharing of its allocations. The default, shareable, will be associated with each resource group on creation since it is zero and resource groups are created with kzalloc. The managing of the mode of a resource group will follow. The default resource group always remain though so ensure that it is reset to the default mode when the resctrl filesystem is unmounted.
Also introduce a utility that can be used to determine the mode of a resource group when it is searched for based on its class of service.
--- arch/x86/kernel/cpu/intel_rdt.h | 3 +++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 22 ++++++++++++++++++++++ 2 files changed, 25 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +3 -0
@@ -122,6 +122,7 @@ struct mongroup {* @type: indicates type of this rdtgroup - either * monitor only or ctrl_mon group * @mon: mongroup related data +* @mode: mode of resource group */ struct rdtgroup {struct kernfs_node *kn;
@@ -132,6 +133,7 @@ struct rdtgroup {atomic_t waitcount;enum rdt_group_type type;struct mongroup mon; +enum rdtgrp_mode mode; };/* rdtgroup.flags */
@@ -461,6 +463,7 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off);int rdtgroup_schemata_show (struct kernfs_open_file *of,struct seq_file *s,void *v); +enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int alloc_rmid (void );void free_rmid (u32 rmid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +22 -0
@@ -126,6 +126,27 @@ static void closid_free(int closid) closid_free_map |=1 << closid; } +/** +* rdtgroup_mode_by_closid - Return mode of resource group with closid +* @closid: closid if the resource group +* +* Each resource group is associated with a @closid. Here the mode +* of a resource group can be queried by searching for it using its closid. +* +* Return: mode as &enum rdtgrp_mode of resource group with closid @closid +*/ +enum rdtgrp_mode rdtgroup_mode_by_closid (int closid) +{ +struct rdtgroup *rdtgrp; + +list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { +if (rdtgrp->closid == closid) +return rdtgrp->mode; + } + +return RDT_NUM_MODES ; +} +/* set uid and gid of rdtgroup dirs and files to that of the creator */ static int rdtgroup_kn_set_ugid (struct kernfs_node *kn) {
@@ -1500,6 +1521,7 @@ static void rdt_kill_sb(struct super_block *sb)reset_all_ctrls (r);cdp_disable_all ();rmdir_all_sub (); + rdtgroup_default.mode =RDT_MODE_SHAREABLE ;static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);static_branch_disable_cpuslocked (&rdt_enable_key);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Associate mode with each RDT resource group
x86/intel_rdt: Associate mode with each RDT resource group
Commit-ID: 472ef09b40c5802a2e014ae2503ad8619486f9e6 Gitweb: https://git.kernel.org/tip/472ef09b40c5802a2e014ae2503ad8619486f9e6 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:55 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:41 +0200
x86/intel_rdt: Associate mode with each RDT resource group
Each RDT resource group is associated with a mode that will reflect the level of sharing of its allocations. The default, shareable, will be associated with each resource group on creation since it is zero and resource groups are created with kzalloc. The managing of the mode of a resource group will follow. The default resource group always remain though so ensure that it is reset to the default mode when the resctrl filesystem is unmounted.
Also introduce a utility that can be used to determine the mode of a resource group when it is searched for based on its class of service.
--- arch/x86/kernel/cpu/intel_rdt.h | 3 +++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 22 ++++++++++++++++++++++ 2 files changed, 25 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +3 -0
@@ -122,6 +122,7 @@ struct mongroup {* @type: indicates type of this rdtgroup - either * monitor only or ctrl_mon group * @mon: mongroup related data +* @mode: mode of resource group */ struct rdtgroup {struct kernfs_node *kn;
@@ -132,6 +133,7 @@ struct rdtgroup {atomic_t waitcount;enum rdt_group_type type;struct mongroup mon; +enum rdtgrp_mode mode; };/* rdtgroup.flags */
@@ -461,6 +463,7 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off);int rdtgroup_schemata_show (struct kernfs_open_file *of,struct seq_file *s,void *v); +enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int alloc_rmid (void );void free_rmid (u32 rmid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +22 -0
@@ -126,6 +126,27 @@ static void closid_free(int closid) closid_free_map |=1 << closid; } +/** +* rdtgroup_mode_by_closid - Return mode of resource group with closid +* @closid: closid if the resource group +* +* Each resource group is associated with a @closid. Here the mode +* of a resource group can be queried by searching for it using its closid. +* +* Return: mode as &enum rdtgrp_mode of resource group with closid @closid +*/ +enum rdtgrp_mode rdtgroup_mode_by_closid (int closid) +{ +struct rdtgroup *rdtgrp; + +list_for_each_entry (rdtgrp, &rdt_all_groups, rdtgroup_list) { +if (rdtgrp->closid == closid) +return rdtgrp->mode; + } + +return RDT_NUM_MODES ; +} +/* set uid and gid of rdtgroup dirs and files to that of the creator */ static int rdtgroup_kn_set_ugid (struct kernfs_node *kn) {
@@ -1491,6 +1512,7 @@ static void rdt_kill_sb(struct super_block *sb)reset_all_ctrls (r);cdp_disable_all ();rmdir_all_sub (); + rdtgroup_default.mode =RDT_MODE_SHAREABLE ;static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);static_branch_disable_cpuslocked (&rdt_enable_key);
PATCH v7 5/41 x86/intel_rdt: Introduce resource group's mode resctrl file
In reply to Reinette ChatreA new resctrl file "mode" associated with each resource group is introduced. This file will display the resource group's current mode and an administrator can also use it to modify the resource group's mode.
Only shareable mode is currently supported.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 83 ++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +83 -0
@@ -147,6 +147,24 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)return RDT_NUM_MODES ; } +static const char *const rdt_mode_str[] = { + [RDT_MODE_SHAREABLE ] ="shareable" , +}; + +/** +* rdtgroup_mode_str - Return the string representation of mode +* @mode: the resource group mode as &enum rdtgroup_mode +* +* Return: string representation of valid mode, "unknown" otherwise +*/ +static const char *rdtgroup_mode_str (enum rdtgrp_mode mode) +{ +if (mode <RDT_MODE_SHAREABLE || mode >=RDT_NUM_MODES ) +return "unknown" ; + +return rdt_mode_str[mode]; +} +/* set uid and gid of rdtgroup dirs and files to that of the creator */ static int rdtgroup_kn_set_ugid (struct kernfs_node *kn) {
@@ -761,6 +779,63 @@ static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,return nbytes; } +/* +* rdtgroup_mode_show - Display mode of this resource group +*/ +static int rdtgroup_mode_show (struct kernfs_open_file *of, +struct seq_file *s,void *v) +{ +struct rdtgroup *rdtgrp; + + rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + } + +seq_printf (s,"%s\n" ,rdtgroup_mode_str (rdtgrp->mode)); + +rdtgroup_kn_unlock (of->kn); +return 0 ; +} + +static ssize_t rdtgroup_mode_write (struct kernfs_open_file *of, +char *buf,size_t nbytes,loff_t off) +{ +struct rdtgroup *rdtgrp; +enum rdtgrp_mode mode; +int ret =0 ; + +/* Valid input requires a trailing newline */ +if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) +return -EINVAL ; + buf[nbytes -1 ] ='\0' ; + + rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + } + +rdt_last_cmd_clear (); + + mode = rdtgrp->mode; + +if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE )) + goto out; + +if (!strcmp (buf,"shareable" )) { + rdtgrp->mode =RDT_MODE_SHAREABLE ; + }else { +rdt_last_cmd_printf ("unknown/unsupported mode\n" ); + ret = -EINVAL ; + } + +out: +rdtgroup_kn_unlock (of->kn); +return ret ?: nbytes; +} +/* rdtgroup information files for one cache resource. */ static struct rftype res_common_files[] = { {
@@ -874,6 +949,14 @@ static struct rftype res_common_files[] = { .seq_show = rdtgroup_schemata_show, .fflags = RF_CTRL_BASE, }, + { + .name = "mode", + .mode = 0644, + .kf_ops = &rdtgroup_kf_single_ops, + .write = rdtgroup_mode_write, + .seq_show = rdtgroup_mode_show, + .fflags = RF_CTRL_BASE, + }, };static int rdtgroup_add_files (struct kernfs_node *kn,unsigned long fflags)
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce resource group's mode resctrl file
x86/intel_rdt: Introduce resource group's mode resctrl file
Commit-ID: d48d7a57f7181b36be748ad8fb806e8de3104e89 Gitweb: https://git.kernel.org/tip/d48d7a57f7181b36be748ad8fb806e8de3104e89 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:56 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:41 +0200
x86/intel_rdt: Introduce resource group's mode resctrl file
A new resctrl file "mode" associated with each resource group is introduced. This file will display the resource group's current mode and an administrator can also use it to modify the resource group's mode.
Only shareable mode is currently supported.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 83 ++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +83 -0
@@ -147,6 +147,24 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)return RDT_NUM_MODES ; } +static const char *const rdt_mode_str[] = { + [RDT_MODE_SHAREABLE ] ="shareable" , +}; + +/** +* rdtgroup_mode_str - Return the string representation of mode +* @mode: the resource group mode as &enum rdtgroup_mode +* +* Return: string representation of valid mode, "unknown" otherwise +*/ +static const char *rdtgroup_mode_str (enum rdtgrp_mode mode) +{ +if (mode <RDT_MODE_SHAREABLE || mode >=RDT_NUM_MODES ) +return "unknown" ; + +return rdt_mode_str[mode]; +} +/* set uid and gid of rdtgroup dirs and files to that of the creator */ static int rdtgroup_kn_set_ugid (struct kernfs_node *kn) {
@@ -761,6 +779,63 @@ static ssize_t max_threshold_occ_write(struct kernfs_open_file *of,return nbytes; } +/* +* rdtgroup_mode_show - Display mode of this resource group +*/ +static int rdtgroup_mode_show (struct kernfs_open_file *of, +struct seq_file *s,void *v) +{ +struct rdtgroup *rdtgrp; + + rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + } + +seq_printf (s,"%s\n" ,rdtgroup_mode_str (rdtgrp->mode)); + +rdtgroup_kn_unlock (of->kn); +return 0 ; +} + +static ssize_t rdtgroup_mode_write (struct kernfs_open_file *of, +char *buf,size_t nbytes,loff_t off) +{ +struct rdtgroup *rdtgrp; +enum rdtgrp_mode mode; +int ret =0 ; + +/* Valid input requires a trailing newline */ +if (nbytes ==0 || buf[nbytes -1 ] !='\n' ) +return -EINVAL ; + buf[nbytes -1 ] ='\0' ; + + rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + } + +rdt_last_cmd_clear (); + + mode = rdtgrp->mode; + +if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE )) + goto out; + +if (!strcmp (buf,"shareable" )) { + rdtgrp->mode =RDT_MODE_SHAREABLE ; + }else { +rdt_last_cmd_printf ("unknown/unsupported mode\n" ); + ret = -EINVAL ; + } + +out: +rdtgroup_kn_unlock (of->kn); +return ret ?: nbytes; +} +/* rdtgroup information files for one cache resource. */ static struct rftype res_common_files[] = { {
@@ -874,6 +949,14 @@ static struct rftype res_common_files[] = { .seq_show = rdtgroup_schemata_show, .fflags = RF_CTRL_BASE, }, + { + .name = "mode", + .mode = 0644, + .kf_ops = &rdtgroup_kf_single_ops, + .write = rdtgroup_mode_write, + .seq_show = rdtgroup_mode_show, + .fflags = RF_CTRL_BASE, + }, };static int rdtgroup_add_files (struct kernfs_node *kn,unsigned long fflags)
PATCH v7 6/41 x86/intel_rdt: Introduce test to determine if closid is in use
In reply to Reinette ChatreDuring CAT feature discovery the capacity bitmasks (CBMs) associated with all the classes of service are initialized to all ones, even if the class of service is not in use. Introduce a test that can be used to determine if a class of service is in use. This test enables code interested in parsing the CBMs to know if its values are meaningful or can be ignored.
Temporarily mark the function as unused to silence compile warnings until it is used.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +12 -0
@@ -126,6 +126,18 @@ static void closid_free(int closid) closid_free_map |=1 << closid; } +/** +* closid_allocated - test if provided closid is in use +* @closid: closid to be tested +* +* Return: true if @closid is currently associated with a resource group, +* false if @closid is free +*/ +static bool __attribute__ ((unused))closid_allocated (unsigned int closid) +{ +return (closid_free_map & (1 << closid)) ==0 ; +} +/** * rdtgroup_mode_by_closid - Return mode of resource group with closid * @closid: closid if the resource group
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce test to determine if closid is in use
x86/intel_rdt: Introduce test to determine if closid is in use
Commit-ID: 0b9aa6562650f42da23c29814adc2318c135cef4 Gitweb: https://git.kernel.org/tip/0b9aa6562650f42da23c29814adc2318c135cef4 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:57 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:41 +0200
x86/intel_rdt: Introduce test to determine if closid is in use
During CAT feature discovery the capacity bitmasks (CBMs) associated with all the classes of service are initialized to all ones, even if the class of service is not in use. Introduce a test that can be used to determine if a class of service is in use. This test enables code interested in parsing the CBMs to know if its values are meaningful or can be ignored.
Temporarily mark the function as unused to silence compile warnings until it is used.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +12 -0
@@ -126,6 +126,18 @@ static void closid_free(int closid) closid_free_map |=1 << closid; } +/** +* closid_allocated - test if provided closid is in use +* @closid: closid to be tested +* +* Return: true if @closid is currently associated with a resource group, +* false if @closid is free +*/ +static bool __attribute__ ((unused))closid_allocated (unsigned int closid) +{ +return (closid_free_map & (1 << closid)) ==0 ; +} +/** * rdtgroup_mode_by_closid - Return mode of resource group with closid * @closid: closid if the resource group
PATCH v7 7/41 x86/intel_rdt: Make useful functions available internally
In reply to Reinette ChatreIn support of the work done to enable resource groups to have different modes some static functions need to be available for sharing amongst all RDT components.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 2 +- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -465,6 +465,8 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, struct seq_file *s, void *v);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r); +int update_domains (struct rdt_resource *r,int closid); +void closid_free (int closid);int alloc_rmid (void );void free_rmid (u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +1 -1
@@ -175,7 +175,7 @@ static int parse_line(char *line, struct rdt_resource *r)return -EINVAL ; } -static int update_domains (struct rdt_resource *r,int closid) +int update_domains (struct rdt_resource *r,int closid) {struct msr_param msr_param;cpumask_var_t cpu_mask;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -1
@@ -121,7 +121,7 @@ static int closid_alloc(void)return closid; } -static void closid_free (int closid) +void closid_free (int closid) { closid_free_map |=1 << closid; }
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Make useful functions available internally
x86/intel_rdt: Make useful functions available internally
Commit-ID: 024d15be3855044faa8bddf829e3613961fd27ba Gitweb: https://git.kernel.org/tip/024d15be3855044faa8bddf829e3613961fd27ba Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:58 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:42 +0200
x86/intel_rdt: Make useful functions available internally
In support of the work done to enable resource groups to have different modes some static functions need to be available for sharing amongst all RDT components.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 2 +- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -465,6 +465,8 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, struct seq_file *s, void *v);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r); +int update_domains (struct rdt_resource *r,int closid); +void closid_free (int closid);int alloc_rmid (void );void free_rmid (u32 rmid);int rdt_get_mon_l3_config (struct rdt_resource *r);
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +1 -1
@@ -175,7 +175,7 @@ next:return -EINVAL ; } -static int update_domains (struct rdt_resource *r,int closid) +int update_domains (struct rdt_resource *r,int closid) {struct msr_param msr_param;cpumask_var_t cpu_mask;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -1
@@ -121,7 +121,7 @@ static int closid_alloc(void)return closid; } -static void closid_free (int closid) +void closid_free (int closid) { closid_free_map |=1 << closid; }
PATCH v7 8/41 x86/intel_rdt: Initialize new resource group with sane defaults
In reply to Reinette ChatreCurrently when a new resource group is created its allocations would be those that belonged to the resource group to which its closid belonged previously.
That is, we can encounter a case like:
mkdir newgroup
cat newgroup/schemata
L2:0=ff;1=ff
echo 'L2:0=0xf0;1=0xf0' > newgroup/schemata
cat newgroup/schemata
L2:0=0xf0;1=0xf0
rmdir newgroup
mkdir newnewgroup
cat newnewgroup/schemata
L2:0=0xf0;1=0xf0
When the new group is created it would be reasonable to expect its allocations to be initialized with all regions that it can possibly use. At this time these regions would be all that are shareable by other resource groups as well as regions that are not currently used. If the available cache region is found to be non-contiguous the available region is adjusted to enforce validity.
When a new resource group is created the hardware is initialized with these new default allocations.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 115 ++++++++++++++++++++++- 1 file changed, 112 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +112 -3
@@ -133,7 +133,7 @@ void closid_free(int closid)* Return: true if @closid is currently associated with a resource group, * false if @closid is free */ -static bool __attribute__ ((unused))closid_allocated (unsigned int closid) +static bool closid_allocated (unsigned int closid) {return (closid_free_map & (1 << closid)) ==0 ; }
@@ -1816,6 +1816,110 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,return ret; } +/** +* cbm_ensure_valid - Enforce validity on provided CBM +* @_val: Candidate CBM +* @r: RDT resource to which the CBM belongs +* +* The provided CBM represents all cache portions available for use. This +* may be represented by a bitmap that does not consist of contiguous ones +* and thus be an invalid CBM. +* Here the provided CBM is forced to be a valid CBM by only considering +* the first set of contiguous bits as valid and clearing all bits. +* The intention here is to provide a valid default CBM with which a new +* resource group is initialized. The user can follow this with a +* modification to the CBM if the default does not satisfy the +* requirements. +*/ +static void cbm_ensure_valid (u32 *_val,struct rdt_resource *r) +{ +/* +* Convert the u32 _val to an unsigned long required by all the bit +* operations within this function. No more than 32 bits of this +* converted value can be accessed because all bit operations are +* additionally provided with cbm_len that is initialized during +* hardware enumeration using five bits from the EAX register and +* thus never can exceed 32 bits. +*/ +unsigned long *val = (unsigned long *)_val; +unsigned int cbm_len = r->cache.cbm_len; +unsigned long first_bit, zero_bit; + +if (*val ==0 ) +return ; + + first_bit =find_first_bit (val, cbm_len); + zero_bit =find_next_zero_bit (val, cbm_len, first_bit); + +/* Clear any remaining bits to ensure contiguous region */ +bitmap_clear (val, zero_bit, cbm_len - zero_bit); +} + +/** +* rdtgroup_init_alloc - Initialize the new RDT group's allocations +* +* A new RDT group is being created on an allocation capable (CAT) +* supporting system. Set this group up to start off with all usable +* allocations. That is, all shareable and unused bits. +* +* All-zero CBM is invalid. If there are no more shareable bits available +* on any domain then the entire allocation will fail. +*/ +static int rdtgroup_init_alloc (struct rdtgroup *rdtgrp) +{ +u32 used_b =0 , unused_b =0 ; +u32 closid = rdtgrp->closid; +struct rdt_resource *r; +enum rdtgrp_mode mode; +struct rdt_domain *d; +int i, ret; +u32 *ctrl; + +for_each_alloc_enabled_rdt_resource (r) { +list_for_each_entry (d, &r->domains, list) { + d->have_new_ctrl = false; + d->new_ctrl = r->cache.shareable_bits; + used_b = r->cache.shareable_bits; + ctrl = d->ctrl_val; +for (i =0 ; i < r->num_closid; i++, ctrl++) { +if (closid_allocated (i) && i != closid) { + mode =rdtgroup_mode_by_closid (i); + used_b |= *ctrl; +if (mode ==RDT_MODE_SHAREABLE ) + d->new_ctrl |= *ctrl; + } + } + unused_b = used_b ^ (BIT_MASK (r->cache.cbm_len) -1 ); + unused_b &=BIT_MASK (r->cache.cbm_len) -1 ; + d->new_ctrl |= unused_b; +/* +* Force the initial CBM to be valid, user can +* modify the CBM based on system availability. +*/ +cbm_ensure_valid (&d->new_ctrl, r); +if (bitmap_weight ((unsigned long *) &d->new_ctrl, + r->cache.cbm_len) < + r->cache.min_cbm_bits) { +rdt_last_cmd_printf ("no space on %s:%d\n" , + r->name, d->id); +return -ENOSPC ; + } + d->have_new_ctrl = true; + } + } + +for_each_alloc_enabled_rdt_resource (r) { + ret =update_domains (r, rdtgrp->closid); +if (ret <0 ) { +rdt_last_cmd_puts ("failed to initialize allocations\n" ); +return ret; + } + rdtgrp->mode =RDT_MODE_SHAREABLE ; + } + +return 0 ; +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, struct kernfs_node *prgrp_kn, const char *name, umode_t mode,
@@ -1974,6 +2078,10 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =0 ; rdtgrp->closid = closid; + ret =rdtgroup_init_alloc (rdtgrp); +if (ret <0 ) + goto out_id_free; +list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups);if (rdt_mon_capable) {
@@ -1984,15 +2092,16 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =mongroup_create_dir (kn,NULL ,"mon_groups" ,NULL );if (ret) {rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_id_free; + goto out_del_list; } } goto out_unlock; +out_del_list: +list_del (&rdtgrp->rdtgroup_list); out_id_free:closid_free (closid); -list_del (&rdtgrp->rdtgroup_list); out_common_fail:mkdir_rdt_prepare_clean (rdtgrp); out_unlock:
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Initialize new resource group with sane defaults
x86/intel_rdt: Initialize new resource group with sane defaults
Commit-ID: 95f0b77efa5749f19e7acfedcb8521da4b13ed0e Gitweb: https://git.kernel.org/tip/95f0b77efa5749f19e7acfedcb8521da4b13ed0e Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:41:59 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:42 +0200
x86/intel_rdt: Initialize new resource group with sane defaults
Currently when a new resource group is created its allocations would be those that belonged to the resource group to which its closid belonged previously.
That is, we can encounter a case like:
mkdir newgroup
cat newgroup/schemata
L2:0=ff;1=ff
echo 'L2:0=0xf0;1=0xf0' > newgroup/schemata
cat newgroup/schemata
L2:0=0xf0;1=0xf0
rmdir newgroup
mkdir newnewgroup
cat newnewgroup/schemata
L2:0=0xf0;1=0xf0
When the new group is created it would be reasonable to expect its allocations to be initialized with all regions that it can possibly use. At this time these regions would be all that are shareable by other resource groups as well as regions that are not currently used. If the available cache region is found to be non-contiguous the available region is adjusted to enforce validity.
When a new resource group is created the hardware is initialized with these new default allocations.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 115 ++++++++++++++++++++++++++++++- 1 file changed, 112 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +112 -3
@@ -133,7 +133,7 @@ void closid_free(int closid)* Return: true if @closid is currently associated with a resource group, * false if @closid is free */ -static bool __attribute__ ((unused))closid_allocated (unsigned int closid) +static bool closid_allocated (unsigned int closid) {return (closid_free_map & (1 << closid)) ==0 ; }
@@ -1807,6 +1807,110 @@ out_destroy:return ret; } +/** +* cbm_ensure_valid - Enforce validity on provided CBM +* @_val: Candidate CBM +* @r: RDT resource to which the CBM belongs +* +* The provided CBM represents all cache portions available for use. This +* may be represented by a bitmap that does not consist of contiguous ones +* and thus be an invalid CBM. +* Here the provided CBM is forced to be a valid CBM by only considering +* the first set of contiguous bits as valid and clearing all bits. +* The intention here is to provide a valid default CBM with which a new +* resource group is initialized. The user can follow this with a +* modification to the CBM if the default does not satisfy the +* requirements. +*/ +static void cbm_ensure_valid (u32 *_val,struct rdt_resource *r) +{ +/* +* Convert the u32 _val to an unsigned long required by all the bit +* operations within this function. No more than 32 bits of this +* converted value can be accessed because all bit operations are +* additionally provided with cbm_len that is initialized during +* hardware enumeration using five bits from the EAX register and +* thus never can exceed 32 bits. +*/ +unsigned long *val = (unsigned long *)_val; +unsigned int cbm_len = r->cache.cbm_len; +unsigned long first_bit, zero_bit; + +if (*val ==0 ) +return ; + + first_bit =find_first_bit (val, cbm_len); + zero_bit =find_next_zero_bit (val, cbm_len, first_bit); + +/* Clear any remaining bits to ensure contiguous region */ +bitmap_clear (val, zero_bit, cbm_len - zero_bit); +} + +/** +* rdtgroup_init_alloc - Initialize the new RDT group's allocations +* +* A new RDT group is being created on an allocation capable (CAT) +* supporting system. Set this group up to start off with all usable +* allocations. That is, all shareable and unused bits. +* +* All-zero CBM is invalid. If there are no more shareable bits available +* on any domain then the entire allocation will fail. +*/ +static int rdtgroup_init_alloc (struct rdtgroup *rdtgrp) +{ +u32 used_b =0 , unused_b =0 ; +u32 closid = rdtgrp->closid; +struct rdt_resource *r; +enum rdtgrp_mode mode; +struct rdt_domain *d; +int i, ret; +u32 *ctrl; + +for_each_alloc_enabled_rdt_resource (r) { +list_for_each_entry (d, &r->domains, list) { + d->have_new_ctrl = false; + d->new_ctrl = r->cache.shareable_bits; + used_b = r->cache.shareable_bits; + ctrl = d->ctrl_val; +for (i =0 ; i < r->num_closid; i++, ctrl++) { +if (closid_allocated (i) && i != closid) { + mode =rdtgroup_mode_by_closid (i); + used_b |= *ctrl; +if (mode ==RDT_MODE_SHAREABLE ) + d->new_ctrl |= *ctrl; + } + } + unused_b = used_b ^ (BIT_MASK (r->cache.cbm_len) -1 ); + unused_b &=BIT_MASK (r->cache.cbm_len) -1 ; + d->new_ctrl |= unused_b; +/* +* Force the initial CBM to be valid, user can +* modify the CBM based on system availability. +*/ +cbm_ensure_valid (&d->new_ctrl, r); +if (bitmap_weight ((unsigned long *) &d->new_ctrl, + r->cache.cbm_len) < + r->cache.min_cbm_bits) { +rdt_last_cmd_printf ("no space on %s:%d\n" , + r->name, d->id); +return -ENOSPC ; + } + d->have_new_ctrl = true; + } + } + +for_each_alloc_enabled_rdt_resource (r) { + ret =update_domains (r, rdtgrp->closid); +if (ret <0 ) { +rdt_last_cmd_puts ("failed to initialize allocations\n" ); +return ret; + } + rdtgrp->mode =RDT_MODE_SHAREABLE ; + } + +return 0 ; +} + static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, struct kernfs_node *prgrp_kn, const char *name, umode_t mode,
@@ -1965,6 +2069,10 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =0 ; rdtgrp->closid = closid; + ret =rdtgroup_init_alloc (rdtgrp); +if (ret <0 ) + goto out_id_free; +list_add (&rdtgrp->rdtgroup_list, &rdt_all_groups);if (rdt_mon_capable) {
@@ -1975,15 +2083,16 @@ static int rdtgroup_mkdir_ctrl_mon(struct kernfs_node *parent_kn, ret =mongroup_create_dir (kn,NULL ,"mon_groups" ,NULL );if (ret) {rdt_last_cmd_puts ("kernfs subdir error\n" ); - goto out_id_free; + goto out_del_list; } } goto out_unlock; +out_del_list: +list_del (&rdtgrp->rdtgroup_list); out_id_free:closid_free (closid); -list_del (&rdtgrp->rdtgroup_list); out_common_fail:mkdir_rdt_prepare_clean (rdtgrp); out_unlock:
PATCH v7 9/41 x86/intel_rdt: Introduce new "exclusive" mode
In reply to Reinette ChatreAt the moment all allocations are shareable. There is no way for a user to designate that an allocation associated with a resource group cannot be shared by another.
Introduce the new mode "exclusive". When a resource group is marked as such it implies that no overlap is allowed between its allocation and that of another resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 1 + 2 files changed, 3 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -83,6 +83,7 @@ enum rdt_group_type {/** * enum rdtgrp_mode - Mode of a RDT resource group * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations +* @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed * * The mode of a resource group enables control over the allowed overlap * between allocations associated with different resource groups (classes
@@ -91,6 +92,7 @@ enum rdt_group_type {*/ enum rdtgrp_mode {RDT_MODE_SHAREABLE =0 , +RDT_MODE_EXCLUSIVE ,/* Must be last */ RDT_NUM_MODES ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -0
@@ -161,6 +161,7 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)static const char *const rdt_mode_str[] = { [RDT_MODE_SHAREABLE ] ="shareable" , + [RDT_MODE_EXCLUSIVE ] ="exclusive" , };/**
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce new "exclusive" mode
x86/intel_rdt: Introduce new "exclusive" mode
Commit-ID: 414dd2b4732949ddc972c2592a13c799434249c6 Gitweb: https://git.kernel.org/tip/414dd2b4732949ddc972c2592a13c799434249c6 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:00 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:42 +0200
x86/intel_rdt: Introduce new "exclusive" mode
At the moment all allocations are shareable. There is no way for a user to designate that an allocation associated with a resource group cannot be shared by another.
Introduce the new mode "exclusive". When a resource group is marked as such it implies that no overlap is allowed between its allocation and that of another resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 1 + 2 files changed, 3 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -83,6 +83,7 @@ enum rdt_group_type {/** * enum rdtgrp_mode - Mode of a RDT resource group * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations +* @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed * * The mode of a resource group enables control over the allowed overlap * between allocations associated with different resource groups (classes
@@ -91,6 +92,7 @@ enum rdt_group_type {*/ enum rdtgrp_mode {RDT_MODE_SHAREABLE =0 , +RDT_MODE_EXCLUSIVE ,/* Must be last */ RDT_NUM_MODES ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -0
@@ -161,6 +161,7 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid)static const char *const rdt_mode_str[] = { [RDT_MODE_SHAREABLE ] ="shareable" , + [RDT_MODE_EXCLUSIVE ] ="exclusive" , };/**
PATCH v7 10/41 x86/intel_rdt: Enable setting of exclusive mode
In reply to Reinette ChatreThe new "mode" file now accepts "exclusive" that means that the allocations of this resource group cannot be shared.
Enable users to modify a resource group's mode to "exclusive". To succeed it is required that there is no overlap between resource group's current schemata and that of all the other active resource groups as well as cache regions potentially used by other hardware entities.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 97 +++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +96 -1
@@ -812,6 +812,93 @@ static int rdtgroup_mode_show(struct kernfs_open_file *of,return 0 ; } +/** +* rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other +* @r: Resource to which domain instance @d belongs. +* @d: The domain instance for which @closid is being tested. +* @cbm: Capacity bitmask being tested. +* @closid: Intended closid for @cbm. +* @exclusive: Only check if overlaps with exclusive resource groups +* +* Checks if provided @cbm intended to be used for @closid on domain +* @d overlaps with any other closids or other hardware usage associated +* with this domain. If @exclusive is true then only overlaps with +* resource groups in exclusive mode will be considered. If @exclusive +* is false then overlaps with any resource group or hardware entities +* will be considered. +* +* Return: false if CBM does not overlap, true if it does. +*/ +static bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive) +{ +unsigned long *cbm = (unsigned long *)&_cbm; +unsigned long *ctrl_b; +enum rdtgrp_mode mode; +u32 *ctrl; +int i; + +/* Check for any overlap with regions used by hardware directly */ +if (!exclusive) { +if (bitmap_intersects (cbm, + (unsigned long *)&r->cache.shareable_bits, + r->cache.cbm_len)) +return true; + } + +/* Check for overlap with other resource groups */ + ctrl = d->ctrl_val; +for (i =0 ; i < r->num_closid; i++, ctrl++) { + ctrl_b = (unsigned long *)ctrl; +if (closid_allocated (i) && i != closid) { +if (bitmap_intersects (cbm, ctrl_b, r->cache.cbm_len)) { + mode =rdtgroup_mode_by_closid (i); +if (exclusive) { +if (mode ==RDT_MODE_EXCLUSIVE ) +return true; +continue ; + } +return true; + } + } + } + +return false; +} + +/** +* rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive +* +* An exclusive resource group implies that there should be no sharing of +* its allocated resources. At the time this group is considered to be +* exclusive this test can determine if its current schemata supports this +* setting by testing for overlap with all other resource groups. +* +* Return: true if resource group can be exclusive, false if there is overlap +* with allocations of other resource groups and thus this resource group +* cannot be exclusive. +*/ +static bool rdtgroup_mode_test_exclusive (struct rdtgroup *rdtgrp) +{ +int closid = rdtgrp->closid; +struct rdt_resource *r; +struct rdt_domain *d; + +for_each_alloc_enabled_rdt_resource (r) { +list_for_each_entry (d, &r->domains, list) { +if (rdtgroup_cbm_overlaps (r, d, d->ctrl_val[closid], + rdtgrp->closid, false)) +return false; + } + } + +return true; +} + +/** +* rdtgroup_mode_write - Modify the resource group's mode +* +*/ static ssize_t rdtgroup_mode_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off) {
@@ -834,11 +921,19 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of, mode = rdtgrp->mode; -if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE )) +if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE ) || + (!strcmp (buf,"exclusive" ) && mode ==RDT_MODE_EXCLUSIVE )) goto out;if (!strcmp (buf,"shareable" )) { rdtgrp->mode =RDT_MODE_SHAREABLE ; + }else if (!strcmp (buf,"exclusive" )) { +if (!rdtgroup_mode_test_exclusive (rdtgrp)) { +rdt_last_cmd_printf ("schemata overlaps\n" ); + ret = -EINVAL ; + goto out; + } + rdtgrp->mode =RDT_MODE_EXCLUSIVE ; }else {rdt_last_cmd_printf ("unknown/unsupported mode\n" ); ret = -EINVAL ;
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Enable setting of exclusive mode
x86/intel_rdt: Enable setting of exclusive mode
Commit-ID: 49f7b4efa1101bbc143a960eff3a9c8f9d6f7358 Gitweb: https://git.kernel.org/tip/49f7b4efa1101bbc143a960eff3a9c8f9d6f7358 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:01 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:42 +0200
x86/intel_rdt: Enable setting of exclusive mode
The new "mode" file now accepts "exclusive" that means that the allocations of this resource group cannot be shared.
Enable users to modify a resource group's mode to "exclusive". To succeed it is required that there is no overlap between resource group's current schemata and that of all the other active resource groups as well as cache regions potentially used by other hardware entities.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 97 +++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +96 -1
@@ -812,6 +812,93 @@ static int rdtgroup_mode_show(struct kernfs_open_file *of,return 0 ; } +/** +* rdtgroup_cbm_overlaps - Does CBM for intended closid overlap with other +* @r: Resource to which domain instance @d belongs. +* @d: The domain instance for which @closid is being tested. +* @cbm: Capacity bitmask being tested. +* @closid: Intended closid for @cbm. +* @exclusive: Only check if overlaps with exclusive resource groups +* +* Checks if provided @cbm intended to be used for @closid on domain +* @d overlaps with any other closids or other hardware usage associated +* with this domain. If @exclusive is true then only overlaps with +* resource groups in exclusive mode will be considered. If @exclusive +* is false then overlaps with any resource group or hardware entities +* will be considered. +* +* Return: false if CBM does not overlap, true if it does. +*/ +static bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive) +{ +unsigned long *cbm = (unsigned long *)&_cbm; +unsigned long *ctrl_b; +enum rdtgrp_mode mode; +u32 *ctrl; +int i; + +/* Check for any overlap with regions used by hardware directly */ +if (!exclusive) { +if (bitmap_intersects (cbm, + (unsigned long *)&r->cache.shareable_bits, + r->cache.cbm_len)) +return true; + } + +/* Check for overlap with other resource groups */ + ctrl = d->ctrl_val; +for (i =0 ; i < r->num_closid; i++, ctrl++) { + ctrl_b = (unsigned long *)ctrl; +if (closid_allocated (i) && i != closid) { +if (bitmap_intersects (cbm, ctrl_b, r->cache.cbm_len)) { + mode =rdtgroup_mode_by_closid (i); +if (exclusive) { +if (mode ==RDT_MODE_EXCLUSIVE ) +return true; +continue ; + } +return true; + } + } + } + +return false; +} + +/** +* rdtgroup_mode_test_exclusive - Test if this resource group can be exclusive +* +* An exclusive resource group implies that there should be no sharing of +* its allocated resources. At the time this group is considered to be +* exclusive this test can determine if its current schemata supports this +* setting by testing for overlap with all other resource groups. +* +* Return: true if resource group can be exclusive, false if there is overlap +* with allocations of other resource groups and thus this resource group +* cannot be exclusive. +*/ +static bool rdtgroup_mode_test_exclusive (struct rdtgroup *rdtgrp) +{ +int closid = rdtgrp->closid; +struct rdt_resource *r; +struct rdt_domain *d; + +for_each_alloc_enabled_rdt_resource (r) { +list_for_each_entry (d, &r->domains, list) { +if (rdtgroup_cbm_overlaps (r, d, d->ctrl_val[closid], + rdtgrp->closid, false)) +return false; + } + } + +return true; +} + +/** +* rdtgroup_mode_write - Modify the resource group's mode +* +*/ static ssize_t rdtgroup_mode_write (struct kernfs_open_file *of,char *buf,size_t nbytes,loff_t off) {
@@ -834,11 +921,19 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of, mode = rdtgrp->mode; -if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE )) +if ((!strcmp (buf,"shareable" ) && mode ==RDT_MODE_SHAREABLE ) || + (!strcmp (buf,"exclusive" ) && mode ==RDT_MODE_EXCLUSIVE )) goto out;if (!strcmp (buf,"shareable" )) { rdtgrp->mode =RDT_MODE_SHAREABLE ; + }else if (!strcmp (buf,"exclusive" )) { +if (!rdtgroup_mode_test_exclusive (rdtgrp)) { +rdt_last_cmd_printf ("schemata overlaps\n" ); + ret = -EINVAL ; + goto out; + } + rdtgrp->mode =RDT_MODE_EXCLUSIVE ; }else {rdt_last_cmd_printf ("unknown/unsupported mode\n" ); ret = -EINVAL ;
PATCH v7 11/41 x86/intel_rdt: Making CBM name and type more explicit
In reply to Reinette Chatrecbm_validate() receives a pointer to the variable that will be initialized with a validated capacity bitmask. The pointer points to a variable of type unsigned long that is immediately assigned to a variable of type u32 by the caller on return from cbm_validate().
Let cbm_validate() initialize a variable of type u32 directly.
At this time also change tha variable name "data" within parse_cbm() to a name more reflective of the content: "cbm_val". This frees up the generic "data" to be used later when it is indeed used for a collection of input.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +5 -4
@@ -87,7 +87,7 @@ int parse_bw(char *buf, struct rdt_resource *r, struct rdt_domain *d)* are allowed (e.g. FFFFH, 0FF0H, 003CH, etc.). * Additionally Haswell requires at least two bits set. */ -static bool cbm_validate (char *buf,unsigned long *data,struct rdt_resource *r) +static bool cbm_validate (char *buf,u32 *data,struct rdt_resource *r) {unsigned long first_bit, zero_bit, val;unsigned int cbm_len = r->cache.cbm_len;
@@ -128,16 +128,17 @@ static bool cbm_validate(char *buf, unsigned long *data, struct rdt_resource *r)*/ int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d) { -unsigned long data; +u32 cbm_val;if (d->have_new_ctrl) {rdt_last_cmd_printf ("duplicate domain %d\n" , d->id);return -EINVAL ; } -if (!cbm_validate (buf, &data, r)) +if (!cbm_validate (buf, &cbm_val, r))return -EINVAL ; - d->new_ctrl = data; + + d->new_ctrl = cbm_val; d->have_new_ctrl = true;return 0 ;
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Making CBM name and type more explicit
x86/intel_rdt: Making CBM name and type more explicit
Commit-ID: 9af4c0a6dc1a1abf5336f2c3f951444db6b71da9 Gitweb: https://git.kernel.org/tip/9af4c0a6dc1a1abf5336f2c3f951444db6b71da9 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:02 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:43 +0200
x86/intel_rdt: Making CBM name and type more explicit
cbm_validate() receives a pointer to the variable that will be initialized with a validated capacity bitmask. The pointer points to a variable of type unsigned long that is immediately assigned to a variable of type u32 by the caller on return from cbm_validate().
Let cbm_validate() initialize a variable of type u32 directly.
At this time also change tha variable name "data" within parse_cbm() to a name more reflective of the content: "cbm_val". This frees up the generic "data" to be used later when it is indeed used for a collection of input.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +5 -4
@@ -87,7 +87,7 @@ int parse_bw(char *buf, struct rdt_resource *r, struct rdt_domain *d)* are allowed (e.g. FFFFH, 0FF0H, 003CH, etc.). * Additionally Haswell requires at least two bits set. */ -static bool cbm_validate (char *buf,unsigned long *data,struct rdt_resource *r) +static bool cbm_validate (char *buf,u32 *data,struct rdt_resource *r) {unsigned long first_bit, zero_bit, val;unsigned int cbm_len = r->cache.cbm_len;
@@ -128,16 +128,17 @@ static bool cbm_validate(char *buf, unsigned long *data, struct rdt_resource *r)*/ int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d) { -unsigned long data; +u32 cbm_val;if (d->have_new_ctrl) {rdt_last_cmd_printf ("duplicate domain %d\n" , d->id);return -EINVAL ; } -if (!cbm_validate (buf, &data, r)) +if (!cbm_validate (buf, &cbm_val, r))return -EINVAL ; - d->new_ctrl = data; + + d->new_ctrl = cbm_val; d->have_new_ctrl = true;return 0 ;
PATCH v7 12/41 x86/intel_rdt: Support flexible data to parsing callbacks
In reply to Reinette ChatreEach resource is associated with a configurable callback that should be used to parse the information provided for the particular resource from user space. In addition to the resource and domain pointers this callback is provided with just the character buffer being parsed.
In support of flexible parsing the callback is modified to support a void pointer as argument. This enables resources that need more data than just the user provided data to pass its required data to the callback without affecting the signatures for the callbacks of all the other resources.
--- arch/x86/kernel/cpu/intel_rdt.h | 6 +++--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 6 ++++-- 2 files changed, 7 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +3 -3
@@ -371,7 +371,7 @@ struct rdt_resource {struct rdt_cache cache;struct rdt_membw membw;const char *format_str; -int (*parse_ctrlval) (char *buf,struct rdt_resource *r, +int (*parse_ctrlval) (void *data,struct rdt_resource *r,struct rdt_domain *d);struct list_head evt_list;int num_rmid;
@@ -379,8 +379,8 @@ struct rdt_resource {unsigned long fflags; }; -int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d); -int parse_bw (char *buf,struct rdt_resource *r,struct rdt_domain *d); +int parse_cbm (void *_data,struct rdt_resource *r,struct rdt_domain *d); +int parse_bw (void *_buf,struct rdt_resource *r,struct rdt_domain *d);extern struct mutex rdtgroup_mutex;
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +4 -2
@@ -64,9 +64,10 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)return true; } -int parse_bw (char *buf,struct rdt_resource *r,struct rdt_domain *d) +int parse_bw (void *_buf,struct rdt_resource *r,struct rdt_domain *d) {unsigned long data; +char *buf = _buf;if (d->have_new_ctrl) {rdt_last_cmd_printf ("duplicate domain %d\n" , d->id);
@@ -126,8 +127,9 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* Read one cache bit mask (hex). Check that it is valid for the current * resource type. */ -int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d) +int parse_cbm (void *_data,struct rdt_resource *r,struct rdt_domain *d) { +char *buf = _data;u32 cbm_val;if (d->have_new_ctrl) {
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Support flexible data to parsing callbacks
x86/intel_rdt: Support flexible data to parsing callbacks
Commit-ID: 7604df6e16ae0b4dba6553ae74abcf7280a512fb Gitweb: https://git.kernel.org/tip/7604df6e16ae0b4dba6553ae74abcf7280a512fb Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:03 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:43 +0200
x86/intel_rdt: Support flexible data to parsing callbacks
Each resource is associated with a configurable callback that should be used to parse the information provided for the particular resource from user space. In addition to the resource and domain pointers this callback is provided with just the character buffer being parsed.
In support of flexible parsing the callback is modified to support a void pointer as argument. This enables resources that need more data than just the user provided data to pass its required data to the callback without affecting the signatures for the callbacks of all the other resources.
--- arch/x86/kernel/cpu/intel_rdt.h | 6 +++--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 6 ++++-- 2 files changed, 7 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +3 -3
@@ -371,7 +371,7 @@ struct rdt_resource {struct rdt_cache cache;struct rdt_membw membw;const char *format_str; -int (*parse_ctrlval) (char *buf,struct rdt_resource *r, +int (*parse_ctrlval) (void *data,struct rdt_resource *r,struct rdt_domain *d);struct list_head evt_list;int num_rmid;
@@ -379,8 +379,8 @@ struct rdt_resource {unsigned long fflags; }; -int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d); -int parse_bw (char *buf,struct rdt_resource *r,struct rdt_domain *d); +int parse_cbm (void *_data,struct rdt_resource *r,struct rdt_domain *d); +int parse_bw (void *_buf,struct rdt_resource *r,struct rdt_domain *d);extern struct mutex rdtgroup_mutex;
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +4 -2
@@ -64,9 +64,10 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)return true; } -int parse_bw (char *buf,struct rdt_resource *r,struct rdt_domain *d) +int parse_bw (void *_buf,struct rdt_resource *r,struct rdt_domain *d) {unsigned long data; +char *buf = _buf;if (d->have_new_ctrl) {rdt_last_cmd_printf ("duplicate domain %d\n" , d->id);
@@ -126,8 +127,9 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)* Read one cache bit mask (hex). Check that it is valid for the current * resource type. */ -int parse_cbm (char *buf,struct rdt_resource *r,struct rdt_domain *d) +int parse_cbm (void *_data,struct rdt_resource *r,struct rdt_domain *d) { +char *buf = _data;u32 cbm_val;if (d->have_new_ctrl) {
PATCH v7 13/41 x86/intel_rdt: Ensure requested schemata respects mode
In reply to Reinette ChatreWhen the administrator requests a change in a resource group's schemata we have to ensure that the new schemata respects the current resource group as well as the other active resource groups' schemata.
The new schemata is not allowed to overlap with the schemata of any exclusive resource groups. Similarly, if the resource group being changed is exclusive then its new schemata is not allowed to overlap with any schemata of any other active resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 49 ++++++++++++++++----- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 4 +- 3 files changed, 41 insertions(+), 14 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -465,6 +465,8 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off);int rdtgroup_schemata_show (struct kernfs_open_file *of,struct seq_file *s,void *v); +bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +37 -12
114 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +2 -2
@@ -829,8 +829,8 @@ static int rdtgroup_mode_show(struct kernfs_open_file *of,* * Return: false if CBM does not overlap, true if it does. */ -static bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, -u32 _cbm,int closid,bool exclusive) +bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive) {unsigned long *cbm = (unsigned long *)&_cbm;unsigned long *ctrl_b;
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Ensure requested schemata respects mode
x86/intel_rdt: Ensure requested schemata respects mode
Commit-ID: 9ab9aa15c3096000678891b61f7309c8729928e0 Gitweb: https://git.kernel.org/tip/9ab9aa15c3096000678891b61f7309c8729928e0 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:04 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:43 +0200
x86/intel_rdt: Ensure requested schemata respects mode
When the administrator requests a change in a resource group's schemata we have to ensure that the new schemata respects the current resource group as well as the other active resource groups' schemata.
The new schemata is not allowed to overlap with the schemata of any exclusive resource groups. Similarly, if the resource group being changed is exclusive then its new schemata is not allowed to overlap with any schemata of any other active resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 49 ++++++++++++++++++++++------- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 4 +-- 3 files changed, 41 insertions(+), 14 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -465,6 +465,8 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off);int rdtgroup_schemata_show (struct kernfs_open_file *of,struct seq_file *s,void *v); +bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +37 -12
114 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +2 -2
@@ -829,8 +829,8 @@ static int rdtgroup_mode_show(struct kernfs_open_file *of,* * Return: false if CBM does not overlap, true if it does. */ -static bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, -u32 _cbm,int closid,bool exclusive) +bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d, +u32 _cbm,int closid,bool exclusive) {unsigned long *cbm = (unsigned long *)&_cbm;unsigned long *ctrl_b;
PATCH v7 14/41 x86/intel_rdt: Introduce "bit_usage" to display cache allocations details
In reply to Reinette ChatreWith cache regions now explicitly marked as "shareable" or "exclusive" we would like to communicate to the user how portions of the cache are used.
Introduce "bit_usage" that indicates for each resource how portions of the cache are configured to be used.
To assist the user to distinguish whether the sharing is from software or hardware we add the following annotation:
0 - currently unused
X - currently available for sharing and used by software and hardware H - currently used by hardware only but available for software use S - currently used and shareable by software only
E - currently used exclusively by one resource group
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 79 ++++++++++++++++++++++++ 1 file changed, 79 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +79 -0
@@ -714,6 +714,78 @@ static int rdt_shareable_bits_show(struct kernfs_open_file *of,return 0 ; } +/** +* rdt_bit_usage_show - Display current usage of resources +* +* A domain is a shared resource that can now be allocated differently. Here +* we display the current regions of the domain as an annotated bitmask. +* For each domain of this resource its allocation bitmask +* is annotated as below to indicate the current usage of the corresponding bit: +* 0 - currently unused +* X - currently available for sharing and used by software and hardware +* H - currently used by hardware only but available for software use +* S - currently used and shareable by software only +* E - currently used exclusively by one resource group +*/ +static int rdt_bit_usage_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v) +{ +struct rdt_resource *r = of->kn->parent->priv; +u32 sw_shareable, hw_shareable, exclusive; +struct rdt_domain *dom; +int i, hwb, swb, excl; +enum rdtgrp_mode mode; +bool sep = false; +u32 *ctrl; + +mutex_lock (&rdtgroup_mutex); + hw_shareable = r->cache.shareable_bits; +list_for_each_entry (dom, &r->domains, list) { +if (sep) +seq_putc (seq,';' ); + ctrl = dom->ctrl_val; + sw_shareable =0 ; + exclusive =0 ; +seq_printf (seq,"%d=" , dom->id); +for (i =0 ; i < r->num_closid; i++, ctrl++) { +if (!closid_allocated (i)) +continue ; + mode =rdtgroup_mode_by_closid (i); +switch (mode) { +case RDT_MODE_SHAREABLE : + sw_shareable |= *ctrl; +break ; +case RDT_MODE_EXCLUSIVE : + exclusive |= *ctrl; +break ; +case RDT_NUM_MODES : +WARN (1 , +"invalid mode for closid %d\n" , i); +break ; + } + } +for (i = r->cache.cbm_len -1 ; i >=0 ; i--) { + hwb =test_bit (i, (unsigned long *)&hw_shareable); + swb =test_bit (i, (unsigned long *)&sw_shareable); + excl =test_bit (i, (unsigned long *)&exclusive); +if (hwb && swb) +seq_putc (seq,'X' ); +else if (hwb && !swb) +seq_putc (seq,'H' ); +else if (!hwb && swb) +seq_putc (seq,'S' ); +else if (excl) +seq_putc (seq,'E' ); +else /* Unused bits remain */ +seq_putc (seq,'0' ); + } + sep = true; + } +seq_putc (seq,'\n' ); +mutex_unlock (&rdtgroup_mutex); +return 0 ; +} +static int rdt_min_bw_show (struct kernfs_open_file *of,struct seq_file *seq,void *v) {
@@ -995,6 +1067,13 @@ static struct rftype res_common_files[] = { .seq_show = rdt_shareable_bits_show, .fflags = RF_CTRL_INFO | RFTYPE_RES_CACHE, }, + { + .name = "bit_usage", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdt_bit_usage_show, + .fflags = RF_CTRL_INFO | RFTYPE_RES_CACHE, + }, { .name = "min_bandwidth", .mode = 0444,
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce "bit_usage" to display cache allocations details
x86/intel_rdt: Introduce "bit_usage" to display cache allocations details
Commit-ID: e651901187ab8bc8a0a969144da6ae0e2e59a148 Gitweb: https://git.kernel.org/tip/e651901187ab8bc8a0a969144da6ae0e2e59a148 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:05 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:44 +0200
x86/intel_rdt: Introduce "bit_usage" to display cache allocations details
With cache regions now explicitly marked as "shareable" or "exclusive" we would like to communicate to the user how portions of the cache are used.
Introduce "bit_usage" that indicates for each resource how portions of the cache are configured to be used.
To assist the user to distinguish whether the sharing is from software or hardware we add the following annotation:
0 - currently unused
X - currently available for sharing and used by software and hardware H - currently used by hardware only but available for software use S - currently used and shareable by software only
E - currently used exclusively by one resource group
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 79 ++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +79 -0
@@ -714,6 +714,78 @@ static int rdt_shareable_bits_show(struct kernfs_open_file *of,return 0 ; } +/** +* rdt_bit_usage_show - Display current usage of resources +* +* A domain is a shared resource that can now be allocated differently. Here +* we display the current regions of the domain as an annotated bitmask. +* For each domain of this resource its allocation bitmask +* is annotated as below to indicate the current usage of the corresponding bit: +* 0 - currently unused +* X - currently available for sharing and used by software and hardware +* H - currently used by hardware only but available for software use +* S - currently used and shareable by software only +* E - currently used exclusively by one resource group +*/ +static int rdt_bit_usage_show (struct kernfs_open_file *of, +struct seq_file *seq,void *v) +{ +struct rdt_resource *r = of->kn->parent->priv; +u32 sw_shareable, hw_shareable, exclusive; +struct rdt_domain *dom; +int i, hwb, swb, excl; +enum rdtgrp_mode mode; +bool sep = false; +u32 *ctrl; + +mutex_lock (&rdtgroup_mutex); + hw_shareable = r->cache.shareable_bits; +list_for_each_entry (dom, &r->domains, list) { +if (sep) +seq_putc (seq,';' ); + ctrl = dom->ctrl_val; + sw_shareable =0 ; + exclusive =0 ; +seq_printf (seq,"%d=" , dom->id); +for (i =0 ; i < r->num_closid; i++, ctrl++) { +if (!closid_allocated (i)) +continue ; + mode =rdtgroup_mode_by_closid (i); +switch (mode) { +case RDT_MODE_SHAREABLE : + sw_shareable |= *ctrl; +break ; +case RDT_MODE_EXCLUSIVE : + exclusive |= *ctrl; +break ; +case RDT_NUM_MODES : +WARN (1 , +"invalid mode for closid %d\n" , i); +break ; + } + } +for (i = r->cache.cbm_len -1 ; i >=0 ; i--) { + hwb =test_bit (i, (unsigned long *)&hw_shareable); + swb =test_bit (i, (unsigned long *)&sw_shareable); + excl =test_bit (i, (unsigned long *)&exclusive); +if (hwb && swb) +seq_putc (seq,'X' ); +else if (hwb && !swb) +seq_putc (seq,'H' ); +else if (!hwb && swb) +seq_putc (seq,'S' ); +else if (excl) +seq_putc (seq,'E' ); +else /* Unused bits remain */ +seq_putc (seq,'0' ); + } + sep = true; + } +seq_putc (seq,'\n' ); +mutex_unlock (&rdtgroup_mutex); +return 0 ; +} +static int rdt_min_bw_show (struct kernfs_open_file *of,struct seq_file *seq,void *v) {
@@ -995,6 +1067,13 @@ static struct rftype res_common_files[] = { .seq_show = rdt_shareable_bits_show, .fflags = RF_CTRL_INFO | RFTYPE_RES_CACHE, }, + { + .name = "bit_usage", + .mode = 0444, + .kf_ops = &rdtgroup_kf_single_ops, + .seq_show = rdt_bit_usage_show, + .fflags = RF_CTRL_INFO | RFTYPE_RES_CACHE, + }, { .name = "min_bandwidth", .mode = 0444,
PATCH v7 15/41 x86/intel_rdt: Display resource groups' allocations' size in bytes
In reply to Reinette ChatreThe schemata file displays the allocations associated with each domain of each resource. The syntax of this file reflects the capacity bitmask (CBM) of the actual allocation. In order to determine the actual size of an allocation the user needs to dig through three different files to query the variables needed to compute it (the cache size, the CBM length, and the schemata).
Introduce a new file "size" associated with each resource group that will mirror the schemata file syntax and display the size in bytes of each allocation.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 81 ++++++++++++++++++++++++ 2 files changed, 83 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -467,6 +467,8 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, struct seq_file *s, void *v);bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d,u32 _cbm,int closid,bool exclusive); +unsigned int rdtgroup_cbm_to_size (struct rdt_resource *r,struct rdt_domain *d, +u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +81 -0
99 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Display resource groups' allocations' size in bytes
x86/intel_rdt: Display resource groups' allocations' size in bytes
Commit-ID: d9b48c86eb380bb272d0ad8f5338d8dc941f3b32 Gitweb: https://git.kernel.org/tip/d9b48c86eb380bb272d0ad8f5338d8dc941f3b32 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:06 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:44 +0200
x86/intel_rdt: Display resource groups' allocations' size in bytes
The schemata file displays the allocations associated with each domain of each resource. The syntax of this file reflects the capacity bitmask (CBM) of the actual allocation. In order to determine the actual size of an allocation the user needs to dig through three different files to query the variables needed to compute it (the cache size, the CBM length, and the schemata).
Introduce a new file "size" associated with each resource group that will mirror the schemata file syntax and display the size in bytes of each allocation.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 81 ++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -467,6 +467,8 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, struct seq_file *s, void *v);bool rdtgroup_cbm_overlaps (struct rdt_resource *r,struct rdt_domain *d,u32 _cbm,int closid,bool exclusive); +unsigned int rdtgroup_cbm_to_size (struct rdt_resource *r,struct rdt_domain *d, +u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +81 -0
99 lines not on this page · raw message
PATCH v7 16/41 x86/intel_rdt: Documentation for Cache Pseudo-Locking
In reply to Reinette ChatreAdd description of Cache Pseudo-Locking feature, its interface, as well as an example of its usage.
--- Documentation/x86/intel_rdt_ui.txt | 280 ++++++++++++++++++++++++++++- 1 file changed, 278 insertions(+), 2 deletions(-)
Documentation/x86/intel_rdt_ui.txt +278 -2
@@ -29,7 +29,11 @@ mount options are: L2 and L3 CDP are controlled seperately. RDT features are orthogonal. A particular system may support only -monitoring, only control, or both monitoring and control. +monitoring, only control, or both monitoring and control. Cache +pseudo-locking is a unique way of using cache control to "pin" or +"lock" data in the cache. Details can be found in +"Cache Pseudo-Locking". + The mount succeeds if either of allocation or monitoring is present, but only those files and directories supported by the system will be created.
@@ -86,6 +90,8 @@ related to allocation: and available for sharing. "E" - Corresponding region is used exclusively by one resource group. No sharing allowed. + "P" - Corresponding region is pseudo-locked. No + sharing allowed. Memory bandwitdh(MB) subdirectory contains the following files with respect to allocation:
@@ -192,7 +198,12 @@ When control is enabled all CTRL_MON groups will also contain: "mode": The "mode" of the resource group dictates the sharing of its allocations. A "shareable" resource group allows sharing of its - allocations while an "exclusive" resource group does not. + allocations while an "exclusive" resource group does not. A + cache pseudo-locked region is created by first writing + "pseudo-locksetup" to the "mode" file before writing the cache + pseudo-locked region's schemata to the resource group's "schemata" + file. On successful pseudo-locked region creation the mode will + automatically change to "pseudo-locked". When monitoring is enabled all MON groups will also contain:
@@ -410,6 +421,170 @@ L3CODE:0=fffff;1=fffff;2=fffff;3=fffff L3DATA:0=fffff;1=fffff;2=3c0;3=fffff L3CODE:0=fffff;1=fffff;2=fffff;3=fffff +Cache Pseudo-Locking +-------------------- +CAT enables a user to specify the amount of cache space that an +application can fill. Cache pseudo-locking builds on the fact that a +CPU can still read and write data pre-allocated outside its current +allocated area on a cache hit. With cache pseudo-locking, data can be +preloaded into a reserved portion of cache that no application can +fill, and from that point on will only serve cache hits. The cache +pseudo-locked memory is made accessible to user space where an +application can map it into its virtual address space and thus have +a region of memory with reduced average read latency. + +The creation of a cache pseudo-locked region is triggered by a request +from the user to do so that is accompanied by a schemata of the region +to be pseudo-locked. The cache pseudo-locked region is created as follows: +- Create a CAT allocation CLOSNEW with a CBM matching the schemata + from the user of the cache region that will contain the pseudo-locked + memory. This region must not overlap with any current CAT allocation/CLOS + on the system and no future overlap with this cache region is allowed + while the pseudo-locked region exists. +- Create a contiguous region of memory of the same size as the cache + region. +- Flush the cache, disable hardware prefetchers, disable preemption. +- Make CLOSNEW the active CLOS and touch the allocated memory to load + it into the cache. +- Set the previous CLOS as active. +- At this point the closid CLOSNEW can be released - the cache + pseudo-locked region is protected as long as its CBM does not appear in + any CAT allocation. Even though the cache pseudo-locked region will from + this point on not appear in any CBM of any CLOS an application running with + any CLOS will be able to access the memory in the pseudo-locked region since + the region continues to serve cache hits. +- The contiguous region of memory loaded into the cache is exposed to + user-space as a character device. + +Cache pseudo-locking increases the probability that data will remain +in the cache via carefully configuring the CAT feature and controlling +application behavior. There is no guarantee that data is placed in +cache. Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict +“locked” data from cache. Power management C-states may shrink or +power off cache. It is thus recommended to limit the processor maximum +C-state, for example, by setting the processor.max_cstate kernel parameter. + +It is required that an application using a pseudo-locked region runs +with affinity to the cores (or a subset of the cores) associated +with the cache on which the pseudo-locked region resides. A sanity check +within the code will not allow an application to map pseudo-locked memory +unless it runs with affinity to cores associated with the cache on which the +pseudo-locked region resides. The sanity check is only done during the +initial mmap() handling, there is no enforcement afterwards and the +application self needs to ensure it remains affine to the correct cores. + +Pseudo-locking is accomplished in two stages: +1) During the first stage the system administrator allocates a portion + of cache that should be dedicated to pseudo-locking. At this time an + equivalent portion of memory is allocated, loaded into allocated + cache portion, and exposed as a character device. +2) During the second stage a user-space application maps (mmap()) the + pseudo-locked memory into its address space. + +Cache Pseudo-Locking Interface +------------------------------ +A pseudo-locked region is created using the resctrl interface as follows: + +1) Create a new resource group by creating a new directory in /sys/fs/resctrl. +2) Change the new resource group's mode to "pseudo-locksetup" by writing + "pseudo-locksetup" to the "mode" file. +3) Write the schemata of the pseudo-locked region to the "schemata" file. All + bits within the schemata should be "unused" according to the "bit_usage" + file. + +On successful pseudo-locked region creation the "mode" file will contain +"pseudo-locked" and a new character device with the same name as the resource +group will exist in /dev/pseudo_lock. This character device can be mmap()'ed +by user space in order to obtain access to the pseudo-locked memory region. + +An example of cache pseudo-locked region creation and usage can be found below. + +Cache Pseudo-Locking Debugging Interface +--------------------------------------- +The pseudo-locking debugging interface is enabled by default (if +CONFIG_DEBUG_FS is enabled) and can be found in /sys/kernel/debug/resctrl. + +There is no explicit way for the kernel to test if a provided memory +location is present in the cache. The pseudo-locking debugging interface uses +the tracing infrastructure to provide two ways to measure cache residency of +the pseudo-locked region: +1) Memory access latency using the pseudo_lock_mem_latency tracepoint. Data + from these measurements are best visualized using a hist trigger (see + example below). In this test the pseudo-locked region is traversed at + a stride of 32 bytes while hardware prefetchers and preemption + are disabled. This also provides a substitute visualization of cache + hits and misses. +2) Cache hit and miss measurements using model specific precision counters if + available. Depending on the levels of cache on the system the pseudo_lock_l2 + and pseudo_lock_l3 tracepoints are available. + WARNING: triggering this measurement uses from two (for just L2 + measurements) to four (for L2 and L3 measurements) precision counters on + the system, if any other measurements are in progress the counters and + their corresponding event registers will be clobbered. + +When a pseudo-locked region is created a new debugfs directory is created for +it in debugfs as /sys/kernel/debug/resctrl/<newdir>. A single +write-only file, pseudo_lock_measure, is present in this directory. The +measurement on the pseudo-locked region depends on the number, 1 or 2, +written to this debugfs file. Since the measurements are recorded with the +tracing infrastructure the relevant tracepoints need to be enabled before the +measurement is triggered. + +Example of latency debugging interface: +In this example a pseudo-locked region named "newlock" was created. Here is +how we can measure the latency in cycles of reading from this region and +visualize this data with a histogram that is available if CONFIG_HIST_TRIGGERS +is set: +# :> /sys/kernel/debug/tracing/trace +# echo 'hist:keys=latency' > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/trigger +# echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable +# echo 1 > /sys/kernel/debug/resctrl/newlock/pseudo_lock_measure +# echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable +# cat /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/hist + +# event histogram +# +# trigger info: hist:keys=latency:vals=hitcount:sort=hitcount:size=2048 [active] +# + +{ latency: 456 } hitcount: 1 +{ latency: 50 } hitcount: 83 +{ latency: 36 } hitcount: 96 +{ latency: 44 } hitcount: 174 +{ latency: 48 } hitcount: 195 +{ latency: 46 } hitcount: 262 +{ latency: 42 } hitcount: 693 +{ latency: 40 } hitcount: 3204 +{ latency: 38 } hitcount: 3484 + +Totals: + Hits: 8192 + Entries: 9 + Dropped: 0 + +Example of cache hits/misses debugging: +In this example a pseudo-locked region named "newlock" was created on the L2 +cache of a platform. Here is how we can obtain details of the cache hits +and misses using the platform's precision counters. + +# :> /sys/kernel/debug/tracing/trace +# echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_l2/enable +# echo 2 > /sys/kernel/debug/resctrl/newlock/pseudo_lock_measure +# echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_l2/enable +# cat /sys/kernel/debug/tracing/trace + +# tracer: nop +# +# _-----=> irqs-off +# / _----=> need-resched +# | / _---=> hardirq/softirq +# || / _--=> preempt-depth +# ||| / delay +# TASK-PID CPU# |||| TIMESTAMP FUNCTION +# | | | |||| | | + pseudo_lock_mea-1672 [002] .... 3132.860500: pseudo_lock_l2: hits=4097 miss=0 + + Examples for RDT allocation usage: Example 1
@@ -596,6 +771,107 @@ A resource group cannot be forced to overlap with an exclusive resource group: # cat info/last_cmd_status overlaps with exclusive group +Example of Cache Pseudo-Locking +------------------------------- +Lock portion of L2 cache from cache id 1 using CBM 0x3. Pseudo-locked +region is exposed at /dev/pseudo_lock/newlock that can be provided to +application for argument to mmap(). + +# mount -t resctrl resctrl /sys/fs/resctrl/ +# cd /sys/fs/resctrl + +Ensure that there are bits available that can be pseudo-locked, since only +unused bits can be pseudo-locked the bits to be pseudo-locked needs to be +removed from the default resource group's schemata: +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSSSS +# echo 'L2:1=0xfc' > schemata +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSS00 + +Create a new resource group that will be associated with the pseudo-locked +region, indicate that it will be used for a pseudo-locked region, and +configure the requested pseudo-locked region capacity bitmask: + +# mkdir newlock +# echo pseudo-locksetup > newlock/mode +# echo 'L2:1=0x3' > newlock/schemata + +On success the resource group's mode will change to pseudo-locked, the +bit_usage will reflect the pseudo-locked region, and the character device +exposing the pseudo-locked region will exist: + +# cat newlock/mode +pseudo-locked +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSSPP +# ls -l /dev/pseudo_lock/newlock +crw------- 1 root root 243, 0 Apr 3 05:01 /dev/pseudo_lock/newlock + +/* + * Example code to access one page of pseudo-locked cache region + * from user space. + */ +#define _GNU_SOURCE +#include <fcntl.h> +#include <sched.h> +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <sys/mman.h> + +/* + * It is required that the application runs with affinity to only + * cores associated with the pseudo-locked region. Here the cpu + * is hardcoded for convenience of example. + */ +static int cpuid = 2; + +int main(int argc, char *argv[]) +{ + cpu_set_t cpuset; + long page_size; + void *mapping; + int dev_fd; + int ret; + + page_size = sysconf(_SC_PAGESIZE); + + CPU_ZERO(&cpuset); + CPU_SET(cpuid, &cpuset); + ret = sched_setaffinity(0, sizeof(cpuset), &cpuset); + if (ret < 0) { + perror("sched_setaffinity"); + exit(EXIT_FAILURE); + } + + dev_fd = open("/dev/pseudo_lock/newlock", O_RDWR); + if (dev_fd < 0) { + perror("open"); + exit(EXIT_FAILURE); + } + + mapping = mmap(0, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, + dev_fd, 0); + if (mapping == MAP_FAILED) { + perror("mmap"); + close(dev_fd); + exit(EXIT_FAILURE); + } + + /* Application interacts with pseudo-locked memory @mapping */ + + ret = munmap(mapping, page_size); + if (ret < 0) { + perror("munmap"); + close(dev_fd); + exit(EXIT_FAILURE); + } + + close(dev_fd); + exit(EXIT_SUCCESS); +} + Locking between applications ----------------------------
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Documentation for Cache Pseudo-Locking
x86/intel_rdt: Documentation for Cache Pseudo-Locking
Commit-ID: e17e733070d4ab312a35848ab248e85b78dcb3f4 Gitweb: https://git.kernel.org/tip/e17e733070d4ab312a35848ab248e85b78dcb3f4 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:07 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:44 +0200
x86/intel_rdt: Documentation for Cache Pseudo-Locking
Add description of Cache Pseudo-Locking feature, its interface, as well as an example of its usage.
--- Documentation/x86/intel_rdt_ui.txt | 280 ++++++++++++++++++++++++++++++++++++- 1 file changed, 278 insertions(+), 2 deletions(-)
Documentation/x86/intel_rdt_ui.txt +278 -2
@@ -29,7 +29,11 @@ mount options are: L2 and L3 CDP are controlled seperately. RDT features are orthogonal. A particular system may support only -monitoring, only control, or both monitoring and control. +monitoring, only control, or both monitoring and control. Cache +pseudo-locking is a unique way of using cache control to "pin" or +"lock" data in the cache. Details can be found in +"Cache Pseudo-Locking". + The mount succeeds if either of allocation or monitoring is present, but only those files and directories supported by the system will be created.
@@ -86,6 +90,8 @@ related to allocation: and available for sharing. "E" - Corresponding region is used exclusively by one resource group. No sharing allowed. + "P" - Corresponding region is pseudo-locked. No + sharing allowed. Memory bandwitdh(MB) subdirectory contains the following files with respect to allocation:
@@ -192,7 +198,12 @@ When control is enabled all CTRL_MON groups will also contain: "mode": The "mode" of the resource group dictates the sharing of its allocations. A "shareable" resource group allows sharing of its - allocations while an "exclusive" resource group does not. + allocations while an "exclusive" resource group does not. A + cache pseudo-locked region is created by first writing + "pseudo-locksetup" to the "mode" file before writing the cache + pseudo-locked region's schemata to the resource group's "schemata" + file. On successful pseudo-locked region creation the mode will + automatically change to "pseudo-locked". When monitoring is enabled all MON groups will also contain:
@@ -410,6 +421,170 @@ L3CODE:0=fffff;1=fffff;2=fffff;3=fffff L3DATA:0=fffff;1=fffff;2=3c0;3=fffff L3CODE:0=fffff;1=fffff;2=fffff;3=fffff +Cache Pseudo-Locking +-------------------- +CAT enables a user to specify the amount of cache space that an +application can fill. Cache pseudo-locking builds on the fact that a +CPU can still read and write data pre-allocated outside its current +allocated area on a cache hit. With cache pseudo-locking, data can be +preloaded into a reserved portion of cache that no application can +fill, and from that point on will only serve cache hits. The cache +pseudo-locked memory is made accessible to user space where an +application can map it into its virtual address space and thus have +a region of memory with reduced average read latency. + +The creation of a cache pseudo-locked region is triggered by a request +from the user to do so that is accompanied by a schemata of the region +to be pseudo-locked. The cache pseudo-locked region is created as follows: +- Create a CAT allocation CLOSNEW with a CBM matching the schemata + from the user of the cache region that will contain the pseudo-locked + memory. This region must not overlap with any current CAT allocation/CLOS + on the system and no future overlap with this cache region is allowed + while the pseudo-locked region exists. +- Create a contiguous region of memory of the same size as the cache + region. +- Flush the cache, disable hardware prefetchers, disable preemption. +- Make CLOSNEW the active CLOS and touch the allocated memory to load + it into the cache. +- Set the previous CLOS as active. +- At this point the closid CLOSNEW can be released - the cache + pseudo-locked region is protected as long as its CBM does not appear in + any CAT allocation. Even though the cache pseudo-locked region will from + this point on not appear in any CBM of any CLOS an application running with + any CLOS will be able to access the memory in the pseudo-locked region since + the region continues to serve cache hits. +- The contiguous region of memory loaded into the cache is exposed to + user-space as a character device. + +Cache pseudo-locking increases the probability that data will remain +in the cache via carefully configuring the CAT feature and controlling +application behavior. There is no guarantee that data is placed in +cache. Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict +“locked” data from cache. Power management C-states may shrink or +power off cache. It is thus recommended to limit the processor maximum +C-state, for example, by setting the processor.max_cstate kernel parameter. + +It is required that an application using a pseudo-locked region runs +with affinity to the cores (or a subset of the cores) associated +with the cache on which the pseudo-locked region resides. A sanity check +within the code will not allow an application to map pseudo-locked memory +unless it runs with affinity to cores associated with the cache on which the +pseudo-locked region resides. The sanity check is only done during the +initial mmap() handling, there is no enforcement afterwards and the +application self needs to ensure it remains affine to the correct cores. + +Pseudo-locking is accomplished in two stages: +1) During the first stage the system administrator allocates a portion + of cache that should be dedicated to pseudo-locking. At this time an + equivalent portion of memory is allocated, loaded into allocated + cache portion, and exposed as a character device. +2) During the second stage a user-space application maps (mmap()) the + pseudo-locked memory into its address space. + +Cache Pseudo-Locking Interface +------------------------------ +A pseudo-locked region is created using the resctrl interface as follows: + +1) Create a new resource group by creating a new directory in /sys/fs/resctrl. +2) Change the new resource group's mode to "pseudo-locksetup" by writing + "pseudo-locksetup" to the "mode" file. +3) Write the schemata of the pseudo-locked region to the "schemata" file. All + bits within the schemata should be "unused" according to the "bit_usage" + file. + +On successful pseudo-locked region creation the "mode" file will contain +"pseudo-locked" and a new character device with the same name as the resource +group will exist in /dev/pseudo_lock. This character device can be mmap()'ed +by user space in order to obtain access to the pseudo-locked memory region. + +An example of cache pseudo-locked region creation and usage can be found below. + +Cache Pseudo-Locking Debugging Interface +--------------------------------------- +The pseudo-locking debugging interface is enabled by default (if +CONFIG_DEBUG_FS is enabled) and can be found in /sys/kernel/debug/resctrl. + +There is no explicit way for the kernel to test if a provided memory +location is present in the cache. The pseudo-locking debugging interface uses +the tracing infrastructure to provide two ways to measure cache residency of +the pseudo-locked region: +1) Memory access latency using the pseudo_lock_mem_latency tracepoint. Data + from these measurements are best visualized using a hist trigger (see + example below). In this test the pseudo-locked region is traversed at + a stride of 32 bytes while hardware prefetchers and preemption + are disabled. This also provides a substitute visualization of cache + hits and misses. +2) Cache hit and miss measurements using model specific precision counters if + available. Depending on the levels of cache on the system the pseudo_lock_l2 + and pseudo_lock_l3 tracepoints are available. + WARNING: triggering this measurement uses from two (for just L2 + measurements) to four (for L2 and L3 measurements) precision counters on + the system, if any other measurements are in progress the counters and + their corresponding event registers will be clobbered. + +When a pseudo-locked region is created a new debugfs directory is created for +it in debugfs as /sys/kernel/debug/resctrl/<newdir>. A single +write-only file, pseudo_lock_measure, is present in this directory. The +measurement on the pseudo-locked region depends on the number, 1 or 2, +written to this debugfs file. Since the measurements are recorded with the +tracing infrastructure the relevant tracepoints need to be enabled before the +measurement is triggered. + +Example of latency debugging interface: +In this example a pseudo-locked region named "newlock" was created. Here is +how we can measure the latency in cycles of reading from this region and +visualize this data with a histogram that is available if CONFIG_HIST_TRIGGERS +is set: +# :> /sys/kernel/debug/tracing/trace +# echo 'hist:keys=latency' > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/trigger +# echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable +# echo 1 > /sys/kernel/debug/resctrl/newlock/pseudo_lock_measure +# echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable +# cat /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/hist + +# event histogram +# +# trigger info: hist:keys=latency:vals=hitcount:sort=hitcount:size=2048 [active] +# + +{ latency: 456 } hitcount: 1 +{ latency: 50 } hitcount: 83 +{ latency: 36 } hitcount: 96 +{ latency: 44 } hitcount: 174 +{ latency: 48 } hitcount: 195 +{ latency: 46 } hitcount: 262 +{ latency: 42 } hitcount: 693 +{ latency: 40 } hitcount: 3204 +{ latency: 38 } hitcount: 3484 + +Totals: + Hits: 8192 + Entries: 9 + Dropped: 0 + +Example of cache hits/misses debugging: +In this example a pseudo-locked region named "newlock" was created on the L2 +cache of a platform. Here is how we can obtain details of the cache hits +and misses using the platform's precision counters. + +# :> /sys/kernel/debug/tracing/trace +# echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_l2/enable +# echo 2 > /sys/kernel/debug/resctrl/newlock/pseudo_lock_measure +# echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_l2/enable +# cat /sys/kernel/debug/tracing/trace + +# tracer: nop +# +# _-----=> irqs-off +# / _----=> need-resched +# | / _---=> hardirq/softirq +# || / _--=> preempt-depth +# ||| / delay +# TASK-PID CPU# |||| TIMESTAMP FUNCTION +# | | | |||| | | + pseudo_lock_mea-1672 [002] .... 3132.860500: pseudo_lock_l2: hits=4097 miss=0 + + Examples for RDT allocation usage: Example 1
@@ -596,6 +771,107 @@ A resource group cannot be forced to overlap with an exclusive resource group: # cat info/last_cmd_status overlaps with exclusive group +Example of Cache Pseudo-Locking +------------------------------- +Lock portion of L2 cache from cache id 1 using CBM 0x3. Pseudo-locked +region is exposed at /dev/pseudo_lock/newlock that can be provided to +application for argument to mmap(). + +# mount -t resctrl resctrl /sys/fs/resctrl/ +# cd /sys/fs/resctrl + +Ensure that there are bits available that can be pseudo-locked, since only +unused bits can be pseudo-locked the bits to be pseudo-locked needs to be +removed from the default resource group's schemata: +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSSSS +# echo 'L2:1=0xfc' > schemata +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSS00 + +Create a new resource group that will be associated with the pseudo-locked +region, indicate that it will be used for a pseudo-locked region, and +configure the requested pseudo-locked region capacity bitmask: + +# mkdir newlock +# echo pseudo-locksetup > newlock/mode +# echo 'L2:1=0x3' > newlock/schemata + +On success the resource group's mode will change to pseudo-locked, the +bit_usage will reflect the pseudo-locked region, and the character device +exposing the pseudo-locked region will exist: + +# cat newlock/mode +pseudo-locked +# cat info/L2/bit_usage +0=SSSSSSSS;1=SSSSSSPP +# ls -l /dev/pseudo_lock/newlock +crw------- 1 root root 243, 0 Apr 3 05:01 /dev/pseudo_lock/newlock + +/* + * Example code to access one page of pseudo-locked cache region + * from user space. + */ +#define _GNU_SOURCE +#include <fcntl.h> +#include <sched.h> +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <sys/mman.h> + +/* + * It is required that the application runs with affinity to only + * cores associated with the pseudo-locked region. Here the cpu + * is hardcoded for convenience of example. + */ +static int cpuid = 2; + +int main(int argc, char *argv[]) +{ + cpu_set_t cpuset; + long page_size; + void *mapping; + int dev_fd; + int ret; + + page_size = sysconf(_SC_PAGESIZE); + + CPU_ZERO(&cpuset); + CPU_SET(cpuid, &cpuset); + ret = sched_setaffinity(0, sizeof(cpuset), &cpuset); + if (ret < 0) { + perror("sched_setaffinity"); + exit(EXIT_FAILURE); + } + + dev_fd = open("/dev/pseudo_lock/newlock", O_RDWR); + if (dev_fd < 0) { + perror("open"); + exit(EXIT_FAILURE); + } + + mapping = mmap(0, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, + dev_fd, 0); + if (mapping == MAP_FAILED) { + perror("mmap"); + close(dev_fd); + exit(EXIT_FAILURE); + } + + /* Application interacts with pseudo-locked memory @mapping */ + + ret = munmap(mapping, page_size); + if (ret < 0) { + perror("munmap"); + close(dev_fd); + exit(EXIT_FAILURE); + } + + close(dev_fd); + exit(EXIT_SUCCESS); +} + Locking between applications ----------------------------
PATCH v7 17/41 x86/intel_rdt: Introduce the Cache Pseudo-Locking modes
In reply to Reinette ChatreThe two modes used to manage Cache Pseudo-Locked regions are introduced. A resource group is assigned "pseudo-locksetup" mode when the user indicates that this resource group will be used for a Cache Pseudo-Locked region. When the Cache Pseudo-Locked region has been set up successfully after the user wrote the requested schemata to the "schemata" file, then the mode will automatically changed to "pseudo-locked". The user is not able to modify the mode to "pseudo-locked" by writing "pseudo-locked" to the "mode" file directly.
--- arch/x86/kernel/cpu/intel_rdt.h | 10 ++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 13 +++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +10 -0
@@ -84,15 +84,25 @@ enum rdt_group_type {* enum rdtgrp_mode - Mode of a RDT resource group * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed +* @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking +* @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations +* allowed AND the allocations are Cache Pseudo-Locked * * The mode of a resource group enables control over the allowed overlap * between allocations associated with different resource groups (classes * of service). User is able to modify the mode of a resource group by * writing to the "mode" resctrl file associated with the resource group. +* +* The "shareable", "exclusive", and "pseudo-locksetup" modes are set by +* writing the appropriate text to the "mode" file. A resource group enters +* "pseudo-locked" mode after the schemata is written while the resource +* group is in "pseudo-locksetup" mode. */ enum rdtgrp_mode {RDT_MODE_SHAREABLE =0 ,RDT_MODE_EXCLUSIVE , +RDT_MODE_PSEUDO_LOCKSETUP , +RDT_MODE_PSEUDO_LOCKED ,/* Must be last */ RDT_NUM_MODES ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +11 -2
@@ -161,8 +161,10 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid) }static const char *const rdt_mode_str[] = { - [RDT_MODE_SHAREABLE ] ="shareable" , - [RDT_MODE_EXCLUSIVE ] ="exclusive" , + [RDT_MODE_SHAREABLE ] ="shareable" , + [RDT_MODE_EXCLUSIVE ] ="exclusive" , + [RDT_MODE_PSEUDO_LOCKSETUP ] ="pseudo-locksetup" , + [RDT_MODE_PSEUDO_LOCKED ] ="pseudo-locked" , };/**
@@ -759,6 +761,13 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,case RDT_MODE_EXCLUSIVE : exclusive |= *ctrl;break ; +/* +* Temporarily handle pseudo-locking enums +* to silence compile warnings until handling +* added in later patches. +*/ +case RDT_MODE_PSEUDO_LOCKSETUP : +case RDT_MODE_PSEUDO_LOCKED :case RDT_NUM_MODES :WARN (1 ,"invalid mode for closid %d\n" , i);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce the Cache Pseudo-Locking modes
x86/intel_rdt: Introduce the Cache Pseudo-Locking modes
Commit-ID: bb9fec69cb41380428d6b8dab3a303189a9b480a Gitweb: https://git.kernel.org/tip/bb9fec69cb41380428d6b8dab3a303189a9b480a Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:08 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:45 +0200
x86/intel_rdt: Introduce the Cache Pseudo-Locking modes
The two modes used to manage Cache Pseudo-Locked regions are introduced. A resource group is assigned "pseudo-locksetup" mode when the user indicates that this resource group will be used for a Cache Pseudo-Locked region. When the Cache Pseudo-Locked region has been set up successfully after the user wrote the requested schemata to the "schemata" file, then the mode will automatically changed to "pseudo-locked". The user is not able to modify the mode to "pseudo-locked" by writing "pseudo-locked" to the "mode" file directly.
--- arch/x86/kernel/cpu/intel_rdt.h | 10 ++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 13 +++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +10 -0
@@ -84,15 +84,25 @@ enum rdt_group_type {* enum rdtgrp_mode - Mode of a RDT resource group * @RDT_MODE_SHAREABLE: This resource group allows sharing of its allocations * @RDT_MODE_EXCLUSIVE: No sharing of this resource group's allocations allowed +* @RDT_MODE_PSEUDO_LOCKSETUP: Resource group will be used for Pseudo-Locking +* @RDT_MODE_PSEUDO_LOCKED: No sharing of this resource group's allocations +* allowed AND the allocations are Cache Pseudo-Locked * * The mode of a resource group enables control over the allowed overlap * between allocations associated with different resource groups (classes * of service). User is able to modify the mode of a resource group by * writing to the "mode" resctrl file associated with the resource group. +* +* The "shareable", "exclusive", and "pseudo-locksetup" modes are set by +* writing the appropriate text to the "mode" file. A resource group enters +* "pseudo-locked" mode after the schemata is written while the resource +* group is in "pseudo-locksetup" mode. */ enum rdtgrp_mode {RDT_MODE_SHAREABLE =0 ,RDT_MODE_EXCLUSIVE , +RDT_MODE_PSEUDO_LOCKSETUP , +RDT_MODE_PSEUDO_LOCKED ,/* Must be last */ RDT_NUM_MODES ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +11 -2
@@ -161,8 +161,10 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid) }static const char *const rdt_mode_str[] = { - [RDT_MODE_SHAREABLE ] ="shareable" , - [RDT_MODE_EXCLUSIVE ] ="exclusive" , + [RDT_MODE_SHAREABLE ] ="shareable" , + [RDT_MODE_EXCLUSIVE ] ="exclusive" , + [RDT_MODE_PSEUDO_LOCKSETUP ] ="pseudo-locksetup" , + [RDT_MODE_PSEUDO_LOCKED ] ="pseudo-locked" , };/**
@@ -759,6 +761,13 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,case RDT_MODE_EXCLUSIVE : exclusive |= *ctrl;break ; +/* +* Temporarily handle pseudo-locking enums +* to silence compile warnings until handling +* added in later patches. +*/ +case RDT_MODE_PSEUDO_LOCKSETUP : +case RDT_MODE_PSEUDO_LOCKED :case RDT_NUM_MODES :WARN (1 ,"invalid mode for closid %d\n" , i);
PATCH v7 18/41 x86/intel_rdt: Respect read and write access
In reply to Reinette ChatreBy default, if the opener has CAP_DAC_OVERRIDE, a kernfs file can be opened regardless of RW permissions. Writing to a kernfs file will thus succeed even if permissions are 0000.
It's required to restrict the actions that can be performed on a resource group from userspace based on the mode of the resource group. This restriction will be done through a modification of the file permissions. That is, for example, if a resource group is locked then the user cannot add tasks to the resource group.
For this restriction through file permissions to work it has to be ensured that the permissions are always respected. To do so the resctrl filesystem is created with the KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK flag that will result in open(2) failing with -EACCESS regardless of CAP_DAC_OVERRIDE if the permission does not have the respective read or write access.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +2 -1
@@ -2554,7 +2554,8 @@ static int __init rdtgroup_setup_root(void)int ret; rdt_root =kernfs_create_root (&rdtgroup_kf_syscall_ops, -KERNFS_ROOT_CREATE_DEACTIVATED , +KERNFS_ROOT_CREATE_DEACTIVATED | +KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK , &rdtgroup_default);if (IS_ERR (rdt_root))return PTR_ERR (rdt_root);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Respect read and write access
x86/intel_rdt: Respect read and write access
Commit-ID: 21220bb199f7d65c8f0a63ac7d3209e40fbdd706 Gitweb: https://git.kernel.org/tip/21220bb199f7d65c8f0a63ac7d3209e40fbdd706 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:09 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:45 +0200
x86/intel_rdt: Respect read and write access
By default, if the opener has CAP_DAC_OVERRIDE, a kernfs file can be opened regardless of RW permissions. Writing to a kernfs file will thus succeed even if permissions are 0000.
It's required to restrict the actions that can be performed on a resource group from userspace based on the mode of the resource group. This restriction will be done through a modification of the file permissions. That is, for example, if a resource group is locked then the user cannot add tasks to the resource group.
For this restriction through file permissions to work it has to be ensured that the permissions are always respected. To do so the resctrl filesystem is created with the KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK flag that will result in open(2) failing with -EACCESS regardless of CAP_DAC_OVERRIDE if the permission does not have the respective read or write access.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +2 -1
@@ -2545,7 +2545,8 @@ static int __init rdtgroup_setup_root(void)int ret; rdt_root =kernfs_create_root (&rdtgroup_kf_syscall_ops, -KERNFS_ROOT_CREATE_DEACTIVATED , +KERNFS_ROOT_CREATE_DEACTIVATED | +KERNFS_ROOT_EXTRA_OPEN_PERM_CHECK , &rdtgroup_default);if (IS_ERR (rdt_root))return PTR_ERR (rdt_root);
PATCH v7 19/41 x86/intel_rdt: Add utility to test if tasks assigned to resource group
In reply to Reinette ChatreIn considering changes to a resource group it becomes necessary to know whether tasks have been assigned to the resource group in question.
Introduce a new utility that can be used to check if any tasks have been assigned to a particular resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 1 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 26 ++++++++++++++++++++++++ 2 files changed, 27 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +1 -0
@@ -480,6 +480,7 @@ bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,unsigned int rdtgroup_cbm_to_size (struct rdt_resource *r,struct rdt_domain *d,u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid); +int rdtgroup_tasks_assigned (struct rdtgroup *r);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +26 -0
@@ -564,6 +564,32 @@ static int __rdtgroup_move_task(struct task_struct *tsk,return ret; } +/** +* rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group +* @r: Resource group +* +* Return: 1 if tasks have been assigned to @r, 0 otherwise +*/ +int rdtgroup_tasks_assigned (struct rdtgroup *r) +{ +struct task_struct *p, *t; +int ret =0 ; + +lockdep_assert_held (&rdtgroup_mutex); + +rcu_read_lock (); +for_each_process_thread (p, t) { +if ((r->type ==RDTCTRL_GROUP && t->closid == r->closid) || + (r->type ==RDTMON_GROUP && t->rmid == r->mon.rmid)) { + ret =1 ; +break ; + } + } +rcu_read_unlock (); + +return ret; +} +static int rdtgroup_task_write_permission (struct task_struct *task,struct kernfs_open_file *of) {
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Add utility to test if tasks assigned to resource group
x86/intel_rdt: Add utility to test if tasks assigned to resource group
Commit-ID: f7a6e3f6f5ffa4926818ea2b3e8994ecef00b84f Gitweb: https://git.kernel.org/tip/f7a6e3f6f5ffa4926818ea2b3e8994ecef00b84f Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:10 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:45 +0200
x86/intel_rdt: Add utility to test if tasks assigned to resource group
In considering changes to a resource group it becomes necessary to know whether tasks have been assigned to the resource group in question.
Introduce a new utility that can be used to check if any tasks have been assigned to a particular resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 1 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +1 -0
@@ -480,6 +480,7 @@ bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,unsigned int rdtgroup_cbm_to_size (struct rdt_resource *r,struct rdt_domain *d,u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid); +int rdtgroup_tasks_assigned (struct rdtgroup *r);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +26 -0
@@ -564,6 +564,32 @@ static int __rdtgroup_move_task(struct task_struct *tsk,return ret; } +/** +* rdtgroup_tasks_assigned - Test if tasks have been assigned to resource group +* @r: Resource group +* +* Return: 1 if tasks have been assigned to @r, 0 otherwise +*/ +int rdtgroup_tasks_assigned (struct rdtgroup *r) +{ +struct task_struct *p, *t; +int ret =0 ; + +lockdep_assert_held (&rdtgroup_mutex); + +rcu_read_lock (); +for_each_process_thread (p, t) { +if ((r->type ==RDTCTRL_GROUP && t->closid == r->closid) || + (r->type ==RDTMON_GROUP && t->rmid == r->mon.rmid)) { + ret =1 ; +break ; + } + } +rcu_read_unlock (); + +return ret; +} +static int rdtgroup_task_write_permission (struct task_struct *task,struct kernfs_open_file *of) {
PATCH v7 20/41 x86/intel_rdt: Add utility to restrict/restore access to resctrl files
In reply to Reinette ChatreWhen a resource group is used for Cache Pseudo-Locking then the region of cache ends up being orphaned with no class of service referring to it. The resctrl files intended to manage how the classes of services are utilized thus become irrelevant.
The fact that a resctrl file is not relevant can be communicated to the user by setting all of its permissions to zero. That is, its read, write, and execute permissions are unset for all users.
Introduce two utilities, rdtgroup_kn_mode_restrict() and rdtgroup_kn_mode_restore(), that can be used to restrict and restore the permissions of a file or directory belonging to a resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 95 ++++++++++++++++++++++++ 2 files changed, 97 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -469,6 +469,8 @@ void rdt_last_cmd_printf(const char *fmt, ...);void rdt_ctrl_update (void *arg);struct rdtgroup *rdtgroup_kn_lock_live (struct kernfs_node *kn);void rdtgroup_kn_unlock (struct kernfs_node *kn); +int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name); +int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name);struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id,struct list_head **pos); ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +95 -0
101 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Add utility to restrict/restore access to resctrl files
x86/intel_rdt: Add utility to restrict/restore access to resctrl files
Commit-ID: 125db711e3629977b5e1f06fa066abe6366db294 Gitweb: https://git.kernel.org/tip/125db711e3629977b5e1f06fa066abe6366db294 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:11 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:46 +0200
x86/intel_rdt: Add utility to restrict/restore access to resctrl files
When a resource group is used for Cache Pseudo-Locking then the region of cache ends up being orphaned with no class of service referring to it. The resctrl files intended to manage how the classes of services are utilized thus become irrelevant.
The fact that a resctrl file is not relevant can be communicated to the user by setting all of its permissions to zero. That is, its read, write, and execute permissions are unset for all users.
Introduce two utilities, rdtgroup_kn_mode_restrict() and rdtgroup_kn_mode_restore(), that can be used to restrict and restore the permissions of a file or directory belonging to a resource group.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 95 ++++++++++++++++++++++++++++++++ 2 files changed, 97 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -469,6 +469,8 @@ void rdt_last_cmd_printf(const char *fmt, ...);void rdt_ctrl_update (void *arg);struct rdtgroup *rdtgroup_kn_lock_live (struct kernfs_node *kn);void rdtgroup_kn_unlock (struct kernfs_node *kn); +int rdtgroup_kn_mode_restrict (struct rdtgroup *r,const char *name); +int rdtgroup_kn_mode_restore (struct rdtgroup *r,const char *name);struct rdt_domain *rdt_find_domain (struct rdt_resource *r,int id,struct list_head **pos); ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +95 -0
101 lines not on this page · raw message
PATCH v7 21/41 x86/intel_rdt: Protect against resource group changes during locking
In reply to Reinette ChatreWe intend to modify file permissions to make the "tasks", "cpus", and "cpus_list" not accessible to the user when cache pseudo-locking in progress. Even so, it is still possible for the user to force the file permissions (using chmod) to make them writeable. Similarly, directory permissions will be modified to prevent future monitor group creation but the user can override these restrictions also.
Add additional checks to the files we intend to restrict to ensure that no modifications from user space are attempted while setting up a pseudo-locking or after a pseudo-locked region is set up.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 10 +++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 32 ++++++++++++++++++--- 2 files changed, 38 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +10 -0
@@ -283,6 +283,16 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, }rdt_last_cmd_clear (); +/* +* No changes to pseudo-locked region allowed. It has to be removed +* and re-created instead. +*/ +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("resource group is pseudo-locked\n" ); + goto out; + } +for_each_alloc_enabled_rdt_resource (r) {list_for_each_entry (dom, &r->domains, list) dom->have_new_ctrl = false;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +28 -4
@@ -449,6 +449,13 @@ static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of, goto unlock; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto unlock; + } +if (is_cpu_list (of)) ret =cpulist_parse (buf, newmask);else
@@ -651,13 +658,22 @@ static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,if (kstrtoint (strstrip (buf),0 , &pid) || pid <0 )return -EINVAL ; rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + }rdt_last_cmd_clear (); -if (rdtgrp) - ret =rdtgroup_move_task (pid, rdtgrp, of); -else - ret = -ENOENT ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto unlock; + } + + ret =rdtgroup_move_task (pid, rdtgrp, of); +unlock:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes;
@@ -2324,6 +2340,14 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_unlock; } +if (rtype ==RDTMON_GROUP && + (prdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + prdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED )) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto out_unlock; + } +/* allocate the rdtgroup. */ rdtgrp =kzalloc (sizeof (*rdtgrp),GFP_KERNEL );if (!rdtgrp) {
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Protect against resource group changes during locking
x86/intel_rdt: Protect against resource group changes during locking
Commit-ID: c966dac8a5ede5d5f9b730512d8bdbcec307fe38 Gitweb: https://git.kernel.org/tip/c966dac8a5ede5d5f9b730512d8bdbcec307fe38 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:12 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:46 +0200
x86/intel_rdt: Protect against resource group changes during locking
We intend to modify file permissions to make the "tasks", "cpus", and "cpus_list" not accessible to the user when cache pseudo-locking in progress. Even so, it is still possible for the user to force the file permissions (using chmod) to make them writeable. Similarly, directory permissions will be modified to prevent future monitor group creation but the user can override these restrictions also.
Add additional checks to the files we intend to restrict to ensure that no modifications from user space are attempted while setting up a pseudo-locking or after a pseudo-locked region is set up.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 10 +++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 32 +++++++++++++++++++++++++---- 2 files changed, 38 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +10 -0
@@ -283,6 +283,16 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, }rdt_last_cmd_clear (); +/* +* No changes to pseudo-locked region allowed. It has to be removed +* and re-created instead. +*/ +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("resource group is pseudo-locked\n" ); + goto out; + } +for_each_alloc_enabled_rdt_resource (r) {list_for_each_entry (dom, &r->domains, list) dom->have_new_ctrl = false;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +28 -4
@@ -449,6 +449,13 @@ static ssize_t rdtgroup_cpus_write(struct kernfs_open_file *of, goto unlock; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto unlock; + } +if (is_cpu_list (of)) ret =cpulist_parse (buf, newmask);else
@@ -651,13 +658,22 @@ static ssize_t rdtgroup_tasks_write(struct kernfs_open_file *of,if (kstrtoint (strstrip (buf),0 , &pid) || pid <0 )return -EINVAL ; rdtgrp =rdtgroup_kn_lock_live (of->kn); +if (!rdtgrp) { +rdtgroup_kn_unlock (of->kn); +return -ENOENT ; + }rdt_last_cmd_clear (); -if (rdtgrp) - ret =rdtgroup_move_task (pid, rdtgrp, of); -else - ret = -ENOENT ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto unlock; + } + + ret =rdtgroup_move_task (pid, rdtgrp, of); +unlock:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes;
@@ -2315,6 +2331,14 @@ static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, goto out_unlock; } +if (rtype ==RDTMON_GROUP && + (prdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + prdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED )) { + ret = -EINVAL ; +rdt_last_cmd_puts ("pseudo-locking in progress\n" ); + goto out_unlock; + } +/* allocate the rdtgroup. */ rdtgrp =kzalloc (sizeof (*rdtgrp),GFP_KERNEL );if (!rdtgrp) {
PATCH v7 22/41 x86/intel_rdt: Utilities to restrict/restore access to specific files
In reply to Reinette ChatreIn support of Cache Pseudo-Locking we need to restrict access to specific resctrl files to protect the state of a resource group used for pseudo-locking from being changed in unsupported ways.
Introduce two utilities that can be used to either restrict or restore the access to all files irrelevant to cache pseudo-locking when pseudo-locking in progress for the resource group.
At this time introduce a new source file, intel_rdt_pseudo_lock.c, that will contain most of the code related to cache pseudo-locking.
Temporarily mark these new functions as unused to silence compile warnings until they are used.
--- arch/x86/kernel/cpu/Makefile | 3 +- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 113 ++++++++++++++++++++ 2 files changed, 115 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c
arch/x86/kernel/cpu/Makefile +2 -1
@@ -35,7 +35,8 @@ obj-$(CONFIG_CPU_SUP_CENTAUR) += centaur.o obj-$(CONFIG_CPU_SUP_TRANSMETA_32) += transmeta.o obj-$(CONFIG_CPU_SUP_UMC_32) += umc.o -obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o intel_rdt_ctrlmondata.o +obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o +obj-$(CONFIG_INTEL_RDT) += intel_rdt_ctrlmondata.o intel_rdt_pseudo_lock.o obj-$(CONFIG_X86_MCE) += mcheck/ obj-$(CONFIG_MTRR) += mtrr/
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c new file +113 -0
113 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Utilities to restrict/restore access to specific files
x86/intel_rdt: Utilities to restrict/restore access to specific files
Commit-ID: 2a5d76a4fc6469ea9dd6f02fcd4dad3a129bc1c0 Gitweb: https://git.kernel.org/tip/2a5d76a4fc6469ea9dd6f02fcd4dad3a129bc1c0 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:13 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:46 +0200
x86/intel_rdt: Utilities to restrict/restore access to specific files
In support of Cache Pseudo-Locking we need to restrict access to specific resctrl files to protect the state of a resource group used for pseudo-locking from being changed in unsupported ways.
Introduce two utilities that can be used to either restrict or restore the access to all files irrelevant to cache pseudo-locking when pseudo-locking in progress for the resource group.
At this time introduce a new source file, intel_rdt_pseudo_lock.c, that will contain most of the code related to cache pseudo-locking.
Temporarily mark these new functions as unused to silence compile warnings until they are used.
--- arch/x86/kernel/cpu/Makefile | 3 +- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 113 ++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/Makefile +2 -1
@@ -35,7 +35,8 @@ obj-$(CONFIG_CPU_SUP_CENTAUR) += centaur.o obj-$(CONFIG_CPU_SUP_TRANSMETA_32) += transmeta.o obj-$(CONFIG_CPU_SUP_UMC_32) += umc.o -obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o intel_rdt_ctrlmondata.o +obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o +obj-$(CONFIG_INTEL_RDT) += intel_rdt_ctrlmondata.o intel_rdt_pseudo_lock.o obj-$(CONFIG_X86_MCE) += mcheck/ obj-$(CONFIG_MTRR) += mtrr/
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c new file +113 -0
113 lines not on this page · raw message
PATCH v7 23/41 x86/intel_rdt: Add check to determine if monitoring in progress
In reply to Reinette ChatreWhen a resource group is pseudo-locked it is orphaned without a class of service associated with it. We thus do not want any monitoring in progress on a resource group that will be used for pseudo-locking.
Introduce a test that can be used to determine if pseudo-locking in progress on a resource group. Temporarily mark it as unused to avoid compile warnings until it is used.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +13 -0
@@ -13,6 +13,19 @@#include "intel_rdt.h" +/** +* rdtgroup_monitor_in_progress - Test if monitoring in progress +* @r: resource group being queried +* +* Return: 1 if monitor groups have been created for this resource +* group, 0 otherwise. +*/ +static int __attribute__ ((unused)) +rdtgroup_monitor_in_progress (struct rdtgroup *rdtgrp) +{ +return !list_empty (&rdtgrp->mon.crdtgrp_list); +} +/** * rdtgroup_locksetup_user_restrict - Restrict user access to group * @rdtgrp: resource group needing access restricted
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Add check to determine if monitoring in progress
x86/intel_rdt: Add check to determine if monitoring in progress
Commit-ID: bbcee99b67c5a8cc4e8037d561be9ed293961fd3 Gitweb: https://git.kernel.org/tip/bbcee99b67c5a8cc4e8037d561be9ed293961fd3 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:14 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:47 +0200
x86/intel_rdt: Add check to determine if monitoring in progress
When a resource group is pseudo-locked it is orphaned without a class of service associated with it. We thus do not want any monitoring in progress on a resource group that will be used for pseudo-locking.
Introduce a test that can be used to determine if pseudo-locking in progress on a resource group. Temporarily mark it as unused to avoid compile warnings until it is used.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +13 -0
@@ -13,6 +13,19 @@#include "intel_rdt.h" +/** +* rdtgroup_monitor_in_progress - Test if monitoring in progress +* @r: resource group being queried +* +* Return: 1 if monitor groups have been created for this resource +* group, 0 otherwise. +*/ +static int __attribute__ ((unused)) +rdtgroup_monitor_in_progress (struct rdtgroup *rdtgrp) +{ +return !list_empty (&rdtgrp->mon.crdtgrp_list); +} +/** * rdtgroup_locksetup_user_restrict - Restrict user access to group * @rdtgrp: resource group needing access restricted
PATCH v7 24/41 x86/intel_rdt: Introduce pseudo-locked region
In reply to Reinette ChatreA pseudo-locked region is introduced representing an instance of a pseudo-locked cache region. Each cache instance (domain) can support one pseudo-locked region. Similarly a resource group can be used for one pseudo-locked region.
Include a pointer to a pseudo-locked region from the domain and resource group structures.
--- arch/x86/kernel/cpu/intel_rdt.h | 64 +++++++++++++++++++++------------ 1 file changed, 41 insertions(+), 23 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +41 -23
@@ -122,6 +122,20 @@ struct mongroup {u32 rmid; }; +/** +* struct pseudo_lock_region - pseudo-lock region information +* @r: RDT resource to which this pseudo-locked region +* belongs +* @d: RDT domain to which this pseudo-locked region +* belongs +* @cbm: bitmask of the pseudo-locked region +*/ +struct pseudo_lock_region { +struct rdt_resource *r; +struct rdt_domain *d; +u32 cbm; +}; +/** * struct rdtgroup - store rdtgroup's data in resctrl file system. * @kn: kernfs node
@@ -135,17 +149,19 @@ struct mongroup {* monitor only or ctrl_mon group * @mon: mongroup related data * @mode: mode of resource group +* @plr: pseudo-locked region */ struct rdtgroup { -struct kernfs_node *kn; -struct list_head rdtgroup_list; -u32 closid; -struct cpumask cpu_mask; -int flags; -atomic_t waitcount; -enum rdt_group_type type; -struct mongroup mon; -enum rdtgrp_mode mode; +struct kernfs_node *kn; +struct list_head rdtgroup_list; +u32 closid; +struct cpumask cpu_mask; +int flags; +atomic_t waitcount; +enum rdt_group_type type; +struct mongroup mon; +enum rdtgrp_mode mode; +struct pseudo_lock_region *plr; };/* rdtgroup.flags */
@@ -246,22 +262,24 @@ struct mbm_state {* @mbps_val: When mba_sc is enabled, this holds the bandwidth in MBps * @new_ctrl: new ctrl value to be loaded * @have_new_ctrl: did user provide new_ctrl for this domain +* @plr: pseudo-locked region (if any) associated with domain */ struct rdt_domain { -struct list_head list; -int id; -struct cpumask cpu_mask; -unsigned long *rmid_busy_llc; -struct mbm_state *mbm_total; -struct mbm_state *mbm_local; -struct delayed_work mbm_over; -struct delayed_work cqm_limbo; -int mbm_work_cpu; -int cqm_work_cpu; -u32 *ctrl_val; -u32 *mbps_val; -u32 new_ctrl; -bool have_new_ctrl; +struct list_head list; +int id; +struct cpumask cpu_mask; +unsigned long *rmid_busy_llc; +struct mbm_state *mbm_total; +struct mbm_state *mbm_local; +struct delayed_work mbm_over; +struct delayed_work cqm_limbo; +int mbm_work_cpu; +int cqm_work_cpu; +u32 *ctrl_val; +u32 *mbps_val; +u32 new_ctrl; +bool have_new_ctrl; +struct pseudo_lock_region *plr; };/**
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Introduce pseudo-locked region
x86/intel_rdt: Introduce pseudo-locked region
Commit-ID: e8140a2d13d429364f18ca41b1e4960708c3a40e Gitweb: https://git.kernel.org/tip/e8140a2d13d429364f18ca41b1e4960708c3a40e Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:15 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:47 +0200
x86/intel_rdt: Introduce pseudo-locked region
A pseudo-locked region is introduced representing an instance of a pseudo-locked cache region. Each cache instance (domain) can support one pseudo-locked region. Similarly a resource group can be used for one pseudo-locked region.
Include a pointer to a pseudo-locked region from the domain and resource group structures.
--- arch/x86/kernel/cpu/intel_rdt.h | 64 ++++++++++++++++++++++++++--------------- 1 file changed, 41 insertions(+), 23 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +41 -23
@@ -122,6 +122,20 @@ struct mongroup {u32 rmid; }; +/** +* struct pseudo_lock_region - pseudo-lock region information +* @r: RDT resource to which this pseudo-locked region +* belongs +* @d: RDT domain to which this pseudo-locked region +* belongs +* @cbm: bitmask of the pseudo-locked region +*/ +struct pseudo_lock_region { +struct rdt_resource *r; +struct rdt_domain *d; +u32 cbm; +}; +/** * struct rdtgroup - store rdtgroup's data in resctrl file system. * @kn: kernfs node
@@ -135,17 +149,19 @@ struct mongroup {* monitor only or ctrl_mon group * @mon: mongroup related data * @mode: mode of resource group +* @plr: pseudo-locked region */ struct rdtgroup { -struct kernfs_node *kn; -struct list_head rdtgroup_list; -u32 closid; -struct cpumask cpu_mask; -int flags; -atomic_t waitcount; -enum rdt_group_type type; -struct mongroup mon; -enum rdtgrp_mode mode; +struct kernfs_node *kn; +struct list_head rdtgroup_list; +u32 closid; +struct cpumask cpu_mask; +int flags; +atomic_t waitcount; +enum rdt_group_type type; +struct mongroup mon; +enum rdtgrp_mode mode; +struct pseudo_lock_region *plr; };/* rdtgroup.flags */
@@ -246,22 +262,24 @@ struct mbm_state {* @mbps_val: When mba_sc is enabled, this holds the bandwidth in MBps * @new_ctrl: new ctrl value to be loaded * @have_new_ctrl: did user provide new_ctrl for this domain +* @plr: pseudo-locked region (if any) associated with domain */ struct rdt_domain { -struct list_head list; -int id; -struct cpumask cpu_mask; -unsigned long *rmid_busy_llc; -struct mbm_state *mbm_total; -struct mbm_state *mbm_local; -struct delayed_work mbm_over; -struct delayed_work cqm_limbo; -int mbm_work_cpu; -int cqm_work_cpu; -u32 *ctrl_val; -u32 *mbps_val; -u32 new_ctrl; -bool have_new_ctrl; +struct list_head list; +int id; +struct cpumask cpu_mask; +unsigned long *rmid_busy_llc; +struct mbm_state *mbm_total; +struct mbm_state *mbm_local; +struct delayed_work mbm_over; +struct delayed_work cqm_limbo; +int mbm_work_cpu; +int cqm_work_cpu; +u32 *ctrl_val; +u32 *mbps_val; +u32 new_ctrl; +bool have_new_ctrl; +struct pseudo_lock_region *plr; };/**
PATCH v7 25/41 x86/intel_rdt: Support enter/exit of locksetup mode
In reply to Reinette ChatreThe locksetup mode is the way in which the user communicates that the resource group will be used for a pseudo-locked region. Locksetup mode should thus ensure that all restrictions on a resource group are met before locksetup mode can be entered. The resource group should also be configured to ensure that it cannot be modified in unsupported ways when a pseudo-locked region.
Introduce the support where the request for entering locksetup mode can be validated. This includes: CDP is not active, no cpus or tasks are assigned to the resource group, monitoring is not in progress on the resource group. Once the resource group is determined ready for a pseudo-locked region it is configured to not allow future changes to these properties.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 187 +++++++++++++++++++- 2 files changed, 183 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -501,6 +501,8 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d, u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);int rdtgroup_tasks_assigned (struct rdtgroup *r); +int rdtgroup_locksetup_enter (struct rdtgroup *rdtgrp); +int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +181 -6
216 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Support enter/exit of locksetup mode
x86/intel_rdt: Support enter/exit of locksetup mode
Commit-ID: 63657c1cdf89a37d3b69471ad4e5b55c77e86d3f Gitweb: https://git.kernel.org/tip/63657c1cdf89a37d3b69471ad4e5b55c77e86d3f Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:16 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:47 +0200
x86/intel_rdt: Support enter/exit of locksetup mode
The locksetup mode is the way in which the user communicates that the resource group will be used for a pseudo-locked region. Locksetup mode should thus ensure that all restrictions on a resource group are met before locksetup mode can be entered. The resource group should also be configured to ensure that it cannot be modified in unsupported ways when a pseudo-locked region.
Introduce the support where the request for entering locksetup mode can be validated. This includes: CDP is not active, no cpus or tasks are assigned to the resource group, monitoring is not in progress on the resource group. Once the resource group is determined ready for a pseudo-locked region it is configured to not allow future changes to these properties.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 187 +++++++++++++++++++++++++++- 2 files changed, 183 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -501,6 +501,8 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_domain *d, u32 cbm);enum rdtgrp_mode rdtgroup_mode_by_closid (int closid);int rdtgroup_tasks_assigned (struct rdtgroup *r); +int rdtgroup_locksetup_enter (struct rdtgroup *rdtgrp); +int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +181 -6
216 lines not on this page · raw message
PATCH v7 26/41 x86/intel_rdt: Enable entering of pseudo-locksetup mode
In reply to Reinette ChatreThe user can request entering pseudo-locksetup mode by writing "pseudo-locksetup" to the mode file. Act on this request as well as support switching from a pseudo-locksetup mode (before pseudo-locked mode was entered). It is not supported to modify the mode once pseudo-locked mode has been entered.
The schemata reflects the new mode by adding "uninitialized" to all resources. The size resctrl file reports zero for all cache domains in support of the uninitialized nature. Since there are no users of this class of service its allocations can be ignored when searching for appropriate default allocations for new resource groups. For the same reason resource groups in pseudo-locksetup mode are not considered when testing if new resource groups may overlap.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 16 +++++--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 41 ++++++++++++++++++--- 2 files changed, 47 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +11 -5
@@ -156,7 +156,8 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d) }if (rdtgroup_cbm_overlaps (r, d, cbm_val, rdtgrp->closid, false)) { -if (rdtgrp->mode ==RDT_MODE_EXCLUSIVE ) { +if (rdtgrp->mode ==RDT_MODE_EXCLUSIVE || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {rdt_last_cmd_printf ("overlaps with other group\n" );return -EINVAL ; }
@@ -356,10 +357,15 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, rdtgrp =rdtgroup_kn_lock_live (of->kn);if (rdtgrp) { - closid = rdtgrp->closid; -for_each_alloc_enabled_rdt_resource (r) { -if (closid < r->num_closid) -show_doms (s, r, closid); +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +for_each_alloc_enabled_rdt_resource (r) +seq_printf (s,"%s:uninitialized\n" , r->name); + }else { + closid = rdtgrp->closid; +for_each_alloc_enabled_rdt_resource (r) { +if (closid < r->num_closid) +show_doms (s, r, closid); + } } }else { ret = -ENOENT ;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +36 -5
76 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Enable entering of pseudo-locksetup mode
x86/intel_rdt: Enable entering of pseudo-locksetup mode
Commit-ID: dfe9674b04ff6b819f9105650a008d164d81725e Gitweb: https://git.kernel.org/tip/dfe9674b04ff6b819f9105650a008d164d81725e Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:17 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:48 +0200
x86/intel_rdt: Enable entering of pseudo-locksetup mode
The user can request entering pseudo-locksetup mode by writing "pseudo-locksetup" to the mode file. Act on this request as well as support switching from a pseudo-locksetup mode (before pseudo-locked mode was entered). It is not supported to modify the mode once pseudo-locked mode has been entered.
The schemata reflects the new mode by adding "uninitialized" to all resources. The size resctrl file reports zero for all cache domains in support of the uninitialized nature. Since there are no users of this class of service its allocations can be ignored when searching for appropriate default allocations for new resource groups. For the same reason resource groups in pseudo-locksetup mode are not considered when testing if new resource groups may overlap.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 16 +++++++---- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 41 +++++++++++++++++++++++++---- 2 files changed, 47 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +11 -5
@@ -156,7 +156,8 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d) }if (rdtgroup_cbm_overlaps (r, d, cbm_val, rdtgrp->closid, false)) { -if (rdtgrp->mode ==RDT_MODE_EXCLUSIVE ) { +if (rdtgrp->mode ==RDT_MODE_EXCLUSIVE || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {rdt_last_cmd_printf ("overlaps with other group\n" );return -EINVAL ; }
@@ -356,10 +357,15 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of, rdtgrp =rdtgroup_kn_lock_live (of->kn);if (rdtgrp) { - closid = rdtgrp->closid; -for_each_alloc_enabled_rdt_resource (r) { -if (closid < r->num_closid) -show_doms (s, r, closid); +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +for_each_alloc_enabled_rdt_resource (r) +seq_printf (s,"%s:uninitialized\n" , r->name); + }else { + closid = rdtgrp->closid; +for_each_alloc_enabled_rdt_resource (r) { +if (closid < r->num_closid) +show_doms (s, r, closid); + } } }else { ret = -ENOENT ;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +36 -5
76 lines not on this page · raw message
PATCH v7 27/41 x86/intel_rdt: Split resource group removal in two
In reply to Reinette ChatreResource groups used for pseudo-locking do not require the same work on removal as the other resource groups.
The resource group removal is split in two in preparation for support of pseudo-locking resource groups. A single re-ordering occurs - the setting of the rdtgrp flag is moved to later. This flag is not used by any of the code between its original and new location.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 25 +++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +16 -9
@@ -2629,6 +2629,21 @@ static int rdtgroup_rmdir_mon(struct kernfs_node *kn, struct rdtgroup *rdtgrp,return 0 ; } +static int rdtgroup_ctrl_remove (struct kernfs_node *kn, +struct rdtgroup *rdtgrp) +{ + rdtgrp->flags =RDT_DELETED ; +list_del (&rdtgrp->rdtgroup_list); + +/* +* one extra hold on this, will drop when we kfree(rdtgrp) +* in rdtgroup_kn_unlock() +*/ +kernfs_get (kn); +kernfs_remove (rdtgrp->kn); +return 0 ; +} +static int rdtgroup_rmdir_ctrl (struct kernfs_node *kn,struct rdtgroup *rdtgrp,cpumask_var_t tmpmask) {
@@ -2654,7 +2669,6 @@ static int rdtgroup_rmdir_ctrl(struct kernfs_node *kn, struct rdtgroup *rdtgrp,cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); - rdtgrp->flags =RDT_DELETED ;closid_free (rdtgrp->closid);free_rmid (rdtgrp->mon.rmid);
@@ -2663,14 +2677,7 @@ static int rdtgroup_rmdir_ctrl(struct kernfs_node *kn, struct rdtgroup *rdtgrp,*/ free_all_child_rdtgrp (rdtgrp); -list_del (&rdtgrp->rdtgroup_list); - -/* -* one extra hold on this, will drop when we kfree(rdtgrp) -* in rdtgroup_kn_unlock() -*/ -kernfs_get (kn); -kernfs_remove (rdtgrp->kn); +rdtgroup_ctrl_remove (kn, rdtgrp);return 0 ; }
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Split resource group removal in two
x86/intel_rdt: Split resource group removal in two
Commit-ID: 17eafd076291ed23eeb17b28132fa33b0688bc57 Gitweb: https://git.kernel.org/tip/17eafd076291ed23eeb17b28132fa33b0688bc57 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:18 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:48 +0200
x86/intel_rdt: Split resource group removal in two
Resource groups used for pseudo-locking do not require the same work on removal as the other resource groups.
The resource group removal is split in two in preparation for support of pseudo-locking resource groups. A single re-ordering occurs - the setting of the rdtgrp flag is moved to later. This flag is not used by any of the code between its original and new location.
--- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +16 -9
@@ -2620,6 +2620,21 @@ static int rdtgroup_rmdir_mon(struct kernfs_node *kn, struct rdtgroup *rdtgrp,return 0 ; } +static int rdtgroup_ctrl_remove (struct kernfs_node *kn, +struct rdtgroup *rdtgrp) +{ + rdtgrp->flags =RDT_DELETED ; +list_del (&rdtgrp->rdtgroup_list); + +/* +* one extra hold on this, will drop when we kfree(rdtgrp) +* in rdtgroup_kn_unlock() +*/ +kernfs_get (kn); +kernfs_remove (rdtgrp->kn); +return 0 ; +} +static int rdtgroup_rmdir_ctrl (struct kernfs_node *kn,struct rdtgroup *rdtgrp,cpumask_var_t tmpmask) {
@@ -2645,7 +2660,6 @@ static int rdtgroup_rmdir_ctrl(struct kernfs_node *kn, struct rdtgroup *rdtgrp,cpumask_or (tmpmask, tmpmask, &rdtgrp->cpu_mask);update_closid_rmid (tmpmask,NULL ); - rdtgrp->flags =RDT_DELETED ;closid_free (rdtgrp->closid);free_rmid (rdtgrp->mon.rmid);
@@ -2654,14 +2668,7 @@ static int rdtgroup_rmdir_ctrl(struct kernfs_node *kn, struct rdtgroup *rdtgrp,*/ free_all_child_rdtgrp (rdtgrp); -list_del (&rdtgrp->rdtgroup_list); - -/* -* one extra hold on this, will drop when we kfree(rdtgrp) -* in rdtgroup_kn_unlock() -*/ -kernfs_get (kn); -kernfs_remove (rdtgrp->kn); +rdtgroup_ctrl_remove (kn, rdtgrp);return 0 ; }
PATCH v7 28/41 x86/intel_rdt: Add utilities to test pseudo-locked region possibility
In reply to Reinette ChatreA pseudo-locked region does not have a class of service associated with it and thus not tracked in the array of control values maintained as part of the domain. Even so, when the user provides a new bitmask for another resource group it needs to be checked for interference with existing pseudo-locked regions.
Additionally only one pseudo-locked region can be created in any cache hierarchy.
Introduce two utilities in support of above scenarios: (1) a utility that can be used to test if a given capacity bitmask overlaps with any pseudo-locked regions associated with a particular cache instance, (2) a utility that can be used to test if a pseudo-locked region exists within a particular cache hierarchy.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 74 +++++++++++++++++++++ 2 files changed, 76 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -503,6 +503,8 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);int rdtgroup_tasks_assigned (struct rdtgroup *r);int rdtgroup_locksetup_enter (struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp); +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm); +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +74 -0
77 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Add utilities to test pseudo-locked region possibility
x86/intel_rdt: Add utilities to test pseudo-locked region possibility
Commit-ID: 72d505056604a305a4fcd8b268d2f6e979e17023 Gitweb: https://git.kernel.org/tip/72d505056604a305a4fcd8b268d2f6e979e17023 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:19 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:49 +0200
x86/intel_rdt: Add utilities to test pseudo-locked region possibility
A pseudo-locked region does not have a class of service associated with it and thus not tracked in the array of control values maintained as part of the domain. Even so, when the user provides a new bitmask for another resource group it needs to be checked for interference with existing pseudo-locked regions.
Additionally only one pseudo-locked region can be created in any cache hierarchy.
Introduce two utilities in support of above scenarios: (1) a utility that can be used to test if a given capacity bitmask overlaps with any pseudo-locked regions associated with a particular cache instance, (2) a utility that can be used to test if a pseudo-locked region exists within a particular cache hierarchy.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 74 +++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -503,6 +503,8 @@ enum rdtgrp_mode rdtgroup_mode_by_closid(int closid);int rdtgroup_tasks_assigned (struct rdtgroup *r);int rdtgroup_locksetup_enter (struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp); +bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm); +bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +74 -0
77 lines not on this page · raw message
PATCH v7 29/41 x86/intel_rdt: Discover supported platforms via prefetch disable bits
In reply to Reinette ChatreKnowing the model specific prefetch disable bits is required to support cache pseudo-locking because the hardware prefetchers need to be disabled when the kernel memory is pseudo-locked to cache. We add these bits only for platforms known to support cache pseudo-locking.
When the user requests locksetup mode to be entered it will fail if the prefetch disabling bits are not known for the platform.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 75 +++++++++++++++++++++ 1 file changed, 75 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +75 -0
@@ -12,8 +12,73 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/slab.h> +#include <asm/intel-family.h> #include "intel_rdt.h" +/* +* MSR_MISC_FEATURE_CONTROL register enables the modification of hardware +* prefetcher state. Details about this register can be found in the MSR +* tables for specific platforms found in Intel's SDM. +*/ +#define MSR_MISC_FEATURE_CONTROL 0x000001a4 + +/* +* The bits needed to disable hardware prefetching varies based on the +* platform. During initialization we will discover which bits to use. +*/ +static u64 prefetch_disable_bits; + +/** +* get_prefetch_disable_bits - prefetch disable bits of supported platforms +* +* Capture the list of platforms that have been validated to support +* pseudo-locking. This includes testing to ensure pseudo-locked regions +* with low cache miss rates can be created under variety of load conditions +* as well as that these pseudo-locked regions can maintain their low cache +* miss rates under variety of load conditions for significant lengths of time. +* +* After a platform has been validated to support pseudo-locking its +* hardware prefetch disable bits are included here as they are documented +* in the SDM. +* +* Return: +* If platform is supported, the bits to disable hardware prefetchers, 0 +* if platform is not supported. +*/ +static u64 get_prefetch_disable_bits (void ) +{ +if (boot_cpu_data.x86_vendor !=X86_VENDOR_INTEL || + boot_cpu_data.x86 !=6 ) +return 0 ; + +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_BROADWELL_X : +/* +* SDM defines bits of MSR_MISC_FEATURE_CONTROL register +* as: +* 0 L2 Hardware Prefetcher Disable (R/W) +* 1 L2 Adjacent Cache Line Prefetcher Disable (R/W) +* 2 DCU Hardware Prefetcher Disable (R/W) +* 3 DCU IP Prefetcher Disable (R/W) +* 63:4 Reserved +*/ +return 0xF ; +case INTEL_FAM6_ATOM_GOLDMONT : +case INTEL_FAM6_ATOM_GEMINI_LAKE : +/* +* SDM defines bits of MSR_MISC_FEATURE_CONTROL register +* as: +* 0 L2 Hardware Prefetcher Disable (R/W) +* 1 Reserved +* 2 DCU Hardware Prefetcher Disable (R/W) +* 63:3 Reserved +*/ +return 0x5 ; + } + +return 0 ; +} +/** * pseudo_lock_init - Initialize a pseudo-lock region * @rdtgrp: resource group to which new pseudo-locked region will belong
@@ -225,6 +290,16 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)return -EINVAL ; } +/* +* Not knowing the bits to disable prefetching implies that this +* platform does not support Cache Pseudo-Locking. +*/ + prefetch_disable_bits =get_prefetch_disable_bits (); +if (prefetch_disable_bits ==0 ) { +rdt_last_cmd_puts ("pseudo-locking not supported\n" ); +return -EINVAL ; + } +if (rdtgroup_monitor_in_progress (rdtgrp)) {rdt_last_cmd_puts ("monitoring in progress\n" );return -EINVAL ;
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Discover supported platforms via prefetch disable bits
x86/intel_rdt: Discover supported platforms via prefetch disable bits
Commit-ID: f2a177292bd052ce12ac453d2ceeb083fe07718a Gitweb: https://git.kernel.org/tip/f2a177292bd052ce12ac453d2ceeb083fe07718a Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:20 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:49 +0200
x86/intel_rdt: Discover supported platforms via prefetch disable bits
Knowing the model specific prefetch disable bits is required to support cache pseudo-locking because the hardware prefetchers need to be disabled when the kernel memory is pseudo-locked to cache. We add these bits only for platforms known to support cache pseudo-locking.
When the user requests locksetup mode to be entered it will fail if the prefetch disabling bits are not known for the platform.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 75 +++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +75 -0
@@ -12,8 +12,73 @@#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt#include <linux/slab.h> +#include <asm/intel-family.h> #include "intel_rdt.h" +/* +* MSR_MISC_FEATURE_CONTROL register enables the modification of hardware +* prefetcher state. Details about this register can be found in the MSR +* tables for specific platforms found in Intel's SDM. +*/ +#define MSR_MISC_FEATURE_CONTROL 0x000001a4 + +/* +* The bits needed to disable hardware prefetching varies based on the +* platform. During initialization we will discover which bits to use. +*/ +static u64 prefetch_disable_bits; + +/** +* get_prefetch_disable_bits - prefetch disable bits of supported platforms +* +* Capture the list of platforms that have been validated to support +* pseudo-locking. This includes testing to ensure pseudo-locked regions +* with low cache miss rates can be created under variety of load conditions +* as well as that these pseudo-locked regions can maintain their low cache +* miss rates under variety of load conditions for significant lengths of time. +* +* After a platform has been validated to support pseudo-locking its +* hardware prefetch disable bits are included here as they are documented +* in the SDM. +* +* Return: +* If platform is supported, the bits to disable hardware prefetchers, 0 +* if platform is not supported. +*/ +static u64 get_prefetch_disable_bits (void ) +{ +if (boot_cpu_data.x86_vendor !=X86_VENDOR_INTEL || + boot_cpu_data.x86 !=6 ) +return 0 ; + +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_BROADWELL_X : +/* +* SDM defines bits of MSR_MISC_FEATURE_CONTROL register +* as: +* 0 L2 Hardware Prefetcher Disable (R/W) +* 1 L2 Adjacent Cache Line Prefetcher Disable (R/W) +* 2 DCU Hardware Prefetcher Disable (R/W) +* 3 DCU IP Prefetcher Disable (R/W) +* 63:4 Reserved +*/ +return 0xF ; +case INTEL_FAM6_ATOM_GOLDMONT : +case INTEL_FAM6_ATOM_GEMINI_LAKE : +/* +* SDM defines bits of MSR_MISC_FEATURE_CONTROL register +* as: +* 0 L2 Hardware Prefetcher Disable (R/W) +* 1 Reserved +* 2 DCU Hardware Prefetcher Disable (R/W) +* 63:3 Reserved +*/ +return 0x5 ; + } + +return 0 ; +} +/** * pseudo_lock_init - Initialize a pseudo-lock region * @rdtgrp: resource group to which new pseudo-locked region will belong
@@ -225,6 +290,16 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp)return -EINVAL ; } +/* +* Not knowing the bits to disable prefetching implies that this +* platform does not support Cache Pseudo-Locking. +*/ + prefetch_disable_bits =get_prefetch_disable_bits (); +if (prefetch_disable_bits ==0 ) { +rdt_last_cmd_puts ("pseudo-locking not supported\n" ); +return -EINVAL ; + } +if (rdtgroup_monitor_in_progress (rdtgrp)) {rdt_last_cmd_puts ("monitoring in progress\n" );return -EINVAL ;
PATCH v7 30/41 x86/intel_rdt: Pseudo-lock region creation/removal core
In reply to Reinette ChatreThe user requests a pseudo-locked region by providing a schemata to a resource group that is in the pseudo-locksetup mode. This is the functionality that consumes the parsed user data and creates the pseudo-locked region.
First, required information is deduced from user provided data. This includes, how much memory does the requested bitmask represent, which CPU the requested region is associated with, and what is the cache line size of that cache (to learn the stride needed for locking). Second, a contiguous block of memory matching the requested bitmask is allocated.
Finally, pseudo-locking is performed. The resource group already has the allocation that reflects the requested bitmask. With this class of service active and interference minimized, the allocated memory is loaded into the cache.
--- arch/x86/kernel/cpu/intel_rdt.h | 17 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 337 +++++++++++++++++++- 2 files changed, 353 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt.h +17 -0
@@ -129,11 +129,26 @@ struct mongroup {* @d: RDT domain to which this pseudo-locked region * belongs * @cbm: bitmask of the pseudo-locked region +* @lock_thread_wq: waitqueue used to wait on the pseudo-locking thread +* completion +* @thread_done: variable used by waitqueue to test if pseudo-locking +* thread completed +* @cpu: core associated with the cache on which the setup code +* will be run +* @line_size: size of the cache lines +* @size: size of pseudo-locked region in bytes +* @kmem: the kernel memory associated with pseudo-locked region */ struct pseudo_lock_region {struct rdt_resource *r;struct rdt_domain *d;u32 cbm; +wait_queue_head_t lock_thread_wq; +int thread_done; +int cpu; +unsigned int line_size; +unsigned int size; +void *kmem; };/**
@@ -505,6 +520,8 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm);bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d); +int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp); +void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +336 -1
380 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Pseudo-lock region creation/removal core
x86/intel_rdt: Pseudo-lock region creation/removal core
Commit-ID: 018961ae5579016d46ee03587c0ecf673821eedb Gitweb: https://git.kernel.org/tip/018961ae5579016d46ee03587c0ecf673821eedb Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:21 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:49 +0200
x86/intel_rdt: Pseudo-lock region creation/removal core
The user requests a pseudo-locked region by providing a schemata to a resource group that is in the pseudo-locksetup mode. This is the functionality that consumes the parsed user data and creates the pseudo-locked region.
First, required information is deduced from user provided data. This includes, how much memory does the requested bitmask represent, which CPU the requested region is associated with, and what is the cache line size of that cache (to learn the stride needed for locking). Second, a contiguous block of memory matching the requested bitmask is allocated.
Finally, pseudo-locking is performed. The resource group already has the allocation that reflects the requested bitmask. With this class of service active and interference minimized, the allocated memory is loaded into the cache.
--- arch/x86/kernel/cpu/intel_rdt.h | 17 ++ arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 337 +++++++++++++++++++++++++++- 2 files changed, 353 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/intel_rdt.h +17 -0
@@ -129,11 +129,26 @@ struct mongroup {* @d: RDT domain to which this pseudo-locked region * belongs * @cbm: bitmask of the pseudo-locked region +* @lock_thread_wq: waitqueue used to wait on the pseudo-locking thread +* completion +* @thread_done: variable used by waitqueue to test if pseudo-locking +* thread completed +* @cpu: core associated with the cache on which the setup code +* will be run +* @line_size: size of the cache lines +* @size: size of pseudo-locked region in bytes +* @kmem: the kernel memory associated with pseudo-locked region */ struct pseudo_lock_region {struct rdt_resource *r;struct rdt_domain *d;u32 cbm; +wait_queue_head_t lock_thread_wq; +int thread_done; +int cpu; +unsigned int line_size; +unsigned int size; +void *kmem; };/**
@@ -505,6 +520,8 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm);bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d); +int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp); +void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);int update_domains (struct rdt_resource *r,int closid);void closid_free (int closid);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +336 -1
380 lines not on this page · raw message
PATCH v7 31/41 x86/intel_rdt: Support creation/removal of pseudo-locked region
In reply to Reinette ChatreThe user triggers the creation of a pseudo-locked region when writing a valid schemata to the schemata file of a resource group in the pseudo-locksetup mode.
A valid schemata is one that: (1) does not overlap with any other resource group, (2) does not involve a cache that already contains a pseudo-locked region within its hierarchy.
After a valid schemata is parsed the system is programmed to associate the to be pseudo-lock bitmask with the closid associated with the resource group. With the system set up the pseudo-locked region can be created.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 42 +++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 25 +++++++++--- 2 files changed, 62 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +42 -0
@@ -143,9 +143,26 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)return -EINVAL ; } +/* +* Cannot set up more than one pseudo-locked region in a cache +* hierarchy. +*/ +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP && +rdtgroup_pseudo_locked_in_hierarchy (d)) { +rdt_last_cmd_printf ("pseudo-locked region in hierarchy\n" ); +return -EINVAL ; + } +if (!cbm_validate (data->buf, &cbm_val, r))return -EINVAL ; +if ((rdtgrp->mode ==RDT_MODE_EXCLUSIVE || + rdtgrp->mode ==RDT_MODE_SHAREABLE ) && +rdtgroup_cbm_overlaps_pseudo_locked (d, cbm_val)) { +rdt_last_cmd_printf ("CBM overlaps with pseudo-locked region\n" ); +return -EINVAL ; + } +/* * The CBM may not overlap with the CBM of another closid if * either is exclusive.
@@ -199,6 +216,21 @@ static int parse_line(char *line, struct rdt_resource *r, data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, r, d))return -EINVAL ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +/* +* In pseudo-locking setup mode and just +* parsed a valid CBM that should be +* pseudo-locked. Only one locked region per +* resource group and domain so just do +* the required initialization for single +* region and return. +*/ + rdtgrp->plr->r = r; + rdtgrp->plr->d = d; + rdtgrp->plr->cbm = d->new_ctrl; + d->plr = rdtgrp->plr; +return 0 ; + } goto next; } }
@@ -322,6 +354,16 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, goto out; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +/* +* If pseudo-locking fails we keep the resource group in +* mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service +* active and updated for just the domain the pseudo-locked +* region was requested for. +*/ + ret =rdtgroup_pseudo_lock_create (rdtgrp); + } + out:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +20 -5
@@ -1771,6 +1771,9 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn)if (atomic_dec_and_test (&rdtgrp->waitcount) && (rdtgrp->flags &RDT_DELETED )) { +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) +rdtgroup_pseudo_lock_remove (rdtgrp);kernfs_unbreak_active_protection (kn);kernfs_put (rdtgrp->kn);kfree (rdtgrp);
@@ -2011,6 +2014,10 @@ static void rmdir_all_sub(void)if (rdtgrp == &rdtgroup_default)continue ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) +rdtgroup_pseudo_lock_remove (rdtgrp); +/* * Give any CPUs back to the default group. We cannot copy * cpu_online_mask because a CPU might have executed the
@@ -2322,6 +2329,8 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) d->new_ctrl |= *ctrl; } } +if (d->plr && d->plr->cbm >0 ) + used_b |= d->plr->cbm; unused_b = used_b ^ (BIT_MASK (r->cache.cbm_len) -1 ); unused_b &=BIT_MASK (r->cache.cbm_len) -1 ; d->new_ctrl |= unused_b;
@@ -2705,13 +2714,19 @@ static int rdtgroup_rmdir(struct kernfs_node *kn)* If the rdtgroup is a mon group and parent directory * is a valid "mon_groups" directory, remove the mon group. */ -if (rdtgrp->type ==RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn) - ret =rdtgroup_rmdir_ctrl (kn, rdtgrp, tmpmask); -else if (rdtgrp->type ==RDTMON_GROUP && -is_mon_groups (parent_kn, kn->name)) +if (rdtgrp->type ==RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn) { +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { + ret =rdtgroup_ctrl_remove (kn, rdtgrp); + }else { + ret =rdtgroup_rmdir_ctrl (kn, rdtgrp, tmpmask); + } + }else if (rdtgrp->type ==RDTMON_GROUP && +is_mon_groups (parent_kn, kn->name)) { ret =rdtgroup_rmdir_mon (kn, rdtgrp, tmpmask); -else + }else { ret = -EPERM ; + } out:rdtgroup_kn_unlock (kn);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Support creation/removal of pseudo-locked region
x86/intel_rdt: Support creation/removal of pseudo-locked region
Commit-ID: e0bdfe8e36f3fbbdc91e70bf927f743ca23917b0 Gitweb: https://git.kernel.org/tip/e0bdfe8e36f3fbbdc91e70bf927f743ca23917b0 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:22 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:50 +0200
x86/intel_rdt: Support creation/removal of pseudo-locked region
The user triggers the creation of a pseudo-locked region when writing a valid schemata to the schemata file of a resource group in the pseudo-locksetup mode.
A valid schemata is one that: (1) does not overlap with any other resource group, (2) does not involve a cache that already contains a pseudo-locked region within its hierarchy.
After a valid schemata is parsed the system is programmed to associate the to be pseudo-lock bitmask with the closid associated with the resource group. With the system set up the pseudo-locked region can be created.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 42 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 25 +++++++++++++---- 2 files changed, 62 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +42 -0
@@ -143,9 +143,26 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)return -EINVAL ; } +/* +* Cannot set up more than one pseudo-locked region in a cache +* hierarchy. +*/ +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP && +rdtgroup_pseudo_locked_in_hierarchy (d)) { +rdt_last_cmd_printf ("pseudo-locked region in hierarchy\n" ); +return -EINVAL ; + } +if (!cbm_validate (data->buf, &cbm_val, r))return -EINVAL ; +if ((rdtgrp->mode ==RDT_MODE_EXCLUSIVE || + rdtgrp->mode ==RDT_MODE_SHAREABLE ) && +rdtgroup_cbm_overlaps_pseudo_locked (d, cbm_val)) { +rdt_last_cmd_printf ("CBM overlaps with pseudo-locked region\n" ); +return -EINVAL ; + } +/* * The CBM may not overlap with the CBM of another closid if * either is exclusive.
@@ -199,6 +216,21 @@ next: data.rdtgrp = rdtgrp;if (r->parse_ctrlval (&data, r, d))return -EINVAL ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +/* +* In pseudo-locking setup mode and just +* parsed a valid CBM that should be +* pseudo-locked. Only one locked region per +* resource group and domain so just do +* the required initialization for single +* region and return. +*/ + rdtgrp->plr->r = r; + rdtgrp->plr->d = d; + rdtgrp->plr->cbm = d->new_ctrl; + d->plr = rdtgrp->plr; +return 0 ; + } goto next; } }
@@ -322,6 +354,16 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of, goto out; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) { +/* +* If pseudo-locking fails we keep the resource group in +* mode RDT_MODE_PSEUDO_LOCKSETUP with its class of service +* active and updated for just the domain the pseudo-locked +* region was requested for. +*/ + ret =rdtgroup_pseudo_lock_create (rdtgrp); + } + out:rdtgroup_kn_unlock (of->kn);return ret ?: nbytes;
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +20 -5
@@ -1771,6 +1771,9 @@ void rdtgroup_kn_unlock(struct kernfs_node *kn)if (atomic_dec_and_test (&rdtgrp->waitcount) && (rdtgrp->flags &RDT_DELETED )) { +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) +rdtgroup_pseudo_lock_remove (rdtgrp);kernfs_unbreak_active_protection (kn);kernfs_put (rdtgrp->kn);kfree (rdtgrp);
@@ -2002,6 +2005,10 @@ static void rmdir_all_sub(void)if (rdtgrp == &rdtgroup_default)continue ; +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) +rdtgroup_pseudo_lock_remove (rdtgrp); +/* * Give any CPUs back to the default group. We cannot copy * cpu_online_mask because a CPU might have executed the
@@ -2313,6 +2320,8 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) d->new_ctrl |= *ctrl; } } +if (d->plr && d->plr->cbm >0 ) + used_b |= d->plr->cbm; unused_b = used_b ^ (BIT_MASK (r->cache.cbm_len) -1 ); unused_b &=BIT_MASK (r->cache.cbm_len) -1 ; d->new_ctrl |= unused_b;
@@ -2696,13 +2705,19 @@ static int rdtgroup_rmdir(struct kernfs_node *kn)* If the rdtgroup is a mon group and parent directory * is a valid "mon_groups" directory, remove the mon group. */ -if (rdtgrp->type ==RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn) - ret =rdtgroup_rmdir_ctrl (kn, rdtgrp, tmpmask); -else if (rdtgrp->type ==RDTMON_GROUP && -is_mon_groups (parent_kn, kn->name)) +if (rdtgrp->type ==RDTCTRL_GROUP && parent_kn == rdtgroup_default.kn) { +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP || + rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { + ret =rdtgroup_ctrl_remove (kn, rdtgrp); + }else { + ret =rdtgroup_rmdir_ctrl (kn, rdtgrp, tmpmask); + } + }else if (rdtgrp->type ==RDTMON_GROUP && +is_mon_groups (parent_kn, kn->name)) { ret =rdtgroup_rmdir_mon (kn, rdtgrp, tmpmask); -else + }else { ret = -EPERM ; + } out:rdtgroup_kn_unlock (kn);
PATCH v7 32/41 x86/intel_rdt: Resctrl files reflect pseudo-locked information
In reply to Reinette ChatreInformation about resources as well as resource groups are contained in a variety of resctrl files. Now that pseudo-locked regions can be created the files can be updated to present appropriate information to the user.
Update the resource group's schemata file to show only the information of the pseudo-locked region.
Update the resource group's size file to show the size in bytes of only the pseudo-locked region.
Update the bit_usage file to use the letter 'P' for all pseudo-locked regions.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 3 ++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 31 +++++++++++++++++---- 2 files changed, 28 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +3 -0
@@ -402,6 +402,9 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {for_each_alloc_enabled_rdt_resource (r)seq_printf (s,"%s:uninitialized\n" , r->name); + }else if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { +seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->r->name, + rdtgrp->plr->d->id, rdtgrp->plr->cbm); }else { closid = rdtgrp->closid;for_each_alloc_enabled_rdt_resource (r) {
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +25 -6
@@ -771,14 +771,16 @@ static int rdt_shareable_bits_show(struct kernfs_open_file *of,* H - currently used by hardware only but available for software use * S - currently used and shareable by software only * E - currently used exclusively by one resource group +* P - currently pseudo-locked by one resource group */ static int rdt_bit_usage_show (struct kernfs_open_file *of,struct seq_file *seq,void *v) {struct rdt_resource *r = of->kn->parent->priv; -u32 sw_shareable, hw_shareable, exclusive; +u32 sw_shareable =0 , hw_shareable =0 ; +u32 exclusive =0 , pseudo_locked =0 ;struct rdt_domain *dom; -int i, hwb, swb, excl; +int i, hwb, swb, excl, psl;enum rdtgrp_mode mode;bool sep = false;u32 *ctrl;
@@ -803,12 +805,15 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,case RDT_MODE_EXCLUSIVE : exclusive |= *ctrl;break ; +case RDT_MODE_PSEUDO_LOCKSETUP :/* -* Temporarily handle pseudo-locking enums -* to silence compile warnings until handling -* added in later patches. +* RDT_MODE_PSEUDO_LOCKSETUP is possible +* here but not included since the CBM +* associated with this CLOSID in this mode +* is not initialized and no task or cpu can be +* assigned this CLOSID. */ -case RDT_MODE_PSEUDO_LOCKSETUP : +break ;case RDT_MODE_PSEUDO_LOCKED :case RDT_NUM_MODES : WARN(1,
@@ -817,9 +822,11 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, } }for (i = r->cache.cbm_len -1 ; i >=0 ; i--) { + pseudo_locked = dom->plr ? dom->plr->cbm :0 ; hwb =test_bit (i, (unsigned long *)&hw_shareable); swb =test_bit (i, (unsigned long *)&sw_shareable); excl =test_bit (i, (unsigned long *)&exclusive); + psl =test_bit (i, (unsigned long *)&pseudo_locked);if (hwb && swb)seq_putc (seq,'X' );else if (hwb && !swb)
@@ -828,6 +835,8 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,seq_putc (seq,'S' ); else if (excl) seq_putc(seq, 'E'); + else if (psl) + seq_putc(seq, 'P'); else /* Unused bits remain */ seq_putc(seq, '0'); }
@@ -1147,6 +1156,15 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,return -ENOENT ; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { +seq_printf (s,"%*s:" , max_name_width, rdtgrp->plr->r->name); + size =rdtgroup_cbm_to_size (rdtgrp->plr->r, + rdtgrp->plr->d, + rdtgrp->plr->cbm); +seq_printf (s,"%d=%u\n" , rdtgrp->plr->d->id, size); + goto out; + } +for_each_alloc_enabled_rdt_resource (r) {seq_printf (s,"%*s:" , max_name_width, r->name);list_for_each_entry (d, &r->domains, list) {
@@ -1164,6 +1182,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,seq_putc (s,'\n' ); } +out:rdtgroup_kn_unlock (of->kn);return 0 ;
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Resctrl files reflect pseudo-locked information
x86/intel_rdt: Resctrl files reflect pseudo-locked information
Commit-ID: f4e80d67a527469245f391976d8665f934a16205 Gitweb: https://git.kernel.org/tip/f4e80d67a527469245f391976d8665f934a16205 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:23 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:50 +0200
x86/intel_rdt: Resctrl files reflect pseudo-locked information
Information about resources as well as resource groups are contained in a variety of resctrl files. Now that pseudo-locked regions can be created the files can be updated to present appropriate information to the user.
Update the resource group's schemata file to show only the information of the pseudo-locked region.
Update the resource group's size file to show the size in bytes of only the pseudo-locked region.
Update the bit_usage file to use the letter 'P' for all pseudo-locked regions.
--- arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 3 +++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 31 +++++++++++++++++++++++------ 2 files changed, 28 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +3 -0
@@ -402,6 +402,9 @@ int rdtgroup_schemata_show(struct kernfs_open_file *of,if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKSETUP ) {for_each_alloc_enabled_rdt_resource (r)seq_printf (s,"%s:uninitialized\n" , r->name); + }else if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { +seq_printf (s,"%s:%d=%x\n" , rdtgrp->plr->r->name, + rdtgrp->plr->d->id, rdtgrp->plr->cbm); }else { closid = rdtgrp->closid;for_each_alloc_enabled_rdt_resource (r) {
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +25 -6
@@ -771,14 +771,16 @@ static int rdt_shareable_bits_show(struct kernfs_open_file *of,* H - currently used by hardware only but available for software use * S - currently used and shareable by software only * E - currently used exclusively by one resource group +* P - currently pseudo-locked by one resource group */ static int rdt_bit_usage_show (struct kernfs_open_file *of,struct seq_file *seq,void *v) {struct rdt_resource *r = of->kn->parent->priv; -u32 sw_shareable, hw_shareable, exclusive; +u32 sw_shareable =0 , hw_shareable =0 ; +u32 exclusive =0 , pseudo_locked =0 ;struct rdt_domain *dom; -int i, hwb, swb, excl; +int i, hwb, swb, excl, psl;enum rdtgrp_mode mode;bool sep = false;u32 *ctrl;
@@ -803,12 +805,15 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,case RDT_MODE_EXCLUSIVE : exclusive |= *ctrl;break ; +case RDT_MODE_PSEUDO_LOCKSETUP :/* -* Temporarily handle pseudo-locking enums -* to silence compile warnings until handling -* added in later patches. +* RDT_MODE_PSEUDO_LOCKSETUP is possible +* here but not included since the CBM +* associated with this CLOSID in this mode +* is not initialized and no task or cpu can be +* assigned this CLOSID. */ -case RDT_MODE_PSEUDO_LOCKSETUP : +break ;case RDT_MODE_PSEUDO_LOCKED :case RDT_NUM_MODES : WARN(1,
@@ -817,9 +822,11 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, } }for (i = r->cache.cbm_len -1 ; i >=0 ; i--) { + pseudo_locked = dom->plr ? dom->plr->cbm :0 ; hwb =test_bit (i, (unsigned long *)&hw_shareable); swb =test_bit (i, (unsigned long *)&sw_shareable); excl =test_bit (i, (unsigned long *)&exclusive); + psl =test_bit (i, (unsigned long *)&pseudo_locked);if (hwb && swb)seq_putc (seq,'X' );else if (hwb && !swb)
@@ -828,6 +835,8 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,seq_putc (seq,'S' ); else if (excl) seq_putc(seq, 'E'); + else if (psl) + seq_putc(seq, 'P'); else /* Unused bits remain */ seq_putc(seq, '0'); }
@@ -1147,6 +1156,15 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,return -ENOENT ; } +if (rdtgrp->mode ==RDT_MODE_PSEUDO_LOCKED ) { +seq_printf (s,"%*s:" , max_name_width, rdtgrp->plr->r->name); + size =rdtgroup_cbm_to_size (rdtgrp->plr->r, + rdtgrp->plr->d, + rdtgrp->plr->cbm); +seq_printf (s,"%d=%u\n" , rdtgrp->plr->d->id, size); + goto out; + } +for_each_alloc_enabled_rdt_resource (r) {seq_printf (s,"%*s:" , max_name_width, r->name);list_for_each_entry (d, &r->domains, list) {
@@ -1164,6 +1182,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,seq_putc (s,'\n' ); } +out:rdtgroup_kn_unlock (of->kn);return 0 ;
PATCH v7 33/41 x86/intel_rdt: Ensure RDT cleanup on exit
In reply to Reinette ChatreThe RDT system's initialization does not have the corresponding exit handling to ensure everything initialized on load is cleaned up also.
Introduce the cleanup routines that complement all initialization. This includes the removal of a duplicate rdtgroup_init() declaration.
--- arch/x86/kernel/cpu/intel_rdt.c | 11 +++++++++++ arch/x86/kernel/cpu/intel_rdt.h | 3 +-- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 7 +++++++ 3 files changed, 19 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.c +11 -0
@@ -859,6 +859,8 @@ static __init bool get_rdt_resources(void)return (rdt_mon_capable || rdt_alloc_capable); } +static enum cpuhp_state rdt_online; +static int __init intel_rdt_late_init (void ) {struct rdt_resource *r;
@@ -880,6 +882,7 @@ static int __init intel_rdt_late_init(void)cpuhp_remove_state (state);return ret; } + rdt_online = state;for_each_alloc_capable_rdt_resource (r)pr_info ("Intel RDT %s allocation detected\n" , r->name);
@@ -891,3 +894,11 @@ static int __init intel_rdt_late_init(void) }late_initcall (intel_rdt_late_init); + +static void __exit intel_rdt_exit(void) +{ +cpuhp_remove_state (rdt_online); +rdtgroup_exit (); +} + +__exitcall (intel_rdt_exit);
arch/x86/kernel/cpu/intel_rdt.h +1 -2
@@ -209,6 +209,7 @@ extern struct list_head rdt_all_groups;extern int max_name_width, max_data_width;int __init rdtgroup_init (void ); +void __exit rdtgroup_exit(void);/** * struct rftype - describe each file in the resctrl file system
@@ -431,8 +432,6 @@ extern struct rdt_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default;DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); -int __init rdtgroup_init (void ); -enum {RDT_RESOURCE_L3 ,RDT_RESOURCE_L3DATA ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +7 -0
@@ -2837,3 +2837,10 @@ int __init rdtgroup_init(void)return ret; } + +void __exit rdtgroup_exit(void) +{ +unregister_filesystem (&rdt_fs_type); +sysfs_remove_mount_point (fs_kobj,"resctrl" ); +kernfs_destroy_root (rdt_root); +}
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Ensure RDT cleanup on exit
x86/intel_rdt: Ensure RDT cleanup on exit
Commit-ID: 0af6a48da481109affc4ea8295034f69993a91ef Gitweb: https://git.kernel.org/tip/0af6a48da481109affc4ea8295034f69993a91ef Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:24 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:50 +0200
x86/intel_rdt: Ensure RDT cleanup on exit
The RDT system's initialization does not have the corresponding exit handling to ensure everything initialized on load is cleaned up also.
Introduce the cleanup routines that complement all initialization. This includes the removal of a duplicate rdtgroup_init() declaration.
--- arch/x86/kernel/cpu/intel_rdt.c | 11 +++++++++++ arch/x86/kernel/cpu/intel_rdt.h | 3 +-- arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 7 +++++++ 3 files changed, 19 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/intel_rdt.c +11 -0
@@ -859,6 +859,8 @@ static __init bool get_rdt_resources(void)return (rdt_mon_capable || rdt_alloc_capable); } +static enum cpuhp_state rdt_online; +static int __init intel_rdt_late_init (void ) {struct rdt_resource *r;
@@ -880,6 +882,7 @@ static int __init intel_rdt_late_init(void)cpuhp_remove_state (state);return ret; } + rdt_online = state;for_each_alloc_capable_rdt_resource (r)pr_info ("Intel RDT %s allocation detected\n" , r->name);
@@ -891,3 +894,11 @@ static int __init intel_rdt_late_init(void) }late_initcall (intel_rdt_late_init); + +static void __exit intel_rdt_exit(void) +{ +cpuhp_remove_state (rdt_online); +rdtgroup_exit (); +} + +__exitcall (intel_rdt_exit);
arch/x86/kernel/cpu/intel_rdt.h +1 -2
@@ -209,6 +209,7 @@ extern struct list_head rdt_all_groups;extern int max_name_width, max_data_width;int __init rdtgroup_init (void ); +void __exit rdtgroup_exit(void);/** * struct rftype - describe each file in the resctrl file system
@@ -431,8 +432,6 @@ extern struct rdt_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default;DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); -int __init rdtgroup_init (void ); -enum {RDT_RESOURCE_L3 ,RDT_RESOURCE_L3DATA ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +7 -0
@@ -2828,3 +2828,10 @@ cleanup_root:return ret; } + +void __exit rdtgroup_exit(void) +{ +unregister_filesystem (&rdt_fs_type); +sysfs_remove_mount_point (fs_kobj,"resctrl" ); +kernfs_destroy_root (rdt_root); +}
PATCH v7 34/41 x86/intel_rdt: Create resctrl debug area
In reply to Reinette ChatreIn preparation for support of debugging of RDT sub features the user can now enable a RDT debugfs region.
The debug area is always enabled when CONFIG_DEBUG_FS is set as advised in
http://lkml.kernel.org/r/[email protected]
Also from same discussion in above linked email, no error checking on the debugfs creation return value since code should not behave differently when debugging passes or fails. Even on failure the returned value can be passed safely to other debugfs calls.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 27 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -432,6 +432,8 @@ extern struct rdt_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default;DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); +extern struct dentry *debugfs_resctrl; +enum {RDT_RESOURCE_L3 ,RDT_RESOURCE_L3DATA ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +27 -0
@@ -22,6 +22,7 @@#include <linux/cacheinfo.h> #include <linux/cpu.h> +#include <linux/debugfs.h> #include <linux/fs.h> #include <linux/sysfs.h> #include <linux/kernfs.h>
@@ -56,6 +57,8 @@ static struct kernfs_node *kn_mondata;static struct seq_buf last_cmd_status;static char last_cmd_status_buf[512 ]; +struct dentry *debugfs_resctrl; +void rdt_last_cmd_clear (void ) {lockdep_assert_held (&rdtgroup_mutex);
@@ -2828,6 +2831,29 @@ int __init rdtgroup_init(void)if (ret) goto cleanup_mountpoint; +/* +* Adding the resctrl debugfs directory here may not be ideal since +* it would let the resctrl debugfs directory appear on the debugfs +* filesystem before the resctrl filesystem is mounted. +* It may also be ok since that would enable debugging of RDT before +* resctrl is mounted. +* The reason why the debugfs directory is created here and not in +* rdt_mount() is because rdt_mount() takes rdtgroup_mutex and +* during the debugfs directory creation also &sb->s_type->i_mutex_key +* (the lockdep class of inode->i_rwsem). Other filesystem +* interactions (eg. SyS_getdents) have the lock ordering: +* &sb->s_type->i_mutex_key --> &mm->mmap_sem +* During mmap(), called with &mm->mmap_sem, the rdtgroup_mutex +* is taken, thus creating dependency: +* &mm->mmap_sem --> rdtgroup_mutex for the latter that can cause +* issues considering the other two lock dependencies. +* By creating the debugfs directory here we avoid a dependency +* that may cause deadlock (even though file operations cannot +* occur until the filesystem is mounted, but I do not know how to +* tell lockdep that). +*/ + debugfs_resctrl =debugfs_create_dir ("resctrl" ,NULL ); +return 0 ; cleanup_mountpoint:
@@ -2840,6 +2866,7 @@ int __init rdtgroup_init(void) void __exit rdtgroup_exit(void) { +debugfs_remove_recursive (debugfs_resctrl);unregister_filesystem (&rdt_fs_type);sysfs_remove_mount_point (fs_kobj,"resctrl" );kernfs_destroy_root (rdt_root);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Create resctrl debug area
x86/intel_rdt: Create resctrl debug area
Commit-ID: 37707ec6cba6668f22a47016fdee612b0ff62fe9 Gitweb: https://git.kernel.org/tip/37707ec6cba6668f22a47016fdee612b0ff62fe9 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:25 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:51 +0200
x86/intel_rdt: Create resctrl debug area
In preparation for support of debugging of RDT sub features the user can now enable a RDT debugfs region.
The debug area is always enabled when CONFIG_DEBUG_FS is set as advised in
http://lkml.kernel.org/r/[email protected]
Also from same discussion in above linked email, no error checking on the debugfs creation return value since code should not behave differently when debugging passes or fails. Even on failure the returned value can be passed safely to other debugfs calls.
--- arch/x86/kernel/cpu/intel_rdt.h | 2 ++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 27 +++++++++++++++++++++++++++ 2 files changed, 29 insertions(+)
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -432,6 +432,8 @@ extern struct rdt_resource rdt_resources_all[];extern struct rdtgroup rdtgroup_default;DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key); +extern struct dentry *debugfs_resctrl; +enum {RDT_RESOURCE_L3 ,RDT_RESOURCE_L3DATA ,
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +27 -0
@@ -22,6 +22,7 @@#include <linux/cacheinfo.h> #include <linux/cpu.h> +#include <linux/debugfs.h> #include <linux/fs.h> #include <linux/sysfs.h> #include <linux/kernfs.h>
@@ -56,6 +57,8 @@ static struct kernfs_node *kn_mondata;static struct seq_buf last_cmd_status;static char last_cmd_status_buf[512 ]; +struct dentry *debugfs_resctrl; +void rdt_last_cmd_clear (void ) {lockdep_assert_held (&rdtgroup_mutex);
@@ -2819,6 +2822,29 @@ int __init rdtgroup_init(void)if (ret) goto cleanup_mountpoint; +/* +* Adding the resctrl debugfs directory here may not be ideal since +* it would let the resctrl debugfs directory appear on the debugfs +* filesystem before the resctrl filesystem is mounted. +* It may also be ok since that would enable debugging of RDT before +* resctrl is mounted. +* The reason why the debugfs directory is created here and not in +* rdt_mount() is because rdt_mount() takes rdtgroup_mutex and +* during the debugfs directory creation also &sb->s_type->i_mutex_key +* (the lockdep class of inode->i_rwsem). Other filesystem +* interactions (eg. SyS_getdents) have the lock ordering: +* &sb->s_type->i_mutex_key --> &mm->mmap_sem +* During mmap(), called with &mm->mmap_sem, the rdtgroup_mutex +* is taken, thus creating dependency: +* &mm->mmap_sem --> rdtgroup_mutex for the latter that can cause +* issues considering the other two lock dependencies. +* By creating the debugfs directory here we avoid a dependency +* that may cause deadlock (even though file operations cannot +* occur until the filesystem is mounted, but I do not know how to +* tell lockdep that). +*/ + debugfs_resctrl =debugfs_create_dir ("resctrl" ,NULL ); +return 0 ; cleanup_mountpoint:
@@ -2831,6 +2857,7 @@ cleanup_root: void __exit rdtgroup_exit(void) { +debugfs_remove_recursive (debugfs_resctrl);unregister_filesystem (&rdt_fs_type);sysfs_remove_mount_point (fs_kobj,"resctrl" );kernfs_destroy_root (rdt_root);
PATCH v7 35/41 x86/intel_rdt: Create debugfs files for pseudo-locking testing
In reply to Reinette ChatreThere is no simple yes/no test to determine if pseudo-locking was successful. In order to test pseudo-locking we expose a debugfs file for each pseudo-locked region that will record the latency of reading the pseudo-locked memory at a stride of 32 bytes (hardcoded). These numbers will give us an idea of locking was successful or not since they will reflect cache hits and cache misses (hardware prefetching is disabled during the test).
The new debugfs file "pseudo_lock_measure" will, when the pseudo_lock_mem_latency tracepoint is enabled, record the latency of accessing each cache line twice.
Kernel tracepoints offer us histograms (when CONFIG_HIST_TRIGGERS is enabled) that is a simple way to visualize the memory access latency and immediately see any cache misses. For example, the hist trigger below before trigger of the measurement will display the memory access latency and instances at each latency:
echo 'hist:keys=latency' > /sys/kernel/debug/tracing/events/resctrl/\
pseudo_lock_mem_latency/trigger
echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable echo 1 > /sys/kernel/debug/resctrl/<newlock>/pseudo_lock_measure echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable cat /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/hist
--- arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/intel_rdt.h | 3 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 176 +++++++++++++++++- .../kernel/cpu/intel_rdt_pseudo_lock_event.h | 23 +++ 4 files changed, 202 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h
arch/x86/kernel/cpu/Makefile +1 -0
@@ -37,6 +37,7 @@ obj-$(CONFIG_CPU_SUP_UMC_32) += umc.o obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o obj-$(CONFIG_INTEL_RDT) += intel_rdt_ctrlmondata.o intel_rdt_pseudo_lock.o +CFLAGS_intel_rdt_pseudo_lock.o = -I$(src) obj-$(CONFIG_X86_MCE) += mcheck/ obj-$(CONFIG_MTRR) += mtrr/
arch/x86/kernel/cpu/intel_rdt.h +3 -0
@@ -138,6 +138,8 @@ struct mongroup {* @line_size: size of the cache lines * @size: size of pseudo-locked region in bytes * @kmem: the kernel memory associated with pseudo-locked region +* @debugfs_dir: pointer to this region's directory in the debugfs +* filesystem */ struct pseudo_lock_region {struct rdt_resource *r;
@@ -149,6 +151,7 @@ struct pseudo_lock_region {unsigned int line_size;unsigned int size;void *kmem; +struct dentry *debugfs_dir; };/**
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +175 -1
217 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM resctrl + +#if !defined(_TRACE_PSEUDO_LOCK_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_PSEUDO_LOCK_H + +#include <linux/tracepoint.h> + +TRACE_EVENT (pseudo_lock_mem_latency, + TP_PROTO(u32 latency), +TP_ARGS (latency), +TP_STRUCT__entry (__field (u32, latency)), +TP_fast_assign (__entry->latency = latency), +TP_printk ("latency=%u" , __entry->latency) + ); + +#endif /* _TRACE_PSEUDO_LOCK_H */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH . +#define TRACE_INCLUDE_FILE intel_rdt_pseudo_lock_event +#include <trace/define_trace.h>
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Create debugfs files for pseudo-locking testing
x86/intel_rdt: Create debugfs files for pseudo-locking testing
Commit-ID: 443810fe6160542c78e24c66047edd7a3cc830c6 Gitweb: https://git.kernel.org/tip/443810fe6160542c78e24c66047edd7a3cc830c6 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:26 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:51 +0200
x86/intel_rdt: Create debugfs files for pseudo-locking testing
There is no simple yes/no test to determine if pseudo-locking was successful. In order to test pseudo-locking we expose a debugfs file for each pseudo-locked region that will record the latency of reading the pseudo-locked memory at a stride of 32 bytes (hardcoded). These numbers will give us an idea of locking was successful or not since they will reflect cache hits and cache misses (hardware prefetching is disabled during the test).
The new debugfs file "pseudo_lock_measure" will, when the pseudo_lock_mem_latency tracepoint is enabled, record the latency of accessing each cache line twice.
Kernel tracepoints offer us histograms (when CONFIG_HIST_TRIGGERS is enabled) that is a simple way to visualize the memory access latency and immediately see any cache misses. For example, the hist trigger below before trigger of the measurement will display the memory access latency and instances at each latency:
echo 'hist:keys=latency' > /sys/kernel/debug/tracing/events/resctrl/\
pseudo_lock_mem_latency/trigger
echo 1 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable echo 1 > /sys/kernel/debug/resctrl/<newlock>/pseudo_lock_measure echo 0 > /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/enable cat /sys/kernel/debug/tracing/events/resctrl/pseudo_lock_mem_latency/hist
--- arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/intel_rdt.h | 3 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 176 +++++++++++++++++++++- arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h | 23 +++ 4 files changed, 202 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/Makefile +1 -0
@@ -37,6 +37,7 @@ obj-$(CONFIG_CPU_SUP_UMC_32) += umc.o obj-$(CONFIG_INTEL_RDT) += intel_rdt.o intel_rdt_rdtgroup.o intel_rdt_monitor.o obj-$(CONFIG_INTEL_RDT) += intel_rdt_ctrlmondata.o intel_rdt_pseudo_lock.o +CFLAGS_intel_rdt_pseudo_lock.o = -I$(src) obj-$(CONFIG_X86_MCE) += mcheck/ obj-$(CONFIG_MTRR) += mtrr/
arch/x86/kernel/cpu/intel_rdt.h +3 -0
@@ -138,6 +138,8 @@ struct mongroup {* @line_size: size of the cache lines * @size: size of pseudo-locked region in bytes * @kmem: the kernel memory associated with pseudo-locked region +* @debugfs_dir: pointer to this region's directory in the debugfs +* filesystem */ struct pseudo_lock_region {struct rdt_resource *r;
@@ -149,6 +151,7 @@ struct pseudo_lock_region {unsigned int line_size;unsigned int size;void *kmem; +struct dentry *debugfs_dir; };/**
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +175 -1
217 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM resctrl + +#if !defined(_TRACE_PSEUDO_LOCK_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _TRACE_PSEUDO_LOCK_H + +#include <linux/tracepoint.h> + +TRACE_EVENT (pseudo_lock_mem_latency, + TP_PROTO(u32 latency), +TP_ARGS (latency), +TP_STRUCT__entry (__field (u32, latency)), +TP_fast_assign (__entry->latency = latency), +TP_printk ("latency=%u" , __entry->latency) + ); + +#endif /* _TRACE_PSEUDO_LOCK_H */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH . +#define TRACE_INCLUDE_FILE intel_rdt_pseudo_lock_event +#include <trace/define_trace.h>
A message that is not in the archive (sent off-list or not yet received): 201806232005.zVl35hAb%[email protected]
Re: [PATCH V7 35/41] x86/intel_rdt: Create debugfs files for pseudo-locking testing
Quoted 16 lines · kbuild test robot
Thank you for the patch! Yet something to improve:
[auto build test ERROR on v4.18-rc1]
[cannot apply to tip/x86/core tip/auto-latest next-20180622]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]url: https://github.com/0day-ci/linux/commits/Reinette-Chatre/Intel-R-Resource-Director-Technology-Cache-Pseudo-Locking-enabling/20180623-065548
config: x86_64-randconfig-r0-06231921 (attached as .config)
compiler: gcc-6 (Debian 6.4.0-9) 6.4.0 20171026 reproduce:# save the attached .config to linux build tree make ARCH=x86_64Note: the linux-review/Reinette-Chatre/Intel-R-Resource-Director-Technology-Cache-Pseudo-Locking-enabling/20180623-065548 HEAD
c7e9cf1c60a66a41df4d37856b51360dde846b60builds fine.
It only hurts bisectibility.
Quoted 6 lines
All errors (new ones prefixed by >>):
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c: In function 'pseudo_lock_measure_trigger':Quoted 1 line
Quoted 1 line
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c:831:6: error: implicit declaration of function 'copy_from_user' [-Werror=implicit-function-declaration]if (
copy_from_user(buf, user_buf, buf_size))
^~~~~~~~~~~~~~
Thank you very much for catching this.
Clearly an issue but I was concerned that my testing did not pick this up since I do run the ktest.pl patchcheck tests to ensure that each patch from the series compiled cleanly without introducing a new warning.
Thanks to another test from ktest.pl, config_bisect, I was able to learn that I did not see this issue because I always have CONFIG_FTRACE=y and the config used in this test did not set it. This error can be reproduced when the config contains "# CONFIG_FTRACE is not set".
I assume this is why it took us a while to learn about this problem (a few versions of the patch series was sent to mailing list with this content) - since it needed a particular config to be encountered.
Thomas, I am sorry for breaking bisect on x86/cache. To fix this the appropriate header file needs to be added like a snippet below. I am not submitting a patch since I assume that this fix needs to be folded into the culprit ("x86/intel_rdt: Create character device exposing pseudo-locked region") to restore bisect.
The snippet was created against the culprit patch in an attempt for ease of merging. Three of the patches that follows it in the series also modifies that area but in my test it did not cause conflicts when rebased on top of this change. Please let me know what I can do from my side to help fix this.
8<-----------------
(quoted diff) +0 -0
b/arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c
index 5851f3e20742..4ce0fe05fd82 100644
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +1 -0
@@ -17,6 +17,7 @@#include <linux/debugfs.h> #include <linux/kthread.h> #include <linux/slab.h> +#include <linux/uaccess.h> #include <asm/cacheflush.h> #include <asm/intel-family.h> #include <asm/intel_rdt_sched.h>
Reinette
PATCH v7 36/41 x86/intel_rdt: Create character device exposing pseudo-locked region
In reply to Reinette ChatreAfter a pseudo-locked region is created it needs to be made available to user space for usage.
A character device supporting mmap() is created for each pseudo-locked region. A user space application can now use mmap() system call to map pseudo-locked region into its virtual address space.
--- arch/x86/kernel/cpu/intel_rdt.h | 5 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 282 ++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 10 +- 3 files changed, 288 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +5 -0
@@ -138,6 +138,8 @@ struct mongroup {* @line_size: size of the cache lines * @size: size of pseudo-locked region in bytes * @kmem: the kernel memory associated with pseudo-locked region +* @minor: minor number of character device associated with this +* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem */
@@ -151,6 +153,7 @@ struct pseudo_lock_region {unsigned int line_size;unsigned int size;void *kmem; +unsigned int minor;struct dentry *debugfs_dir; };
@@ -524,6 +527,8 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm);bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d); +int rdt_pseudo_lock_init (void ); +void rdt_pseudo_lock_release (void );int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +282 -0
330 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -9
@@ -1808,15 +1808,6 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,struct rdtgroup *prgrp, struct kernfs_node **mon_data_kn); -/* -* Define the hooks for Cache Pseudo-Locking to use within rdt_mount(). -* These are no-ops provided for the new kernfs changes to use as a -* baseline in preparation for a conflict-free merge between it -* (kernfs changes) and the Cache Pseudo-Locking enabling. -*/ -#define rdt_pseudo_lock_init () 0 -#define rdt_pseudo_lock_release () do { } while (0) -static struct dentry *rdt_mount (struct file_system_type *fs_type,int flags,const char *unused_dev_name,void *data)
@@ -2076,6 +2067,7 @@ static void rdt_kill_sb(struct super_block *sb)reset_all_ctrls (r);cdp_disable_all ();rmdir_all_sub (); +rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Create character device exposing pseudo-locked region
x86/intel_rdt: Create character device exposing pseudo-locked region
Commit-ID: 1c3fd056db38a5411ec06ef7ae52067408dee7fa Gitweb: https://git.kernel.org/tip/1c3fd056db38a5411ec06ef7ae52067408dee7fa Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:27 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:51 +0200
x86/intel_rdt: Create character device exposing pseudo-locked region
After a pseudo-locked region is created it needs to be made available to user space for usage.
A character device supporting mmap() is created for each pseudo-locked region. A user space application can now use mmap() system call to map pseudo-locked region into its virtual address space.
--- arch/x86/kernel/cpu/intel_rdt.h | 14 +- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 282 ++++++++++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 1 + 3 files changed, 288 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +5 -9
@@ -138,6 +138,8 @@ struct mongroup {* @line_size: size of the cache lines * @size: size of pseudo-locked region in bytes * @kmem: the kernel memory associated with pseudo-locked region +* @minor: minor number of character device associated with this +* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem */
@@ -151,6 +153,7 @@ struct pseudo_lock_region {unsigned int line_size;unsigned int size;void *kmem; +unsigned int minor;struct dentry *debugfs_dir; };
@@ -524,6 +527,8 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm);bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d); +int rdt_pseudo_lock_init (void ); +void rdt_pseudo_lock_release (void );int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);
@@ -551,13 +556,4 @@ void cqm_handle_limbo(struct work_struct *work);bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free); -/* -* Define the hooks for Cache Pseudo-Locking to use within rdt_mount(). -* These are no-ops provided for the new kernfs changes to use as a -* baseline in preparation for a conflict-free merge between it -* (kernfs changes) and the Cache Pseudo-Locking enabling. -*/ -static inline int rdt_pseudo_lock_init (void ) {return 0 ; } -static inline void rdt_pseudo_lock_release (void ) { } -#endif /* _ASM_X86_INTEL_RDT_H */
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +282 -0
330 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -0
@@ -2067,6 +2067,7 @@ static void rdt_kill_sb(struct super_block *sb)reset_all_ctrls (r);cdp_disable_all ();rmdir_all_sub (); +rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Create character device exposing pseudo-locked region
In reply to Reinette Chatrex86/intel_rdt: Create character device exposing pseudo-locked region
Commit-ID: 746e08590b864cf730d7bd23394e2d3fbb0f22b6 Gitweb: https://git.kernel.org/tip/746e08590b864cf730d7bd23394e2d3fbb0f22b6 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:27 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 24 Jun 2018 15:35:48 +0200
x86/intel_rdt: Create character device exposing pseudo-locked region
After a pseudo-locked region is created it needs to be made available to user space for usage.
A character device supporting mmap() is created for each pseudo-locked region. A user space application can now use mmap() system call to map pseudo-locked region into its virtual address space.
--- arch/x86/kernel/cpu/intel_rdt.h | 14 +- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 285 ++++++++++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 1 + 3 files changed, 291 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +5 -9
@@ -138,6 +138,8 @@ struct mongroup {* @line_size: size of the cache lines * @size: size of pseudo-locked region in bytes * @kmem: the kernel memory associated with pseudo-locked region +* @minor: minor number of character device associated with this +* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem */
@@ -151,6 +153,7 @@ struct pseudo_lock_region {unsigned int line_size;unsigned int size;void *kmem; +unsigned int minor;struct dentry *debugfs_dir; };
@@ -524,6 +527,8 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);int rdtgroup_locksetup_exit (struct rdtgroup *rdtgrp);bool rdtgroup_cbm_overlaps_pseudo_locked (struct rdt_domain *d,u32 _cbm);bool rdtgroup_pseudo_locked_in_hierarchy (struct rdt_domain *d); +int rdt_pseudo_lock_init (void ); +void rdt_pseudo_lock_release (void );int rdtgroup_pseudo_lock_create (struct rdtgroup *rdtgrp);void rdtgroup_pseudo_lock_remove (struct rdtgroup *rdtgrp);struct rdt_domain *get_domain_from_cpu (int cpu,struct rdt_resource *r);
@@ -551,13 +556,4 @@ void cqm_handle_limbo(struct work_struct *work);bool has_busy_rmid (struct rdt_resource *r,struct rdt_domain *d);void __check_limbo (struct rdt_domain *d,bool force_free); -/* -* Define the hooks for Cache Pseudo-Locking to use within rdt_mount(). -* These are no-ops provided for the new kernfs changes to use as a -* baseline in preparation for a conflict-free merge between it -* (kernfs changes) and the Cache Pseudo-Locking enabling. -*/ -static inline int rdt_pseudo_lock_init (void ) {return 0 ; } -static inline void rdt_pseudo_lock_release (void ) { } -#endif /* _ASM_X86_INTEL_RDT_H */
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +285 -0
337 lines not on this page · raw message
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +1 -0
@@ -2067,6 +2067,7 @@ static void rdt_kill_sb(struct super_block *sb)reset_all_ctrls (r);cdp_disable_all ();rmdir_all_sub (); +rdt_pseudo_lock_release (); rdtgroup_default.mode =RDT_MODE_SHAREABLE ;static_branch_disable_cpuslocked (&rdt_alloc_enable_key);static_branch_disable_cpuslocked (&rdt_mon_enable_key);
PATCH v7 37/41 x86/intel_rdt: More precise L2 hit/miss measurements
In reply to Reinette ChatreIntel Goldmont processors supports non-architectural precise events that can be used to give us more insight into the success of L2 cache pseudo-locking on these platforms.
Introduce a new measurement trigger that will enable two precise events, MEM_LOAD_UOPS_RETIRED.L2_HIT and MEM_LOAD_UOPS_RETIRED.L2_MISS, while accessing pseudo-locked data. A new tracepoint, pseudo_lock_l2, is created to make these results visible to the user.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 145 ++++++++++++++++-- .../kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++ 2 files changed, 146 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +136 -9
@@ -21,6 +21,7 @@#include <asm/cacheflush.h> #include <asm/intel-family.h> #include <asm/intel_rdt_sched.h> +#include <asm/perf_event.h> #include "intel_rdt.h" #define CREATE_TRACE_POINTS
@@ -60,6 +61,9 @@ static struct class *pseudo_lock_class;* hardware prefetch disable bits are included here as they are documented * in the SDM. * +* When adding a platform here also add support for its cache events to +* measure_cycles_perf_fn() +* * Return: * If platform is supported, the bits to disable hardware prefetchers, 0 * if platform is not supported.
@@ -98,6 +102,16 @@ static u64 get_prefetch_disable_bits(void)return 0 ; } +/* +* Helper to write 64bit value to MSR without tracing. Used when +* use of the cache should be restricted and use of registers used +* for local variables avoided. +*/ +static inline void pseudo_wrmsrl_notrace (unsigned int msr,u64 val) +{ +__wrmsr (msr, (u32)(val &0xffffffffULL ), (u32)(val >>32 )); +} +/** * pseudo_lock_minor_get - Obtain available minor number * @minor: Pointer to where new minor number will be stored
@@ -831,6 +845,107 @@ static int measure_cycles_lat_fn(void *_plr)return 0 ; } +static int measure_cycles_perf_fn (void *_plr) +{ +struct pseudo_lock_region *plr = _plr; +unsigned long long l2_hits, l2_miss; +u64 l2_hit_bits, l2_miss_bits; +u64 i; +#ifdef CONFIG_KASAN +/* +* The registers used for local register variables are also used +* when KASAN is active. When KASAN is active we use regular variables +* at the cost of including cache access latency to these variables +* in the measurements. +*/ +unsigned int line_size; +unsigned int size; +void *mem_r; +#else + registerunsigned int line_size asm("esi" ); + registerunsigned int size asm("edi" ); +#ifdef CONFIG_X86_64 + registervoid *mem_r asm("rbx" ); +#else + registervoid *mem_r asm("ebx" ); +#endif /* CONFIG_X86_64 */ +#endif /* CONFIG_KASAN */ + +/* +* Non-architectural event for the Goldmont Microarchitecture +* from Intel x86 Architecture Software Developer Manual (SDM): +* MEM_LOAD_UOPS_RETIRED D1H (event number) +* Umask values: +* L1_HIT 01H +* L2_HIT 02H +* L1_MISS 08H +* L2_MISS 10H +*/ + +/* +* Start by setting flags for IA32_PERFEVTSELx: +* OS (Operating system mode) 0x2 +* INT (APIC interrupt enable) 0x10 +* EN (Enable counter) 0x40 +* +* Then add the Umask value and event number to select performance +* event. +*/ + +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_ATOM_GOLDMONT : +case INTEL_FAM6_ATOM_GEMINI_LAKE : + l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ; +break ; +default : + goto out; + } + +local_irq_disable (); +/* +* Call wrmsr direcly to avoid the local register variables from +* being overwritten due to reordering of their assignment with +* the wrmsr calls. +*/ +__wrmsr (MSR_MISC_FEATURE_CONTROL , prefetch_disable_bits,0x0 ); +/* Disable events and reset counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +/* Set and enable the L2 counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); + mem_r = plr->kmem; + size = plr->size; + line_size = plr->line_size; +for (i =0 ; i < size; i += line_size) { + asmvolatile ("mov (%0,%1,1), %%eax\n\t" + : + :"r" (mem_r),"r" (i) + :"%eax" ,"memory" ); + } +/* +* Call wrmsr directly (no tracing) to not influence +* the cache access counters as they are disabled. +*/ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , + l2_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , + l2_miss_bits & ~(0x40ULL <<16 )); + l2_hits =native_read_pmc (0 ); + l2_miss =native_read_pmc (1 ); +wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 ); +local_irq_enable (); +trace_pseudo_lock_l2 (l2_hits, l2_miss); + +out: + plr->thread_done =1 ; +wake_up_interruptible (&plr->lock_thread_wq); +return 0 ; +} +/** * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region *
@@ -841,12 +956,12 @@ static int measure_cycles_lat_fn(void *_plr)* * Return: 0 on success, <0 on failure */ -static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp) +static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp,int sel) {struct pseudo_lock_region *plr = rdtgrp->plr;struct task_struct *thread;unsigned int cpu; -int ret; +int ret =-1 ;cpus_read_lock ();mutex_lock (&rdtgroup_mutex);
@@ -863,9 +978,19 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp) goto out; } - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , cpu); +if (sel ==1 ) + thread =kthread_create_on_node (measure_cycles_lat_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else if (sel ==2 ) + thread =kthread_create_on_node (measure_cycles_perf_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else + goto out; +if (IS_ERR (thread)) { ret =PTR_ERR (thread); goto out;
@@ -894,19 +1019,21 @@ static ssize_t pseudo_lock_measure_trigger(struct file *file,size_t buf_size;char buf[32 ];int ret; -bool bv; +int sel; buf_size =min (count, (sizeof (buf) -1 ));if (copy_from_user (buf, user_buf, buf_size))return -EFAULT ; buf[buf_size] ='\0' ; - ret =strtobool (buf, &bv); -if (ret ==0 && bv) { + ret =kstrtoint (buf,10 , &sel); +if (ret ==0 ) { +if (sel !=1 && sel !=2 ) +return -EINVAL ; ret =debugfs_file_get (file->f_path.dentry);if (ret)return ret; - ret =pseudo_lock_measure_cycles (rdtgrp); + ret =pseudo_lock_measure_cycles (rdtgrp, sel);if (ret ==0 ) ret = count;debugfs_file_put (file->f_path.dentry);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -15,6 +15,16 @@ TRACE_EVENT(pseudo_lock_mem_latency,TP_printk ("latency=%u" , __entry->latency) ); +TRACE_EVENT (pseudo_lock_l2, + TP_PROTO(u64 l2_hits, u64 l2_miss), +TP_ARGS (l2_hits, l2_miss), + TP_STRUCT__entry(__field(u64, l2_hits) +__field (u64, l2_miss)), + TP_fast_assign(__entry->l2_hits = l2_hits; + __entry->l2_miss = l2_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l2_hits, __entry->l2_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: More precise L2 hit/miss measurements
x86/intel_rdt: More precise L2 hit/miss measurements
Commit-ID: 7ef28739c453bb77dd2dc7a5ddcb534e488e0826 Gitweb: https://git.kernel.org/tip/7ef28739c453bb77dd2dc7a5ddcb534e488e0826 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:28 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:52 +0200
x86/intel_rdt: More precise L2 hit/miss measurements
Intel Goldmont processors supports non-architectural precise events that can be used to give us more insight into the success of L2 cache pseudo-locking on these platforms.
Introduce a new measurement trigger that will enable two precise events, MEM_LOAD_UOPS_RETIRED.L2_HIT and MEM_LOAD_UOPS_RETIRED.L2_MISS, while accessing pseudo-locked data. A new tracepoint, pseudo_lock_l2, is created to make these results visible to the user.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 145 ++++++++++++++++++++-- arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++ 2 files changed, 146 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +136 -9
@@ -21,6 +21,7 @@#include <asm/cacheflush.h> #include <asm/intel-family.h> #include <asm/intel_rdt_sched.h> +#include <asm/perf_event.h> #include "intel_rdt.h" #define CREATE_TRACE_POINTS
@@ -60,6 +61,9 @@ static struct class *pseudo_lock_class;* hardware prefetch disable bits are included here as they are documented * in the SDM. * +* When adding a platform here also add support for its cache events to +* measure_cycles_perf_fn() +* * Return: * If platform is supported, the bits to disable hardware prefetchers, 0 * if platform is not supported.
@@ -98,6 +102,16 @@ static u64 get_prefetch_disable_bits(void)return 0 ; } +/* +* Helper to write 64bit value to MSR without tracing. Used when +* use of the cache should be restricted and use of registers used +* for local variables avoided. +*/ +static inline void pseudo_wrmsrl_notrace (unsigned int msr,u64 val) +{ +__wrmsr (msr, (u32)(val &0xffffffffULL ), (u32)(val >>32 )); +} +/** * pseudo_lock_minor_get - Obtain available minor number * @minor: Pointer to where new minor number will be stored
@@ -831,6 +845,107 @@ static int measure_cycles_lat_fn(void *_plr)return 0 ; } +static int measure_cycles_perf_fn (void *_plr) +{ +struct pseudo_lock_region *plr = _plr; +unsigned long long l2_hits, l2_miss; +u64 l2_hit_bits, l2_miss_bits; +unsigned long i; +#ifdef CONFIG_KASAN +/* +* The registers used for local register variables are also used +* when KASAN is active. When KASAN is active we use regular variables +* at the cost of including cache access latency to these variables +* in the measurements. +*/ +unsigned int line_size; +unsigned int size; +void *mem_r; +#else + registerunsigned int line_size asm("esi" ); + registerunsigned int size asm("edi" ); +#ifdef CONFIG_X86_64 + registervoid *mem_r asm("rbx" ); +#else + registervoid *mem_r asm("ebx" ); +#endif /* CONFIG_X86_64 */ +#endif /* CONFIG_KASAN */ + +/* +* Non-architectural event for the Goldmont Microarchitecture +* from Intel x86 Architecture Software Developer Manual (SDM): +* MEM_LOAD_UOPS_RETIRED D1H (event number) +* Umask values: +* L1_HIT 01H +* L2_HIT 02H +* L1_MISS 08H +* L2_MISS 10H +*/ + +/* +* Start by setting flags for IA32_PERFEVTSELx: +* OS (Operating system mode) 0x2 +* INT (APIC interrupt enable) 0x10 +* EN (Enable counter) 0x40 +* +* Then add the Umask value and event number to select performance +* event. +*/ + +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_ATOM_GOLDMONT : +case INTEL_FAM6_ATOM_GEMINI_LAKE : + l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ; +break ; +default : + goto out; + } + +local_irq_disable (); +/* +* Call wrmsr direcly to avoid the local register variables from +* being overwritten due to reordering of their assignment with +* the wrmsr calls. +*/ +__wrmsr (MSR_MISC_FEATURE_CONTROL , prefetch_disable_bits,0x0 ); +/* Disable events and reset counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +/* Set and enable the L2 counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); + mem_r = plr->kmem; + size = plr->size; + line_size = plr->line_size; +for (i =0 ; i < size; i += line_size) { + asmvolatile ("mov (%0,%1,1), %%eax\n\t" + : + :"r" (mem_r),"r" (i) + :"%eax" ,"memory" ); + } +/* +* Call wrmsr directly (no tracing) to not influence +* the cache access counters as they are disabled. +*/ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , + l2_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , + l2_miss_bits & ~(0x40ULL <<16 )); + l2_hits =native_read_pmc (0 ); + l2_miss =native_read_pmc (1 ); +wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 ); +local_irq_enable (); +trace_pseudo_lock_l2 (l2_hits, l2_miss); + +out: + plr->thread_done =1 ; +wake_up_interruptible (&plr->lock_thread_wq); +return 0 ; +} +/** * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region *
@@ -841,12 +956,12 @@ static int measure_cycles_lat_fn(void *_plr)* * Return: 0 on success, <0 on failure */ -static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp) +static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp,int sel) {struct pseudo_lock_region *plr = rdtgrp->plr;struct task_struct *thread;unsigned int cpu; -int ret; +int ret =-1 ;cpus_read_lock ();mutex_lock (&rdtgroup_mutex);
@@ -863,9 +978,19 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp) goto out; } - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , cpu); +if (sel ==1 ) + thread =kthread_create_on_node (measure_cycles_lat_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else if (sel ==2 ) + thread =kthread_create_on_node (measure_cycles_perf_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else + goto out; +if (IS_ERR (thread)) { ret =PTR_ERR (thread); goto out;
@@ -894,19 +1019,21 @@ static ssize_t pseudo_lock_measure_trigger(struct file *file,size_t buf_size;char buf[32 ];int ret; -bool bv; +int sel; buf_size =min (count, (sizeof (buf) -1 ));if (copy_from_user (buf, user_buf, buf_size))return -EFAULT ; buf[buf_size] ='\0' ; - ret =strtobool (buf, &bv); -if (ret ==0 && bv) { + ret =kstrtoint (buf,10 , &sel); +if (ret ==0 ) { +if (sel !=1 && sel !=2 ) +return -EINVAL ; ret =debugfs_file_get (file->f_path.dentry);if (ret)return ret; - ret =pseudo_lock_measure_cycles (rdtgrp); + ret =pseudo_lock_measure_cycles (rdtgrp, sel);if (ret ==0 ) ret = count;debugfs_file_put (file->f_path.dentry);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -15,6 +15,16 @@ TRACE_EVENT(pseudo_lock_mem_latency,TP_printk ("latency=%u" , __entry->latency) ); +TRACE_EVENT (pseudo_lock_l2, + TP_PROTO(u64 l2_hits, u64 l2_miss), +TP_ARGS (l2_hits, l2_miss), + TP_STRUCT__entry(__field(u64, l2_hits) +__field (u64, l2_miss)), + TP_fast_assign(__entry->l2_hits = l2_hits; + __entry->l2_miss = l2_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l2_hits, __entry->l2_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: More precise L2 hit/miss measurements
In reply to Reinette Chatrex86/intel_rdt: More precise L2 hit/miss measurements
Commit-ID: 8a2fc0e1bc0cd856101927188884d7c370b62188 Gitweb: https://git.kernel.org/tip/8a2fc0e1bc0cd856101927188884d7c370b62188 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:28 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 24 Jun 2018 15:35:48 +0200
x86/intel_rdt: More precise L2 hit/miss measurements
Intel Goldmont processors supports non-architectural precise events that can be used to give us more insight into the success of L2 cache pseudo-locking on these platforms.
Introduce a new measurement trigger that will enable two precise events, MEM_LOAD_UOPS_RETIRED.L2_HIT and MEM_LOAD_UOPS_RETIRED.L2_MISS, while accessing pseudo-locked data. A new tracepoint, pseudo_lock_l2, is created to make these results visible to the user.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 145 ++++++++++++++++++++-- arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++ 2 files changed, 146 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +136 -9
@@ -23,6 +23,7 @@#include <asm/cacheflush.h> #include <asm/intel-family.h> #include <asm/intel_rdt_sched.h> +#include <asm/perf_event.h> #include "intel_rdt.h"
@@ -63,6 +64,9 @@ static struct class *pseudo_lock_class;* hardware prefetch disable bits are included here as they are documented * in the SDM. * +* When adding a platform here also add support for its cache events to +* measure_cycles_perf_fn() +* * Return: * If platform is supported, the bits to disable hardware prefetchers, 0 * if platform is not supported.
@@ -101,6 +105,16 @@ static u64 get_prefetch_disable_bits(void)return 0 ; } +/* +* Helper to write 64bit value to MSR without tracing. Used when +* use of the cache should be restricted and use of registers used +* for local variables avoided. +*/ +static inline void pseudo_wrmsrl_notrace (unsigned int msr,u64 val) +{ +__wrmsr (msr, (u32)(val &0xffffffffULL ), (u32)(val >>32 )); +} +/** * pseudo_lock_minor_get - Obtain available minor number * @minor: Pointer to where new minor number will be stored
@@ -834,6 +848,107 @@ static int measure_cycles_lat_fn(void *_plr)return 0 ; } +static int measure_cycles_perf_fn (void *_plr) +{ +struct pseudo_lock_region *plr = _plr; +unsigned long long l2_hits, l2_miss; +u64 l2_hit_bits, l2_miss_bits; +unsigned long i; +#ifdef CONFIG_KASAN +/* +* The registers used for local register variables are also used +* when KASAN is active. When KASAN is active we use regular variables +* at the cost of including cache access latency to these variables +* in the measurements. +*/ +unsigned int line_size; +unsigned int size; +void *mem_r; +#else + registerunsigned int line_size asm("esi" ); + registerunsigned int size asm("edi" ); +#ifdef CONFIG_X86_64 + registervoid *mem_r asm("rbx" ); +#else + registervoid *mem_r asm("ebx" ); +#endif /* CONFIG_X86_64 */ +#endif /* CONFIG_KASAN */ + +/* +* Non-architectural event for the Goldmont Microarchitecture +* from Intel x86 Architecture Software Developer Manual (SDM): +* MEM_LOAD_UOPS_RETIRED D1H (event number) +* Umask values: +* L1_HIT 01H +* L2_HIT 02H +* L1_MISS 08H +* L2_MISS 10H +*/ + +/* +* Start by setting flags for IA32_PERFEVTSELx: +* OS (Operating system mode) 0x2 +* INT (APIC interrupt enable) 0x10 +* EN (Enable counter) 0x40 +* +* Then add the Umask value and event number to select performance +* event. +*/ + +switch (boot_cpu_data.x86_model) { +case INTEL_FAM6_ATOM_GOLDMONT : +case INTEL_FAM6_ATOM_GEMINI_LAKE : + l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ; +break ; +default : + goto out; + } + +local_irq_disable (); +/* +* Call wrmsr direcly to avoid the local register variables from +* being overwritten due to reordering of their assignment with +* the wrmsr calls. +*/ +__wrmsr (MSR_MISC_FEATURE_CONTROL , prefetch_disable_bits,0x0 ); +/* Disable events and reset counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +/* Set and enable the L2 counters */ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); + mem_r = plr->kmem; + size = plr->size; + line_size = plr->line_size; +for (i =0 ; i < size; i += line_size) { + asmvolatile ("mov (%0,%1,1), %%eax\n\t" + : + :"r" (mem_r),"r" (i) + :"%eax" ,"memory" ); + } +/* +* Call wrmsr directly (no tracing) to not influence +* the cache access counters as they are disabled. +*/ +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , + l2_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , + l2_miss_bits & ~(0x40ULL <<16 )); + l2_hits =native_read_pmc (0 ); + l2_miss =native_read_pmc (1 ); +wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 ); +local_irq_enable (); +trace_pseudo_lock_l2 (l2_hits, l2_miss); + +out: + plr->thread_done =1 ; +wake_up_interruptible (&plr->lock_thread_wq); +return 0 ; +} +/** * pseudo_lock_measure_cycles - Trigger latency measure to pseudo-locked region *
@@ -844,12 +959,12 @@ static int measure_cycles_lat_fn(void *_plr)* * Return: 0 on success, <0 on failure */ -static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp) +static int pseudo_lock_measure_cycles (struct rdtgroup *rdtgrp,int sel) {struct pseudo_lock_region *plr = rdtgrp->plr;struct task_struct *thread;unsigned int cpu; -int ret; +int ret =-1 ;cpus_read_lock ();mutex_lock (&rdtgroup_mutex);
@@ -866,9 +981,19 @@ static int pseudo_lock_measure_cycles(struct rdtgroup *rdtgrp) goto out; } - thread =kthread_create_on_node (measure_cycles_lat_fn, plr, -cpu_to_node (cpu), -"pseudo_lock_measure/%u" , cpu); +if (sel ==1 ) + thread =kthread_create_on_node (measure_cycles_lat_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else if (sel ==2 ) + thread =kthread_create_on_node (measure_cycles_perf_fn, plr, +cpu_to_node (cpu), +"pseudo_lock_measure/%u" , + cpu); +else + goto out; +if (IS_ERR (thread)) { ret =PTR_ERR (thread); goto out;
@@ -897,19 +1022,21 @@ static ssize_t pseudo_lock_measure_trigger(struct file *file,size_t buf_size;char buf[32 ];int ret; -bool bv; +int sel; buf_size =min (count, (sizeof (buf) -1 ));if (copy_from_user (buf, user_buf, buf_size))return -EFAULT ; buf[buf_size] ='\0' ; - ret =strtobool (buf, &bv); -if (ret ==0 && bv) { + ret =kstrtoint (buf,10 , &sel); +if (ret ==0 ) { +if (sel !=1 && sel !=2 ) +return -EINVAL ; ret =debugfs_file_get (file->f_path.dentry);if (ret)return ret; - ret =pseudo_lock_measure_cycles (rdtgrp); + ret =pseudo_lock_measure_cycles (rdtgrp, sel);if (ret ==0 ) ret = count;debugfs_file_put (file->f_path.dentry);
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -15,6 +15,16 @@ TRACE_EVENT(pseudo_lock_mem_latency,TP_printk ("latency=%u" , __entry->latency) ); +TRACE_EVENT (pseudo_lock_l2, + TP_PROTO(u64 l2_hits, u64 l2_miss), +TP_ARGS (l2_hits, l2_miss), + TP_STRUCT__entry(__field(u64, l2_hits) +__field (u64, l2_miss)), + TP_fast_assign(__entry->l2_hits = l2_hits; + __entry->l2_miss = l2_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l2_hits, __entry->l2_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
PATCH v7 38/41 x86/intel_rdt: Support L3 cache performance event of Broadwell
In reply to Reinette ChatreBroadwell microarchitecture supports pseudo-locking. Add support for the L3 cache related performance events of these systems so that the success of pseudo-locking can be measured more accurately on these platforms.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 56 +++++++++++++++++++ .../kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++++ 2 files changed, 66 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +56 -0
@@ -847,6 +847,8 @@ static int measure_cycles_lat_fn(void *_plr)static int measure_cycles_perf_fn (void *_plr) { +unsigned long long l3_hits =0 , l3_miss =0 ; +u64 l3_hit_bits =0 , l3_miss_bits =0 ;struct pseudo_lock_region *plr = _plr;unsigned long long l2_hits, l2_miss;u64 l2_hit_bits, l2_miss_bits;
@@ -880,6 +882,16 @@ static int measure_cycles_perf_fn(void *_plr)* L2_HIT 02H * L1_MISS 08H * L2_MISS 10H +* +* On Broadwell Microarchitecture the MEM_LOAD_UOPS_RETIRED event +* has two "no fix" errata associated with it: BDM35 and BDM100. On +* this platform we use the following events instead: +* L2_RQSTS 24H (Documented in https://download.01.org/perfmon/BDW/) +* REFERENCES FFH +* MISS 3FH +* LONGEST_LAT_CACHE 2EH (Documented in SDM) +* REFERENCE 4FH +* MISS 41H */ /*
@@ -898,6 +910,14 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ;break ; +case INTEL_FAM6_BROADWELL_X : +/* On BDW the l2_hit_bits count references, not hits */ + l2_hit_bits = (0x52ULL <<16 ) | (0xff <<8 ) |0x24 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x3f <<8 ) |0x24 ; +/* On BDW the l3_hit_bits count references, not hits */ + l3_hit_bits = (0x52ULL <<16 ) | (0x4f <<8 ) |0x2e ; + l3_miss_bits = (0x52ULL <<16 ) | (0x41 <<8 ) |0x2e ; +break ;default : goto out; }
@@ -914,9 +934,21 @@ static int measure_cycles_perf_fn(void *_plr)pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +3 ,0x0 ); + }/* Set and enable the L2 counters */ pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits);pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits); + } mem_r = plr->kmem; size = plr->size; line_size = plr->line_size;
@@ -934,11 +966,35 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits & ~(0x40ULL << 16));pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits & ~(0x40ULL <<16 )); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits & ~(0x40ULL <<16 )); + } l2_hits =native_read_pmc (0 ); l2_miss =native_read_pmc (1 ); +if (l3_hit_bits >0 ) { + l3_hits =native_read_pmc (2 ); + l3_miss =native_read_pmc (3 ); + }wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 );local_irq_enable (); +/* +* On BDW we count references and misses, need to adjust. Sometimes +* the "hits" counter is a bit more than the references, for +* example, x references but x + 1 hits. To not report invalid +* hit values in this case we treat that as misses eaqual to +* references. +*/ +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l2_hits -= (l2_miss > l2_hits ? l2_hits : l2_miss);trace_pseudo_lock_l2 (l2_hits, l2_miss); +if (l3_hit_bits >0 ) { +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l3_hits -= (l3_miss > l3_hits ? l3_hits : l3_miss); +trace_pseudo_lock_l3 (l3_hits, l3_miss); + } out: plr->thread_done =1 ;
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -25,6 +25,16 @@ TRACE_EVENT(pseudo_lock_l2,TP_printk ("hits=%llu miss=%llu" , __entry->l2_hits, __entry->l2_miss)); +TRACE_EVENT (pseudo_lock_l3, + TP_PROTO(u64 l3_hits, u64 l3_miss), +TP_ARGS (l3_hits, l3_miss), + TP_STRUCT__entry(__field(u64, l3_hits) +__field (u64, l3_miss)), + TP_fast_assign(__entry->l3_hits = l3_hits; + __entry->l3_miss = l3_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l3_hits, __entry->l3_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Support L3 cache performance event of Broadwell
x86/intel_rdt: Support L3 cache performance event of Broadwell
Commit-ID: 5c3968994582c1265cdb81884533371cbe868a93 Gitweb: https://git.kernel.org/tip/5c3968994582c1265cdb81884533371cbe868a93 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:29 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:52 +0200
x86/intel_rdt: Support L3 cache performance event of Broadwell
Broadwell microarchitecture supports pseudo-locking. Add support for the L3 cache related performance events of these systems so that the success of pseudo-locking can be measured more accurately on these platforms.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 56 +++++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++++ 2 files changed, 66 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +56 -0
@@ -847,6 +847,8 @@ static int measure_cycles_lat_fn(void *_plr)static int measure_cycles_perf_fn (void *_plr) { +unsigned long long l3_hits =0 , l3_miss =0 ; +u64 l3_hit_bits =0 , l3_miss_bits =0 ;struct pseudo_lock_region *plr = _plr;unsigned long long l2_hits, l2_miss;u64 l2_hit_bits, l2_miss_bits;
@@ -880,6 +882,16 @@ static int measure_cycles_perf_fn(void *_plr)* L2_HIT 02H * L1_MISS 08H * L2_MISS 10H +* +* On Broadwell Microarchitecture the MEM_LOAD_UOPS_RETIRED event +* has two "no fix" errata associated with it: BDM35 and BDM100. On +* this platform we use the following events instead: +* L2_RQSTS 24H (Documented in https://download.01.org/perfmon/BDW/) +* REFERENCES FFH +* MISS 3FH +* LONGEST_LAT_CACHE 2EH (Documented in SDM) +* REFERENCE 4FH +* MISS 41H */ /*
@@ -898,6 +910,14 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ;break ; +case INTEL_FAM6_BROADWELL_X : +/* On BDW the l2_hit_bits count references, not hits */ + l2_hit_bits = (0x52ULL <<16 ) | (0xff <<8 ) |0x24 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x3f <<8 ) |0x24 ; +/* On BDW the l3_hit_bits count references, not hits */ + l3_hit_bits = (0x52ULL <<16 ) | (0x4f <<8 ) |0x2e ; + l3_miss_bits = (0x52ULL <<16 ) | (0x41 <<8 ) |0x2e ; +break ;default : goto out; }
@@ -914,9 +934,21 @@ static int measure_cycles_perf_fn(void *_plr)pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +3 ,0x0 ); + }/* Set and enable the L2 counters */ pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits);pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits); + } mem_r = plr->kmem; size = plr->size; line_size = plr->line_size;
@@ -934,11 +966,35 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits & ~(0x40ULL << 16));pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits & ~(0x40ULL <<16 )); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits & ~(0x40ULL <<16 )); + } l2_hits =native_read_pmc (0 ); l2_miss =native_read_pmc (1 ); +if (l3_hit_bits >0 ) { + l3_hits =native_read_pmc (2 ); + l3_miss =native_read_pmc (3 ); + }wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 );local_irq_enable (); +/* +* On BDW we count references and misses, need to adjust. Sometimes +* the "hits" counter is a bit more than the references, for +* example, x references but x + 1 hits. To not report invalid +* hit values in this case we treat that as misses eaqual to +* references. +*/ +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l2_hits -= (l2_miss > l2_hits ? l2_hits : l2_miss);trace_pseudo_lock_l2 (l2_hits, l2_miss); +if (l3_hit_bits >0 ) { +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l3_hits -= (l3_miss > l3_hits ? l3_hits : l3_miss); +trace_pseudo_lock_l3 (l3_hits, l3_miss); + } out: plr->thread_done =1 ;
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -25,6 +25,16 @@ TRACE_EVENT(pseudo_lock_l2,TP_printk ("hits=%llu miss=%llu" , __entry->l2_hits, __entry->l2_miss)); +TRACE_EVENT (pseudo_lock_l3, + TP_PROTO(u64 l3_hits, u64 l3_miss), +TP_ARGS (l3_hits, l3_miss), + TP_STRUCT__entry(__field(u64, l3_hits) +__field (u64, l3_miss)), + TP_fast_assign(__entry->l3_hits = l3_hits; + __entry->l3_miss = l3_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l3_hits, __entry->l3_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Support L3 cache performance event of Broadwell
In reply to Reinette Chatrex86/intel_rdt: Support L3 cache performance event of Broadwell
Commit-ID: f3be1e7b2cf8bc096386a3588fc640b0db6b28d7 Gitweb: https://git.kernel.org/tip/f3be1e7b2cf8bc096386a3588fc640b0db6b28d7 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:29 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 24 Jun 2018 15:35:48 +0200
x86/intel_rdt: Support L3 cache performance event of Broadwell
Broadwell microarchitecture supports pseudo-locking. Add support for the L3 cache related performance events of these systems so that the success of pseudo-locking can be measured more accurately on these platforms.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 56 +++++++++++++++++++++++ arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h | 10 ++++ 2 files changed, 66 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +56 -0
@@ -850,6 +850,8 @@ static int measure_cycles_lat_fn(void *_plr)static int measure_cycles_perf_fn (void *_plr) { +unsigned long long l3_hits =0 , l3_miss =0 ; +u64 l3_hit_bits =0 , l3_miss_bits =0 ;struct pseudo_lock_region *plr = _plr;unsigned long long l2_hits, l2_miss;u64 l2_hit_bits, l2_miss_bits;
@@ -883,6 +885,16 @@ static int measure_cycles_perf_fn(void *_plr)* L2_HIT 02H * L1_MISS 08H * L2_MISS 10H +* +* On Broadwell Microarchitecture the MEM_LOAD_UOPS_RETIRED event +* has two "no fix" errata associated with it: BDM35 and BDM100. On +* this platform we use the following events instead: +* L2_RQSTS 24H (Documented in https://download.01.org/perfmon/BDW/) +* REFERENCES FFH +* MISS 3FH +* LONGEST_LAT_CACHE 2EH (Documented in SDM) +* REFERENCE 4FH +* MISS 41H */ /*
@@ -901,6 +913,14 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits = (0x52ULL <<16 ) | (0x2 <<8 ) |0xd1 ; l2_miss_bits = (0x52ULL <<16 ) | (0x10 <<8 ) |0xd1 ;break ; +case INTEL_FAM6_BROADWELL_X : +/* On BDW the l2_hit_bits count references, not hits */ + l2_hit_bits = (0x52ULL <<16 ) | (0xff <<8 ) |0x24 ; + l2_miss_bits = (0x52ULL <<16 ) | (0x3f <<8 ) |0x24 ; +/* On BDW the l3_hit_bits count references, not hits */ + l3_hit_bits = (0x52ULL <<16 ) | (0x4f <<8 ) |0x2e ; + l3_miss_bits = (0x52ULL <<16 ) | (0x41 <<8 ) |0x2e ; +break ;default : goto out; }
@@ -917,9 +937,21 @@ static int measure_cycles_perf_fn(void *_plr)pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 ,0x0 );pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +1 ,0x0 ); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +2 ,0x0 ); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_PERFCTR0 +3 ,0x0 ); + }/* Set and enable the L2 counters */ pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 , l2_hit_bits);pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits); + } mem_r = plr->kmem; size = plr->size; line_size = plr->line_size;
@@ -937,11 +969,35 @@ static int measure_cycles_perf_fn(void *_plr) l2_hit_bits & ~(0x40ULL << 16));pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +1 , l2_miss_bits & ~(0x40ULL <<16 )); +if (l3_hit_bits >0 ) { +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +2 , + l3_hit_bits & ~(0x40ULL <<16 )); +pseudo_wrmsrl_notrace (MSR_ARCH_PERFMON_EVENTSEL0 +3 , + l3_miss_bits & ~(0x40ULL <<16 )); + } l2_hits =native_read_pmc (0 ); l2_miss =native_read_pmc (1 ); +if (l3_hit_bits >0 ) { + l3_hits =native_read_pmc (2 ); + l3_miss =native_read_pmc (3 ); + }wrmsr (MSR_MISC_FEATURE_CONTROL ,0x0 ,0x0 );local_irq_enable (); +/* +* On BDW we count references and misses, need to adjust. Sometimes +* the "hits" counter is a bit more than the references, for +* example, x references but x + 1 hits. To not report invalid +* hit values in this case we treat that as misses eaqual to +* references. +*/ +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l2_hits -= (l2_miss > l2_hits ? l2_hits : l2_miss);trace_pseudo_lock_l2 (l2_hits, l2_miss); +if (l3_hit_bits >0 ) { +if (boot_cpu_data.x86_model ==INTEL_FAM6_BROADWELL_X ) + l3_hits -= (l3_miss > l3_hits ? l3_hits : l3_miss); +trace_pseudo_lock_l3 (l3_hits, l3_miss); + } out: plr->thread_done =1 ;
arch/x86/kernel/cpu/intel_rdt_pseudo_lock_event.h +10 -0
@@ -25,6 +25,16 @@ TRACE_EVENT(pseudo_lock_l2,TP_printk ("hits=%llu miss=%llu" , __entry->l2_hits, __entry->l2_miss)); +TRACE_EVENT (pseudo_lock_l3, + TP_PROTO(u64 l3_hits, u64 l3_miss), +TP_ARGS (l3_hits, l3_miss), + TP_STRUCT__entry(__field(u64, l3_hits) +__field (u64, l3_miss)), + TP_fast_assign(__entry->l3_hits = l3_hits; + __entry->l3_miss = l3_miss;), +TP_printk ("hits=%llu miss=%llu" , + __entry->l3_hits, __entry->l3_miss)); +#endif /* _TRACE_PSEUDO_LOCK_H */ #undef TRACE_INCLUDE_PATH
PATCH v7 39/41 x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
In reply to Reinette ChatreDeeper C-states impact cache content through shrinking of the cache or flushing entire cache to memory before reducing power to the cache. Deeper C-states will thus negatively impact the pseudo-locked regions.
To avoid impacting pseudo-locked regions C-states are limited on pseudo-locked region creation so that cores associated with the pseudo-locked region are prevented from entering deeper C-states. This is accomplished by requesting a CPU latency target which will prevent the core from entering C6 across all supported platforms.
--- Documentation/x86/intel_rdt_ui.txt | 4 +- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 85 ++++++++++++++++++++- 3 files changed, 87 insertions(+), 4 deletions(-)
Documentation/x86/intel_rdt_ui.txt +2 -2
@@ -461,8 +461,8 @@ in the cache via carefully configuring the CAT feature and controlling application behavior. There is no guarantee that data is placed in cache. Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict “locked” data from cache. Power management C-states may shrink or -power off cache. It is thus recommended to limit the processor maximum -C-state, for example, by setting the processor.max_cstate kernel parameter. +power off cache. Deeper C-states will automatically be restricted on +pseudo-locked region creation. It is required that an application using a pseudo-locked region runs with affinity to the cores (or a subset of the cores) associated
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -142,6 +142,7 @@ struct mongroup {* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem +* @pm_reqs: Power management QoS requests related to this region */ struct pseudo_lock_region {struct rdt_resource *r;
@@ -155,6 +156,7 @@ struct pseudo_lock_region {void *kmem;unsigned int minor;struct dentry *debugfs_dir; +struct list_head pm_reqs; };/**
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +83 -2
133 lines not on this page · raw message
Signature · 1 line
2.17.0
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
Commit-ID: 8b0d3ba3140f8ed7563b2e0788bc595e25fee808 Gitweb: https://git.kernel.org/tip/8b0d3ba3140f8ed7563b2e0788bc595e25fee808 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:30 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sat, 23 Jun 2018 13:03:52 +0200
x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
Deeper C-states impact cache content through shrinking of the cache or flushing entire cache to memory before reducing power to the cache. Deeper C-states will thus negatively impact the pseudo-locked regions.
To avoid impacting pseudo-locked regions C-states are limited on pseudo-locked region creation so that cores associated with the pseudo-locked region are prevented from entering deeper C-states. This is accomplished by requesting a CPU latency target which will prevent the core from entering C6 across all supported platforms.
--- Documentation/x86/intel_rdt_ui.txt | 4 +- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 85 ++++++++++++++++++++++++++++- 3 files changed, 87 insertions(+), 4 deletions(-)
Documentation/x86/intel_rdt_ui.txt +2 -2
@@ -461,8 +461,8 @@ in the cache via carefully configuring the CAT feature and controlling application behavior. There is no guarantee that data is placed in cache. Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict “locked” data from cache. Power management C-states may shrink or -power off cache. It is thus recommended to limit the processor maximum -C-state, for example, by setting the processor.max_cstate kernel parameter. +power off cache. Deeper C-states will automatically be restricted on +pseudo-locked region creation. It is required that an application using a pseudo-locked region runs with affinity to the cores (or a subset of the cores) associated
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -142,6 +142,7 @@ struct mongroup {* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem +* @pm_reqs: Power management QoS requests related to this region */ struct pseudo_lock_region {struct rdt_resource *r;
@@ -155,6 +156,7 @@ struct pseudo_lock_region {void *kmem;unsigned int minor;struct dentry *debugfs_dir; +struct list_head pm_reqs; };/**
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +83 -2
133 lines not on this page · raw message
tip-bot for Reinette Chatre
merge bot[tip:x86/cache] x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
In reply to Reinette Chatrex86/intel_rdt: Limit C-states dynamically when pseudo-locking active
Commit-ID: 6fc0de37f663278af160e8e1f0c38b27e6c06206 Gitweb: https://git.kernel.org/tip/6fc0de37f663278af160e8e1f0c38b27e6c06206 Author: Reinette Chatre <[email protected]> AuthorDate: Fri, 22 Jun 2018 15:42:30 -0700 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 24 Jun 2018 15:35:48 +0200
x86/intel_rdt: Limit C-states dynamically when pseudo-locking active
Deeper C-states impact cache content through shrinking of the cache or flushing entire cache to memory before reducing power to the cache. Deeper C-states will thus negatively impact the pseudo-locked regions.
To avoid impacting pseudo-locked regions C-states are limited on pseudo-locked region creation so that cores associated with the pseudo-locked region are prevented from entering deeper C-states. This is accomplished by requesting a CPU latency target which will prevent the core from entering C6 across all supported platforms.
--- Documentation/x86/intel_rdt_ui.txt | 4 +- arch/x86/kernel/cpu/intel_rdt.h | 2 + arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 85 ++++++++++++++++++++++++++++- 3 files changed, 87 insertions(+), 4 deletions(-)
Documentation/x86/intel_rdt_ui.txt +2 -2
@@ -461,8 +461,8 @@ in the cache via carefully configuring the CAT feature and controlling application behavior. There is no guarantee that data is placed in cache. Instructions like INVD, WBINVD, CLFLUSH, etc. can still evict “locked” data from cache. Power management C-states may shrink or -power off cache. It is thus recommended to limit the processor maximum -C-state, for example, by setting the processor.max_cstate kernel parameter. +power off cache. Deeper C-states will automatically be restricted on +pseudo-locked region creation. It is required that an application using a pseudo-locked region runs with affinity to the cores (or a subset of the cores) associated
arch/x86/kernel/cpu/intel_rdt.h +2 -0
@@ -142,6 +142,7 @@ struct mongroup {* region * @debugfs_dir: pointer to this region's directory in the debugfs * filesystem +* @pm_reqs: Power management QoS requests related to this region */ struct pseudo_lock_region {struct rdt_resource *r;
@@ -155,6 +156,7 @@ struct pseudo_lock_region {void *kmem;unsigned int minor;struct dentry *debugfs_dir; +struct list_head pm_reqs; };/**
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +83 -2
133 lines not on this page · raw message
PATCH v7 40/41 x86/intel_rdt: Fix passing of value to 32-bit register
In reply to Reinette Chatre0-day kbuild test robot reported the following issue:
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c: In function 'pseudo_lock_fn':
Quoted 1 line
Quoted 1 line
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c:363:1: warning: unsupported size for integer register
}
^Quoted 1 line
Quoted 1 line
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c:363:1: warning: unsupported size for integer register
vim +363 arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c
361 wake_up_interruptible(&plr->lock_thread_wq);
362 return 0;
Quoted 1 line
363 }
364
The issue is earlier in the code, and in more locations, where (in 32-bit) a u64 variable (i below) is used as the value of a 32-bit register:
asm volatile("mov (%0,%1,1), %%eax\n\t"
:
: "r" (mem_r), "r" (i)
: "%eax", "memory");The behavior in this case would be that only the 32 bit lower part of the variable is passed. The variable is used as a counter in a for loop that iterates over the pseudo-locked region and its maximum value is thus the largest size of a pseudo-locked region. No pseudo-locked region currently supported would need more than 32-bits for its size. There is thus no potential for harm in the current state when using the u64 variable, but this should be fixed.
Modify the variable with which the registers are initialized to be 64-bit when used for 64-bit register and 32-bit when used for 32-bit register.
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +12 -0
@@ -415,7 +415,11 @@ static int pseudo_lock_fn(void *_rdtgrp)struct rdtgroup *rdtgrp = _rdtgrp;struct pseudo_lock_region *plr = rdtgrp->plr;u32 rmid_p, closid_p; +#ifdef CONFIG_X86_64 u64 i; +#else +u32 i; +#endif #ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
@@ -870,7 +874,11 @@ static int measure_cycles_lat_fn(void *_plr) {struct pseudo_lock_region *plr = _plr;u64 start, end; +#ifdef CONFIG_X86_64 u64 i; +#else +u32 i; +#endif #ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
@@ -924,7 +932,11 @@ static int measure_cycles_perf_fn(void *_plr)struct pseudo_lock_region *plr = _plr;unsigned long long l2_hits, l2_miss;u64 l2_hit_bits, l2_miss_bits; +#ifdef CONFIG_X86_64 u64 i; +#else +u32 i; +#endif #ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
Signature · 1 line
2.17.0
PATCH v7 41/41 x86/intel_rdt: Simplify index type
In reply to Reinette ChatreFrom: Ingo Molnar <[email protected]>
Simplify this pattern:
#ifdef CONFIG_X86_64 u64 i; #else u32 i;
#endif
... to the more natural and shorter one:
unsigned long i;
No change in functionality.
Acked-by Thomas Gleixner <[email protected]>
--- arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/intel_rdt_pseudo_lock.c +3 -15
@@ -415,11 +415,7 @@ static int pseudo_lock_fn(void *_rdtgrp)struct rdtgroup *rdtgrp = _rdtgrp;struct pseudo_lock_region *plr = rdtgrp->plr;u32 rmid_p, closid_p; -#ifdef CONFIG_X86_64 -u64 i; -#else -u32 i; -#endif +unsigned long i;#ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
@@ -874,11 +870,7 @@ static int measure_cycles_lat_fn(void *_plr) {struct pseudo_lock_region *plr = _plr;u64 start, end; -#ifdef CONFIG_X86_64 -u64 i; -#else -u32 i; -#endif +unsigned long i;#ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
@@ -932,11 +924,7 @@ static int measure_cycles_perf_fn(void *_plr)struct pseudo_lock_region *plr = _plr;unsigned long long l2_hits, l2_miss;u64 l2_hit_bits, l2_miss_bits; -#ifdef CONFIG_X86_64 -u64 i; -#else -u32 i; -#endif +unsigned long i;#ifdef CONFIG_KASAN /* * The registers used for local register variables are also used
Signature · 1 line
2.17.0
Do you have a public git branch with these patches on it somewhere?
Thanks,
David
Hi David,
Quoted 1 line · David Howells
Do you have a public git branch with these patches on it somewhere?
I just added them to branch cache-pseudo-locking/v7 of
https://github.com/rchatre/linux.git
Reinette
Reinette!
Quoted 8 lines · Reinette Chatre
The Cache Pseudo-Locking enabling series that was recently merged to the x86/cache branch of tip was found to conflict with the new kernfs support for mounting with
fs_context.In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
Thanks for doing this so quick! I've picked up the lot and slightly modified the first patch by moving the stubs to the header file to get them completely out of the conflicting area.
The immutable branch for David to pull is:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git for-vfs
I've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git for-vfs-merged
or pick up the patch below and do the merge and conflict resolution yourself.
The x86/cache branch is rebased and merges cleanly with for-vfs-merged. The last two patches are dropped; I folded them back to the patches they fix.
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/cache
Reinette, can you please verify that x86/cache + for-vfs-merged are doing the right thing?
Thanks,
tglx
8<----------------------------
Subject: x86/intel_rdt: Mop up the fs_context conversion
From: Thomas Gleixner <[email protected]>
Date: Sat, 23 Jun 2018 12:27:32 +0200
- Stick the struct into the local header file and not at some random place in the source.
- Get rid of the fugly camel case
- Move the enablement into a separate function
--- arch/x86/kernel/cpu/intel_rdt.h | 15 ++++++++ arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 54 ++++++++++++------------------- 2 files changed, 36 insertions(+), 33 deletions(-)
arch/x86/kernel/cpu/intel_rdt.h +15 -0
@@ -33,6 +33,21 @@#define RMID_VAL_ERROR BIT_ULL(63)#define RMID_VAL_UNAVAIL BIT_ULL(62) + +struct rdt_fs_context { +struct kernfs_fs_context kfc; +bool enable_cdpl2; +bool enable_cdpl3; +bool enable_mba_mbps; +}; + +static inline struct rdt_fs_context *rdt_fc2context (struct fs_context *fc) +{ +struct kernfs_fs_context *kfc = fc->fs_private; + + return container_of(kfc, struct rdt_fs_context, kfc); +} +DECLARE_STATIC_KEY_FALSE (rdt_enable_key);/**
arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +21 -33
85 lines not on this page · raw message
Quoted 20 lines · Thomas Gleixner
Reinette!
Quoted 8 lines · Reinette Chatre
The Cache Pseudo-Locking enabling series that was recently merged to the x86/cache branch of tip was found to conflict with the new kernfs support for mounting with
fs_context.In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
Thanks for doing this so quick! I've picked up the lot and slightly modified the first patch by moving the stubs to the header file to get them completely out of the conflicting area.
The immutable branch for David to pull is:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs
Quoted 8 lines
I've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs-mergedor pick up the patch below and do the merge and conflict resolution yourself.
I've force updated it after pushing out the variant which did not compile instead of folding the fix back first. New head is:
tip/for-vfs-merged 63b411121807: Merge branch 'for-vfs' into for-vfs-merged
/me hates computers and goes to mow the lawn
Quoted 6 lines · Thomas Gleixner
I've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs-merged
I have some other fixes folded into my copy, so I won't pull this, but thanks anyway.
David
Quoted 11 lines · David Howells
Quoted 6 lines · Thomas Gleixner
I've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs-mergedI have some other fixes folded into my copy, so I won't pull this, but thanks anyway.
Sure. I just did this to verify that the results merge cleanly so I pushed it out for reference or consumption.
Thanks,
tglx
Quoted 2 lines · Thomas Gleixner
- Stick the struct into the local header file and not at some random place in the source.
Why? It's only used in that one file. There doesn't seem to be any particular need to share it around.
Quoted 1 line
- Get rid of the fugly camel case
It's not camel case. It's the name of the option it's encoding. But if it makes you happy...
Quoted 1 line
- Move the enablement into a separate function
Okay.
David
Quoted 7 lines · David Howells
Quoted 2 lines · Thomas Gleixner
- Stick the struct into the local header file and not at some random place in the source.
Why? It's only used in that one file. There doesn't seem to be any particular need to share it around.
It's not in a global header. It's a local header exactly to stick stuff like that into it.
Quoted 4 lines
Quoted 1 line
- Get rid of the fugly camel case
It's not camel case. It's the name of the option it's encoding. But if it makes you happy...
Well, the option name slipped through, but that doesn't make an excuse for making the struct member equally ugly.
Thanks,
tglx
Hi Thomas,
Quoted 23 lines · Thomas Gleixner
Quoted 8 lines · Reinette Chatre
The Cache Pseudo-Locking enabling series that was recently merged to the x86/cache branch of tip was found to conflict with the new kernfs support for mounting with
fs_context.In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
Thanks for doing this so quick! I've picked up the lot and slightly modified the first patch by moving the stubs to the header file to get them completely out of the conflicting area.
The immutable branch for David to pull is:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfsI've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
Quoted 8 lines
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs-mergedor pick up the patch below and do the merge and conflict resolution yourself.
The x86/cache branch is rebased and merges cleanly with for-vfs-merged. The last two patches are dropped; I folded them back to the patches they fix.
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitx86/cache
Thank you very much for setting up all the branches to support a smooth merge.
Quoted 3 lines
Reinette, can you please verify that x86/cache + for-vfs-merged are doing the right thing?
Will do. While tracking down the bisect issue related to copy_from_user() I managed to lose connectivity to my test system. I will continue the testing when I am back in the office on Monday.
Reinette
Quoted 3 lines · Reinette Chatre
Will do. While tracking down the bisect issue related to
copy_from_user()I managed to lose connectivity to my test system. I will continue the testing when I am back in the office on Monday.
No worry. Enjoy your weekend!
Thanks,
tglx
Hi Thomas,
Quoted 27 lines · Thomas Gleixner
Quoted 8 lines · Reinette Chatre
The Cache Pseudo-Locking enabling series that was recently merged to the x86/cache branch of tip was found to conflict with the new kernfs support for mounting with
fs_context.In preparation for a conflict-free merge between the two repos some no-op hooks are created within the RDT mount function being changed by the two features. The goal is for this commit to be placed on a minimal no-rebase branch to be consumed by both features.
Thanks for doing this so quick! I've picked up the lot and slightly modified the first patch by moving the stubs to the header file to get them completely out of the conflicting area.
The immutable branch for David to pull is:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfsI've also created a branch based on Davids mount-context branch, applied the cleanup patch below to it and merged the for-vfs branch on top.
David feel free to pull the result from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitfor-vfs-mergedor pick up the patch below and do the merge and conflict resolution yourself.
Quoted 8 lines
The x86/cache branch is rebased and merges cleanly with for-vfs-merged. The last two patches are dropped; I folded them back to the patches they fix.
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitx86/cacheReinette, can you please verify that x86/cache + for-vfs-merged are doing the right thing?
I tested a merge of x86/cache and for-vfs-merged and Cache Pseudo-Locking works.
The for-vfs-merged changes modify how mount options are handled so I tried to cover this as well by testing on a (L3) CDP supporting system. Mounting resctrl with and without the cdp mount option behaves as expected. In fact, if I am looking right it seems that for-vfs-merged includes a fix for the current resctrl mount option handling. The resctrl filesystem supports mount options for CDP and the MBA software controller. At the moment, if an error occurs after the mount options are handled, only the CDP option handling is reverted, not the MBA software controller handling. The for-vfs-merged changes includes an additional snippet on the error path for handling of the MBA software controller mount option:
+out_mba:
+ if (ctx->enable_mba_mbps)
+ set_mba_sc(false);
Reinette