RFC DRM resource management cgroup, try 2.
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest6 patches 13 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/6 | mm/page_counter: Move calculating protection values to page_counterinclude/ mm/ mm/ | +180 | 2 replies | |
| 2/6 | drm/cgroup: Add memory accounting DRM cgroupDocumentation/ Documentation/ include/ +4 more | +1045 | 11 replies | |
| 3/6 | drm/ttm: Handle cgroup based eviction in TTMdrivers/ drivers/ drivers/ +3 more | +84 | 0 replies | |
| 4/6 | drm/xe: Implement cgroup for vramdrivers/ drivers/ drivers/ | +18 | 0 replies | |
| 5/6 | drm/amdgpu: Add cgroups implementationdrivers/ drivers/ drivers/ | +14 | 0 replies | |
| 6/6 | drm/xe: Hack to test with mapped pages instead of vram.drivers/ | +14 | 0 replies |
Reviews trailers per patch
1 of 6 patches has a Reviewed-by or Acked-by.
| Patch | Roman Gushchin | Shakeel Butt |
|---|---|---|
| 1/6 | A-b | A-b |
| 2/6 | ||
| 3/6 | ||
| 4/6 | ||
| 5/6 | ||
| 6/6 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [RFC PATCH 0/6] DRM resource management cgroup, try 2.
Hey,
A new version of my attempt at managing VRAM through cgroups.
Even though it's called the DRM resource management cgroup, it would be trivial to rename it to devmem or whatever, since there is nothing DRM specific about it.
This series allows setting limits on VRAM similar to system memory, with min/low/max limits.
This allows various cgroups to have their own limits for usage.
It sounds very abstract, but it can be used to prioritise the foreground application (by setting low), or hard partition memory so multiple processes sharing a single GPU use a proportional amount of memory each in a fair way, or to prevent long running compute jobs from having their memory evicted.
This is a minimal proof of concept to get discussion going again. It works, but it only tracks active use of VRAM. In the ideal world, we would track it better in a way that also integrates better with the memory cgroup controller. Ideally for every VRAM allocation, we would know we could push it out to swap if needed, charging the original process not the process evicting.
I'm hoping to restart the discussion, so that we can plug the holes and finally move forward.
New in this version:
- Complete rewrite using page_counter.
- Support setting min/low/max, respected in the same way as memory cgroup. (Could be useful to add/allow high? To go over limit for temporary bindings during eviction on GART.)
- Locking reworked. Fastpath should now be lockless with RCU.
- Add a second implementation for AMD, to show how easy it is to make it work. (Should we completely move this to TTM instead?)
- TTM now always respects min/low when evicting, bailing out with -ENOSPC instead where required.
I'm hoping for some good feedback on the path forward for upstreaming. I feel this version has a lot better chance of being upstreamed than the previous. It should be a lot more scalable thanks to the usage of RCU and page_counter.
Cheers,
Maarten
Maarten Lankhorst (6):
mm/page_counter: Move calculating protection values to page_counter drm/cgroup: Add memory accounting DRM cgroup
drm/ttm: Handle cgroup based eviction in TTM
drm/xe: Implement cgroup for vram
drm/amdgpu: Add cgroups implementation
drm/xe: Hack to test with mapped pages instead of vram.
Documentation/admin-guide/cgroup-v2.rst | 51 ++ Documentation/gpu/drm-compute.rst | 54 ++ drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 6 + drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 + drivers/gpu/drm/ttm/tests/ttm_bo_test.c | 18 +- drivers/gpu/drm/ttm/tests/ttm_resource_test.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 38 +- drivers/gpu/drm/ttm/ttm_resource.c | 28 +- drivers/gpu/drm/xe/xe_device.c | 4 + drivers/gpu/drm/xe/xe_device_types.h | 4 + drivers/gpu/drm/xe/xe_ttm_sys_mgr.c | 14 + drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 10 + include/drm/ttm/ttm_bo.h | 3 +- include/drm/ttm/ttm_resource.h | 16 +- include/linux/cgroup_drm.h | 115 +++ include/linux/cgroup_subsys.h | 4 + include/linux/page_counter.h | 4 + init/Kconfig | 7 + kernel/cgroup/Makefile | 1 + kernel/cgroup/drm.c | 813 ++++++++++++++++++ mm/memcontrol.c | 154 +--- mm/page_counter.c | 173 ++++ 23 files changed, 1355 insertions(+), 172 deletions(-) create mode 100644 Documentation/gpu/drm-compute.rst create mode 100644 include/linux/cgroup_drm.h create mode 100644 kernel/cgroup/drm.c
Signature · 1 line
2.45.2