RFC Introduce DRM dep queue
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest12 patches 48 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/12 | workqueue: Add interface to teach lockdep to warn on reclaim violationsinclude/ kernel/ | +44 | 5 replies | |
| 2/12 | drm/dep: Add DRM dependency queue layerdrivers/ drivers/ drivers/ +7 more | +3404 | 42 replies | |
| 3/12 | drm/xe: Use WQ_MEM_WARN_ON_RECLAIM on all workqueues in the reclaim pathdrivers/ drivers/ drivers/ +2 more | +10 | 0 replies | |
| 4/12 | drm/xe: Issue GGTT invalidation under lock in ggtt_node_removedrivers/ | +2 | 1 reply | |
| 5/12 | drm/xe: Return fence from xe_sched_job_arm and adjust job referencesdrivers/ drivers/ drivers/ +7 more | +49 | 0 replies | |
| 6/12 | drm/xe: Convert to DRM dep queue scheduler layerdrivers/ drivers/ drivers/ +16 more | +255 | 0 replies | |
| 7/12 | drm/xe: Make scheduler message lock IRQ-safedrivers/ drivers/ drivers/ | +32 | 0 replies | |
| 8/12 | drm/xe: Rework exec queue object on top of DRM depdrivers/ drivers/ drivers/ +8 more | +143 | 0 replies | |
| 9/12 | drm/xe: Enable IRQ job put in DRM depdrivers/ drivers/ drivers/ | +6 | 0 replies | |
| 10/12 | drm/xe: Use DRM dep queue kill semanticsdrivers/ drivers/ | +97 | 0 replies | |
| 11/12 | accel/amdxdna: Convert to drm_dep scheduler layerdrivers/ drivers/ drivers/ +3 more | +88 | 0 replies | |
| 12/12 | drm/panthor: Convert to drm_dep scheduler layerdrivers/ drivers/ drivers/ +5 more | +223 | 0 replies |
Reviews trailers per patch
2 of 12 patches have a Reviewed-by or Acked-by.
| Patch | Matthew Brost | Tejun Heo | Dnyaneshwar Bhadane |
|---|---|---|---|
| 1/12 | A-b | ||
| 2/12 | |||
| 3/12 | |||
| 4/12 | Signed-by | R-b | |
| 5/12 | |||
| 6/12 | |||
| 7/12 | |||
| 8/12 | |||
| 9/12 | |||
| 10/12 | |||
| 11/12 | |||
| 12/12 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [RFC PATCH 00/12] Introduce DRM dep queue
Diverging requirements between GPU drivers using firmware scheduling and those using hardware scheduling have shown that drm_gpu_scheduler is no longer sufficient for firmware-scheduled GPU drivers. The technical debt, lack of memory-safety guarantees, absence of clear object-lifetime rules, and numerous driver-specific hacks have rendered drm_gpu_scheduler unmaintainable. It is time for a fresh design for firmware-scheduled GPU drivers—one that addresses all of the aforementioned shortcomings.
Key changes from DRM scheduler:
- Unification of drm_gpu_scheduler, drm_sched_entity into drm_dep_queue
- Reference counting
- Clear object-lifetime ownership
- Privatisation of drm dep fence
- Drop scheduled fence from drm dep fence
- Submit workqueue bypass path (optional)
- IRQ removal of jobs (i.e., bypass worker to drop ref to job) (optional)
- Asynchronous teardown
- Extensive lockdep annotations and asserts
- Extensive kernel doc
Xe has been fully converted to drm dep and extensively tested with and without verbose kernel debug Kconfig options. Everything seems to be working locally. All permutations of drm_dep_queue_flags tested.
Compile tested only conversions - Panthor and AMDXDNA.
Other candidates to convert are: Nouveau, PVR, and AMDGPU VM queue.
Forward looking - this design seems suitable for Rust bindings and use by Nova, Tyr given the clean object-lifetime rules and reference counting.
The Xe team is aligned on moving forward with this; hopefully the rest of the community embraces it.
In addition to DRM dep, workqueues have been updated with interfaces to enforce reclaim safety with DRM dep, and Xe is utilizing these interfaces. This part could be split out into a separate series.
I'm also fairly certain that a sane HW queue component could be built on top of DRM dep as well, replacing DRM sched for all drivers they choose.
Matt
Matthew Brost (12):
workqueue: Add interface to teach lockdep to warn on reclaim
violations
drm/dep: Add DRM dependency queue layer
drm/xe: Use WQ_MEM_WARN_ON_RECLAIM on all workqueues in the reclaim
path
drm/xe: Issue GGTT invalidation under lock in ggtt_node_remove drm/xe: Return fence from xe_sched_job_arm and adjust job references drm/xe: Convert to DRM dep queue scheduler layer
drm/xe: Make scheduler message lock IRQ-safe
drm/xe: Rework exec queue object on top of DRM dep drm/xe: Enable IRQ job put in DRM dep
drm/xe: Use DRM dep queue kill semantics
accel/amdxdna: Convert to drm_dep scheduler layer drm/panthor: Convert to drm_dep scheduler layer
drivers/accel/amdxdna/Kconfig | 2 +- drivers/accel/amdxdna/aie2_ctx.c | 144 +- drivers/accel/amdxdna/aie2_pci.h | 4 +- drivers/accel/amdxdna/amdxdna_ctx.c | 5 +- drivers/accel/amdxdna/amdxdna_ctx.h | 4 +- drivers/gpu/drm/Kconfig | 4 + drivers/gpu/drm/Makefile | 1 + drivers/gpu/drm/dep/Makefile | 5 + drivers/gpu/drm/dep/drm_dep_fence.c | 406 +++++ drivers/gpu/drm/dep/drm_dep_fence.h | 25 + drivers/gpu/drm/dep/drm_dep_job.c | 675 +++++++ drivers/gpu/drm/dep/drm_dep_job.h | 13 + drivers/gpu/drm/dep/drm_dep_queue.c | 1647 ++++++++++++++++++ drivers/gpu/drm/dep/drm_dep_queue.h | 31 + drivers/gpu/drm/panthor/Kconfig | 2 +- drivers/gpu/drm/panthor/panthor_device.c | 5 +- drivers/gpu/drm/panthor/panthor_device.h | 2 +- drivers/gpu/drm/panthor/panthor_drv.c | 35 +- drivers/gpu/drm/panthor/panthor_mmu.c | 160 +- drivers/gpu/drm/panthor/panthor_mmu.h | 14 +- drivers/gpu/drm/panthor/panthor_sched.c | 242 ++- drivers/gpu/drm/panthor/panthor_sched.h | 12 +- drivers/gpu/drm/xe/Kconfig | 2 +- drivers/gpu/drm/xe/tests/xe_migrate.c | 10 +- drivers/gpu/drm/xe/xe_dep_job_types.h | 8 +- drivers/gpu/drm/xe/xe_dep_scheduler.c | 81 +- drivers/gpu/drm/xe/xe_dep_scheduler.h | 7 +- drivers/gpu/drm/xe/xe_device.c | 3 +- drivers/gpu/drm/xe/xe_exec.c | 12 +- drivers/gpu/drm/xe/xe_exec_queue.c | 96 +- drivers/gpu/drm/xe/xe_exec_queue.h | 9 +- drivers/gpu/drm/xe/xe_exec_queue_types.h | 10 +- drivers/gpu/drm/xe/xe_execlist.c | 39 +- drivers/gpu/drm/xe/xe_execlist_types.h | 4 - drivers/gpu/drm/xe/xe_ggtt.c | 12 +- drivers/gpu/drm/xe/xe_gpu_scheduler.c | 66 +- drivers/gpu/drm/xe/xe_gpu_scheduler.h | 67 +- drivers/gpu/drm/xe/xe_gpu_scheduler_types.h | 9 +- drivers/gpu/drm/xe/xe_gsc.c | 5 +- drivers/gpu/drm/xe/xe_gsc_submit.c | 5 +- drivers/gpu/drm/xe/xe_gt.c | 8 +- drivers/gpu/drm/xe/xe_guc_ct.c | 3 +- drivers/gpu/drm/xe/xe_guc_exec_queue_types.h | 14 +- drivers/gpu/drm/xe/xe_guc_submit.c | 360 ++-- drivers/gpu/drm/xe/xe_migrate.c | 27 +- drivers/gpu/drm/xe/xe_oa.c | 5 +- drivers/gpu/drm/xe/xe_pt.c | 2 +- drivers/gpu/drm/xe/xe_pxp_submit.c | 10 +- drivers/gpu/drm/xe/xe_sched_job.c | 59 +- drivers/gpu/drm/xe/xe_sched_job.h | 9 +- drivers/gpu/drm/xe/xe_sched_job_types.h | 8 +- drivers/gpu/drm/xe/xe_sync.c | 2 +- drivers/gpu/drm/xe/xe_tlb_inval.c | 3 +- drivers/gpu/drm/xe/xe_tlb_inval_job.c | 86 +- include/drm/drm_dep.h | 597 +++++++ include/linux/workqueue.h | 3 + include/trace/events/amdxdna.h | 12 +- kernel/workqueue.c | 41 + 58 files changed, 4268 insertions(+), 864 deletions(-) create mode 100644 drivers/gpu/drm/dep/Makefile create mode 100644 drivers/gpu/drm/dep/drm_dep_fence.c create mode 100644 drivers/gpu/drm/dep/drm_dep_fence.h create mode 100644 drivers/gpu/drm/dep/drm_dep_job.c create mode 100644 drivers/gpu/drm/dep/drm_dep_job.h create mode 100644 drivers/gpu/drm/dep/drm_dep_queue.c create mode 100644 drivers/gpu/drm/dep/drm_dep_queue.h create mode 100644 include/drm/drm_dep.h
Signature · 1 line
2.34.1