Skip to content

drm/xe: Protect against premature module unloads

Thomas Hellström intel-xe sashiko-reviews 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest3 patches 3 repliesIn review
  2. v33 patches 10 repliesSuperseded
  3. v23 patches 4 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v13 patches0 repliesSuperseded

Changes in v4 from the cover letter

- Added a patch fixing a similar unprotected teardown gap in execlist exec queue destruction
- Dropped the core DRM patch that provided a global SRCU-based drm_dev_release_barrier() and the corresponding call to it in xe's module-exit path, in favor of relying solely on the xe_device instance count already tracked by this series
- Add code comments (patch 2)
- Use the xe->destroy_wq for vm->destroy_work (patch 2, Matt Brost)

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3drm/xe: Don't unload the driver until all drm devices are freeddrivers/gpu/drm/xe/xe_device.c drivers/gpu/drm/xe/xe_device.h drivers/gpu/drm/xe/xe_module.c+48 -11 reply
2/3drm/xe: Route deferred xe_vma/xe_vm teardown off system_dfl_wqdrivers/gpu/drm/xe/xe_module.c drivers/gpu/drm/xe/xe_vm.c+19 -50 replies
3/3drm/xe: Route execlist exec queue teardown off system_dfl_wqdrivers/gpu/drm/xe/xe_execlist.c+5 -11 reply

Reviews trailers per patch

All 3 patches have a Reviewed-by or Acked-by.

PatchMatthew Brost
1/3R-b
2/3R-b
3/3R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH v4 0/3] drm/xe: Protect against premature module unloads

Driver code is increasingly relying on bare drm_device references (drm_dev_get()/drm_dev_put()) to keep a device's software state around, without also pairing that with a reference on the owning kernel module. Xe itself does this in several places, and so does drm_gpuvm (used by xe) for the lifetime of a GPU VM. None of these references currently prevent the xe module from being unloaded while they, or the teardown work they can still trigger, are outstanding, meaning driver code can end up executing after the module's own text has already been freed.

This series closes that gap for xe:

- Patch 1 makes xe hold up module unload until every xe_device instance has actually been released, rather than only until the module's own refcount happens to reach zero, with a diagnostic if this ends up taking an unexpectedly long time.

- Patch 2 fixes a related, previously unprotected case where the teardown of a GPU VM or its address space mappings can be deferred to run at an arbitrary later time, including after module unload has already completed.

- Patch 3 fixes the same class of problem for execlist exec queue teardown, which could likewise be deferred past module unload.

Together, these changes ensure rmmod xe cannot free the module's memory while any of its devices, or asynchronous work stemming from them, might still be executing.

v2:
- Updated the commit message of patch 1 (then patch 2) to describe the actual implementation (a single 20s wait_var_event_timeout() followed by one pr_warn() and an unbounded wait_var_event(), rather than a loop retrying with a diagnostic every 10s) and its uninterruptible-sleep tradeoff (sashiko)

v3:
- drm_dev_release_barrier() now uses a single, global SRCU domain shared by all drivers instead of requiring each driver to register its own via a new &drm_driver.release_srcu field, accepting that a driver's call may then occasionally block on unrelated drivers' release callbacks (patch 1, patch 2, Christian König). This entire patch was later dropped in v4.

v4:
- Added a patch fixing a similar unprotected teardown gap in execlist exec queue destruction
- Dropped the core DRM patch that provided a global SRCU-based drm_dev_release_barrier() and the corresponding call to it in xe's module-exit path, in favor of relying solely on the xe_device instance count already tracked by this series
- Add code comments (patch 2)
- Use the xe->destroy_wq for vm->destroy_work (patch 2, Matt Brost)

Thomas Hellström (3):
drm/xe: Don't unload the driver until all drm devices are freed drm/xe: Route deferred xe_vma/xe_vm teardown off system_dfl_wq drm/xe: Route execlist exec queue teardown off system_dfl_wq

 drivers/gpu/drm/xe/xe_device.c   | 29 +++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_device.h   |  2 ++
 drivers/gpu/drm/xe/xe_execlist.c |  6 +++++-
 drivers/gpu/drm/xe/xe_module.c   | 24 +++++++++++++++++++++---
 drivers/gpu/drm/xe/xe_vm.c       | 18 +++++++++++++++---
 5 files changed, 72 insertions(+), 7 deletions(-)
Signature · 1 line
2.55.0