drm/xe/madvise: Add support for purgeable buffer objects
Subsystems (3)
v4 cover drm/xe/madvise: Add support for purgeable buffer objects
This patch series introduces comprehensive support for purgeable buffer objects in the Xe driver, enabling userspace to provide memory usage hints for better memory management under system pressure.
Overview:
Purgeable memory allows applications to mark buffer objects as "not currently needed" (DONTNEED), making them eligible for kernel reclamation during memory pressure. This helps prevent OOM conditions and enables more efficient GPU memory utilization for workloads with temporary or regeneratable data (caches, intermediate results, decoded frames, etc.).
Purgeable BO Lifecycle:
1. WILLNEED (default): BO actively needed, kernel preserves backing store
2. DONTNEED (user hint): BO contents discardable, eligible for purging
3. PURGED (kernel action): Backing store reclaimed during memory pressure
Key Design Principles:
- i915 compatibility: "Once purged, always purged" semantics - purged BOs remain permanently invalid and must be destroyed/recreated
- Per-VMA state tracking: Each VMA tracks its own purgeable state, BO is only marked DONTNEED when ALL VMAs across ALL VMs agree (Thomas Hellström)
- Safety first: Imported/exported dma-bufs blocked from purgeable state - no visibility into external device usage (Matt Roper)
- Multiple protection layers: Validation in madvise, VM bind, mmap, and fault handlers
- Async TLB invalidation: Uses xe_bo_trigger_rebind() for non-blocking GPU mapping invalidation
- Scratch PTE support: Fault-mode VMs use scratch pages for safe zero reads on purged BO access.
- Purgeable state is not applied to imported/exported dma-bufs, those BOs always behave as WILLNEED.
- TTM shrinker integration: Encapsulated helpers manage xe_ttm_tt->purgeable flag and shrinker page accounting (shrinkable vs purgeable buckets)
v2 Changes:
- Reordered patches: Moved shared BO helper before main implementation for proper dependency order
- Fixed reference counting in mmap offset validation (use drm_gem_object_put)
- Removed incorrect claims about madvise(WILLNEED) restoring purged BOs
- Fixed error code documentation inconsistencies
- Initialize purge_state_val fields to prevent kernel memory leaks
- Use xe_bo_trigger_rebind() for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
v3 Changes (addressing Matt and Thomas Hellström feedback):
- Per-VMA purgeable state tracking: Added xe_vma->purgeable_state field
- Complete VMA check: xe_bo_all_vmas_dontneed() walks all VMAs across all VMs to ensure unanimous DONTNEED before marking BO purgeable
- VMA unbind recheck: Added xe_bo_recheck_purgeable_on_vma_unbind() to re-evaluate BO state when VMAs are destroyed
- Block external dma-bufs: Added xe_bo_is_external_dmabuf() check using
drm_gem_is_imported() and obj->dma_buf to prevent purging imported/exported BOs
- Consistent lockdep enforcement: Added xe_bo_assert_held() to all helpers that access madv_purgeable state
- Simplified page table logic: Renamed is_null to is_null_or_purged in xe_pt_stage_bind_entry() - purged BOs treated identically to null VMAs
- Removed unnecessary checks: Dropped redundant "&& bo" check in xe_ttm_bo_purge()
- Xe-specific warnings: Changed drm_warn() to XE_WARN_ON() in purge path
- Moved purge checks under locks: Purge state validation now done after acquiring dma-resv lock in vma_lock_and_validate() and xe_pagefault_begin()
- Race-free fault handling: Removed unlocked purge check from xe_pagefault_handle_vma(), moved to locked xe_pagefault_begin()
- Shrinker helper functions: Added xe_bo_set_purgeable_shrinker() and xe_bo_clear_purgeable_shrinker() to encapsulate TTM purgeable flag updates and shrinker page accounting, improving code clarity and maintainability
v4 Changes (addressing Matt and Thomas Hellström feedback):
- UAPI: Removed '__u64 reserved' field from purge_state_val union to fit 16-byte size constraint (Matt)
- Changed madv_purgeable from atomic_t to u32 across all patches (Matt)
- CPU fault handling: Added purged check to fastpath (xe_bo_cpu_fault_fastpath) to prevent hang when accessing existing mmap of purged BO
Arvind Yadav (7):
drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo drm/xe/madvise: Implement purgeable buffer object support drm/xe/bo: Handle CPU faults on purged buffer objects drm/xe/vm: Prevent binding of purged buffer objects drm/xe/madvise: Implement per-VMA purgeable state tracking drm/xe/madvise: Block imported and exported dma-bufs drm/xe/bo: Add purgeable shrinker state helpers
Himal Prasad Ghimiray (1):
drm/xe/uapi: Add UAPI support for purgeable buffer objects
drivers/gpu/drm/xe/xe_bo.c | 127 +++++++++++++++-- drivers/gpu/drm/xe/xe_bo.h | 59 ++++++++ drivers/gpu/drm/xe/xe_bo_types.h | 3 + drivers/gpu/drm/xe/xe_pagefault.c | 12 ++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++- drivers/gpu/drm/xe/xe_vm.c | 46 +++++-- drivers/gpu/drm/xe/xe_vm_madvise.c | 210 +++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_vm_madvise.h | 3 + drivers/gpu/drm/xe/xe_vm_types.h | 11 ++ include/uapi/drm/xe_drm.h | 37 +++++ 10 files changed, 519 insertions(+), 27 deletions(-)
Signature · 1 line
2.43.0
PATCH v4 1/8 drm/xe/uapi: Add UAPI support for purgeable buffer objects
From: Himal Prasad Ghimiray <[email protected]>
Extend the DRM_XE_MADVISE ioctl to support purgeable buffer object management by adding DRM_XE_VMA_ATTR_PURGEABLE_STATE attribute type.
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
- WILLNEED: Buffer is needed and should not be purged. If the BO was previously purged, retained field returns 0 indicating backing store was lost (once purged, always purged semantics matching i915).
- DONTNEED: Buffer is not currently needed and may be purged by the kernel under memory pressure to free resources. Only applies to non-shared BOs.
The implementation includes a 'retained' output field (matching i915's drm_i915_gem_madvise.retained) that indicates whether the BO's backing store still exists (1) or has been purged (0).
v2:
- Add PURGED state for read-only status, change ioctl to DRM_IOWR, add retained field for i915 compatibility
v3:
- UAPI rule should not be changed (Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)
v4:
- You cannot make this part of the union (purge_state_val) larger than the existing union (16 bytes). So just drop the '__u64 reserved' field. (Matt)
--- include/uapi/drm/xe_drm.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
include/uapi/drm/xe_drm.h +37 -0
@@ -2099,6 +2099,7 @@ struct drm_xe_madvise {#define DRM_XE_MEM_RANGE_ATTR_PREFERRED_LOC 0#define DRM_XE_MEM_RANGE_ATTR_ATOMIC 1#define DRM_XE_MEM_RANGE_ATTR_PAT 2 +#define DRM_XE_VMA_ATTR_PURGEABLE_STATE 3/** @type: type of attribute */ __u32 type;
@@ -2189,6 +2190,42 @@ struct drm_xe_madvise {/** @pat_index.reserved: Reserved */ __u64 reserved; } pat_index; + +/** +* @purge_state_val: Purgeable state configuration +* +* Used when @type == DRM_XE_VMA_ATTR_PURGEABLE_STATE. +* +* Configures the purgeable state of buffer objects in the specified +* virtual address range. This allows applications to hint to the kernel +* about bo's usage patterns for better memory management. +* +* Supported values for @purge_state_val.val: +* - DRM_XE_VMA_PURGEABLE_STATE_WILLNEED (0): Marks BO as needed. +* If BO was purged, returns retained=0 (backing store lost). +* +* - DRM_XE_VMA_PURGEABLE_STATE_DONTNEED (1): Hints that BO is not +* currently needed. Kernel may purge it under memory pressure. +* Only applies to non-shared BOs. Returns retained=1 if not purged. +*/ +struct { +#define DRM_XE_VMA_PURGEABLE_STATE_WILLNEED 0 +#define DRM_XE_VMA_PURGEABLE_STATE_DONTNEED 1 +/** @purge_state_val.val: value for DRM_XE_VMA_ATTR_PURGEABLE_STATE */ +__u32 val; + +/* @purge_state_val.pad */ +__u32 pad; +/** +* @purge_state_val.retained: Pointer to output field for backing +* store status. +* +* Userspace provides a pointer to u32. Kernel writes to it: +* 1 if backing store exists, 0 if purged. +* Similar to i915's drm_i915_gem_madvise.retained field. +*/ +__u64 retained; + } purge_state_val; };/** @reserved: Reserved */
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 1/8] drm/xe/uapi: Add UAPI support for purgeable buffer objects
Quoted 85 lines · Arvind Yadav
From: Himal Prasad Ghimiray <[email protected]>
Extend the
DRM_XE_MADVISEioctl to support purgeable buffer object management by addingDRM_XE_VMA_ATTR_PURGEABLE_STATEattribute type.This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
- WILLNEED: Buffer is needed and should not be purged. If the BO was previously purged, retained field returns 0 indicating backing store was lost (once purged, always purged semantics matching i915).
- DONTNEED: Buffer is not currently needed and may be purged by the kernel under memory pressure to free resources. Only applies to non-shared BOs.
The implementation includes a 'retained' output field (matching i915's drm_i915_gem_madvise.retained) that indicates whether the BO's backing store still exists (1) or has been purged (0).
v2:
- Add PURGED state for read-only status, change ioctl toDRM_IOWR, add retained field for i915 compatibilityv3:
- UAPI rule should not be changed (Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- You cannot make this part of the union (purge_state_val) larger than the existing union (16 bytes). So just drop the '__u64reserved' field. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Signed-off-by Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- include/uapi/drm/xe_drm.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)include/uapi/drm/xe_drm.h +26
-0@@ -2099,6 +2099,7 @@ struct drm_xe_madvise {#define DRM_XE_MEM_RANGE_ATTR_PREFERRED_LOC 0#define DRM_XE_MEM_RANGE_ATTR_ATOMIC 1#define DRM_XE_MEM_RANGE_ATTR_PAT 2 +#define DRM_XE_VMA_ATTR_PURGEABLE_STATE 3/** @type: type of attribute */ __u32 type;@@ -2189,6 +2190,42 @@ struct drm_xe_madvise {/** @pat_index.reserved: Reserved */ __u64 reserved; } pat_index; + +/** +* @purge_state_val: Purgeable state configuration +* +* Used when @type == DRM_XE_VMA_ATTR_PURGEABLE_STATE. +* +* Configures the purgeable state of buffer objects in the specified +* virtual address range. This allows applications to hint to the kernel +* about bo's usage patterns for better memory management. +* +* Supported values for @purge_state_val.val: +* - DRM_XE_VMA_PURGEABLE_STATE_WILLNEED (0): Marks BO as needed. +* If BO was purged, returns retained=0 (backing store lost). +* +* - DRM_XE_VMA_PURGEABLE_STATE_DONTNEED (1): Hints that BO is not +* currently needed. Kernel may purge it under memory pressure. +* Only applies to non-shared BOs. Returns retained=1 if not purged. +*/ +struct { +#define DRM_XE_VMA_PURGEABLE_STATE_WILLNEED 0 +#define DRM_XE_VMA_PURGEABLE_STATE_DONTNEED 1 +/** @purge_state_val.val: value for DRM_XE_VMA_ATTR_PURGEABLE_STATE */ +__u32 val; + +/* @purge_state_val.pad */
include/uapi/drm/xe_drm.h struct drm_xe_madvise { · line 2227
Quoted 10 lines
include/uapi/drm/xe_drm.h +10
-0+__u32 pad; +/** +* @purge_state_val.retained: Pointer to output field for backing +* store status. +* +* Userspace provides a pointer to u32. Kernel writes to it: +* 1 if backing store exists, 0 if purged. +* Similar to i915's drm_i915_gem_madvise.retained field. +*/ +__u64 retained;
@Rodrigo + @Thomas.
Here we are adding a userptr for copying the result back to userspace because the existing madvise IOCTL is IOW. If we could change this IOCTL to IOWR, we could let DRM handle this for us. I'm not sure whether changing an IOCTL from IOW to IOWR is allowed. My understanding is that the ioctl encoding is basically the same aside from some control bits, so switching from IOW → IOWR shouldn’t break anything, but I'm unclear on the rules.
What do you two think?
Matt
Quoted 7 lines
include/uapi/drm/xe_drm.h +1
-0+ } purge_state_val; };/** @reserved: Reserved */ Signature · 1 line
2.43.0
Vivi, Rodrigo
maintainerRe: [PATCH v4 1/8] drm/xe/uapi: Add UAPI support for purgeable buffer objects
Quoted 121 lines · Matthew Brost
Quoted 115 lines · Arvind Yadav
From: Himal Prasad Ghimiray <[email protected]>
Extend the
DRM_XE_MADVISEioctl to support purgeable buffer object management by addingDRM_XE_VMA_ATTR_PURGEABLE_STATEattribute type.This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
- WILLNEED: Buffer is needed and should not be purged. If the BO was
previously purged, retained field returns 0 indicating backing store
was lost (once purged, always purged semantics matching i915).- DONTNEED: Buffer is not currently needed and may be purged by the kernel under memory pressure to free resources. Only applies to non-shared BOs.
The implementation includes a 'retained' output field (matching i915's
drm_i915_gem_madvise.retained) that indicates whether the BO's backing
store still exists (1) or has been purged (0).v2:
- Add PURGED state for read-only status, change ioctl toDRM_IOWR,
add retained field for i915 compatibilityv3:
- UAPI rule should not be changed (Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- You cannot make this part of the union (purge_state_val) larger than the existing union (16 bytes). So just drop the '__u64reserved'
field. (Matt)Cc: Matthew Brost <[email protected]>
Cc: Thomas Hellström <[email protected]>
Signed-off-by: Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
include/uapi/drm/xe_drm.h | 37
+++++++++++++++++++++++++++++++++++++1 file changed, 37 insertions(+)include/uapi/drm/xe_drm.h +1
-0@@ -2099,6 +2099,7 @@ struct drm_xe_madvise { #define DRM_XE_MEM_RANGE_ATTR_PREFERRED_LOC 0 #define DRM_XE_MEM_RANGE_ATTR_ATOMIC 1 #define DRM_XE_MEM_RANGE_ATTR_PAT 2 +#define DRM_XE_VMA_ATTR_PURGEABLE_STATE 3 /** @type: type of attribute */ __u32 type;include/uapi/drm/xe_drm.h +5
-0@@ -2189,6 +2190,42 @@ struct drm_xe_madvise { /** @pat_index.reserved: Reserved */ __u64 reserved; } pat_index; + +/** +* @purge_state_val: Purgeable state configuration +* +* Used when @type ==
DRM_XE_VMA_ATTR_PURGEABLE_STATE.include/uapi/drm/xe_drm.h +2
-0+* +* Configures the purgeable state of buffer objects in the specified
include/uapi/drm/xe_drm.h +1
-0+* virtual address range. This allows applications to hint to the kernel
include/uapi/drm/xe_drm.h +1
-0+* about bo's usage patterns for better memory management.
include/uapi/drm/xe_drm.h +3
-0+* +* Supported values for @purge_state_val.val: +* - DRM_XE_VMA_PURGEABLE_STATE_WILLNEED (0): Marks BO as needed.
include/uapi/drm/xe_drm.h +1
-0+ * If BO was purged, returns retained=0(backing store lost).
include/uapi/drm/xe_drm.h +2
-0+ * + * - DRM_XE_VMA_PURGEABLE_STATE_DONTNEED (1):Hints that BO is not
include/uapi/drm/xe_drm.h +1
-0+ * currently needed. Kernel may purge it undermemory pressure.
include/uapi/drm/xe_drm.h +1
-0+ * Only applies to non-shared BOs. Returnsretained=1 if not purged.
include/uapi/drm/xe_drm.h +5
-0+*/ +struct { +#define DRM_XE_VMA_PURGEABLE_STATE_WILLNEED 0 +#define DRM_XE_VMA_PURGEABLE_STATE_DONTNEED 1 +/** @purge_state_val.val: value for
DRM_XE_VMA_ATTR_PURGEABLE_STATE*/include/uapi/drm/xe_drm.h +6
-0+__u32 val; + +/* @purge_state_val.pad */ +__u32 pad; +/** +* @purge_state_val.retained: Pointer to output field for backing
include/uapi/drm/xe_drm.h +3
-0+* store status. +* +* Userspace provides a pointer to u32. Kernel writes to it:
include/uapi/drm/xe_drm.h +2
-0+* 1 if backing store exists, 0 if purged. +* Similar to i915's drm_i915_gem_madvise.retained field.
include/uapi/drm/xe_drm.h
Quoted 2 lines
include/uapi/drm/xe_drm.h +2
-0+*/ +__u64 retained;@Rodrigo + @Thomas.
Quoted 11 lines
Here we are adding a userptr for copying the result back to userspace because the existing madvise IOCTL is IOW. If we could change this IOCTL
to IOWR, we could let DRM handle this for us. I'm not sure whether changing an IOCTL from IOW to IOWR is allowed. My understanding is that
the ioctl encoding is basically the same aside from some control bits,
so switching from IOW → IOWR shouldn’t break anything, but I'm unclear
on the rules.
I'm not so confident it wouldn't break anything as I'm wondering if this gets into the final number construction... and old userspace with new kernel would hit a different slot.
Quoted 2 lines
What do you two think?
Perhaps we would need a new ioctl and deprecate this with time?
Quoted 10 lines
Matt
Quoted 7 lines
include/uapi/drm/xe_drm.h +1
-0+ } purge_state_val;};
/** @reserved: Reserved */Signature · 1 line
2.43.0
PATCH v4 2/8 drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo
In reply to Arvind YadavAdd infrastructure for tracking purgeable state of buffer objects. This includes:
Introduce enum xe_madv_purgeable_state with three states:
- XE_MADV_PURGEABLE_WILLNEED (0): BO is needed and should not be purged. This is the default state for all BOs.
- XE_MADV_PURGEABLE_DONTNEED (1): BO is not currently needed and can be purged by the kernel under memory pressure to reclaim resources. Only non-shared BOs can be marked as DONTNEED.
- XE_MADV_PURGEABLE_PURGED (2): BO has been purged by the kernel. Accessing a purged BO results in error. Follows i915 semantics where once purged, the BO remains permanently invalid ("once purged, always purged").
Add atomic_t madv field to struct xe_bo for state tracking
of purgeable state across concurrent access paths
v2:
- Add xe_bo_is_purged() helper, improve state documentation
v3:
- Add the kernel doc(Matthew Brost)
- Add the new helpers xe_bo_madv_is_dontneed(Matthew Brost)
v4:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
--- drivers/gpu/drm/xe/xe_bo.h | 56 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_bo_types.h | 3 ++ 2 files changed, 59 insertions(+)
drivers/gpu/drm/xe/xe_bo.h +56 -0
@@ -86,6 +86,28 @@#define XE_PCI_BARRIER_MMAP_OFFSET (0x50 << XE_PTE_SHIFT) +/** +* enum xe_madv_purgeable_state - Buffer object purgeable state enumeration +* +* This enum defines the possible purgeable states for a buffer object, +* allowing userspace to provide memory usage hints to the kernel for +* better memory management under pressure. +* +* @XE_MADV_PURGEABLE_WILLNEED: The buffer object is needed and should not be purged. +* This is the default state. +* @XE_MADV_PURGEABLE_DONTNEED: The buffer object is not currently needed and can be +* purged by the kernel under memory pressure. +* @XE_MADV_PURGEABLE_PURGED: The buffer object has been purged by the kernel. +* +* Accessing a purged buffer will result in an error. Per i915 semantics, +* once purged, a BO remains permanently invalid and must be destroyed and recreated. +*/ +enum xe_madv_purgeable_state { +XE_MADV_PURGEABLE_WILLNEED , +XE_MADV_PURGEABLE_DONTNEED , +XE_MADV_PURGEABLE_PURGED , +}; +struct sg_table ;struct xe_bo *xe_bo_alloc (void );
@@ -214,6 +236,40 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo)return bo->pxp_key_instance; } +/** +* xe_bo_is_purged() - Check if buffer object has been purged +* @bo: The buffer object to check +* +* Checks if the buffer object's backing store has been discarded by the +* kernel due to memory pressure after being marked as purgeable (DONTNEED). +* Once purged, the BO cannot be restored and any attempt to use it will fail. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO has been purged, false otherwise +*/ +static inline bool xe_bo_is_purged (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ; +} + +/** +* xe_bo_madv_is_dontneed() - Check if BO is marked as DONTNEED +* @bo: The buffer object to check +* +* Checks if userspace has marked this BO as DONTNEED (i.e., its contents +* are not currently needed and can be discarded under memory pressure). +* This is used internally to decide whether a BO is eligible for purging. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO is marked DONTNEED, false otherwise +*/ +static inline bool xe_bo_madv_is_dontneed (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; +} +static inline void xe_bo_unpin_map_no_vm (struct xe_bo *bo) {if (likely (bo)) {
drivers/gpu/drm/xe/xe_bo_types.h +3 -0
@@ -108,6 +108,9 @@ struct xe_bo {* from default */ u64 min_align; + +/** @madv_purgeable: user space advise on BO purgeability */ + u32 madv_purgeable; };#endif
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 2/8] drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo
Quoted 117 lines · Arvind Yadav
Add infrastructure for tracking purgeable state of buffer objects. This includes:
Introduce enum
xe_madv_purgeable_statewith three states:
-XE_MADV_PURGEABLE_WILLNEED(0): BO is needed and should not be purged. This is the default state for all BOs.-
XE_MADV_PURGEABLE_DONTNEED(1): BO is not currently needed and can be purged by the kernel under memory pressure to reclaim resources. Only non-shared BOs can be marked as DONTNEED.-
XE_MADV_PURGEABLE_PURGED(2): BO has been purged by the kernel. Accessing a purged BO results in error. Follows i915 semantics where once purged, the BO remains permanently invalid ("once purged, always purged").Add
atomic_tmadv field to structxe_bofor state tracking
of purgeable state across concurrent access pathsv2:
- Addxe_bo_is_purged()helper, improve state documentationv3:
- Add the kernel doc(Matthew Brost)
- Add the new helpersxe_bo_madv_is_dontneed(Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.h | 56 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_bo_types.h | 3 ++ 2 files changed, 59 insertions(+)drivers/gpu/drm/xe/xe_bo.h +56
-0@@ -86,6 +86,28 @@#define XE_PCI_BARRIER_MMAP_OFFSET (0x50 << XE_PTE_SHIFT) +/** +* enum xe_madv_purgeable_state - Buffer object purgeable state enumeration +* +* This enum defines the possible purgeable states for a buffer object, +* allowing userspace to provide memory usage hints to the kernel for +* better memory management under pressure. +* +* @XE_MADV_PURGEABLE_WILLNEED: The buffer object is needed and should not be purged. +* This is the default state. +* @XE_MADV_PURGEABLE_DONTNEED: The buffer object is not currently needed and can be +* purged by the kernel under memory pressure. +* @XE_MADV_PURGEABLE_PURGED: The buffer object has been purged by the kernel. +* +* Accessing a purged buffer will result in an error. Per i915 semantics, +* once purged, a BO remains permanently invalid and must be destroyed and recreated. +*/ +enum xe_madv_purgeable_state { +XE_MADV_PURGEABLE_WILLNEED , +XE_MADV_PURGEABLE_DONTNEED , +XE_MADV_PURGEABLE_PURGED , +}; +struct sg_table ;struct xe_bo *xe_bo_alloc (void );@@ -214,6 +236,40 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo)return bo->pxp_key_instance; } +/** +* xe_bo_is_purged() - Check if buffer object has been purged +* @bo: The buffer object to check +* +* Checks if the buffer object's backing store has been discarded by the +* kernel due to memory pressure after being marked as purgeable (DONTNEED). +* Once purged, the BO cannot be restored and any attempt to use it will fail. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO has been purged, false otherwise +*/ +static inline bool xe_bo_is_purged (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ; +} + +/** +* xe_bo_madv_is_dontneed() - Check if BO is marked as DONTNEED +* @bo: The buffer object to check +* +* Checks if userspace has marked this BO as DONTNEED (i.e., its contents +* are not currently needed and can be discarded under memory pressure). +* This is used internally to decide whether a BO is eligible for purging. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO is marked DONTNEED, false otherwise +*/ +static inline bool xe_bo_madv_is_dontneed (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; +} +static inline void xe_bo_unpin_map_no_vm (struct xe_bo *bo) {if (likely (bo)) {drivers/gpu/drm/xe/xe_bo_types.h +0
-0
drivers/gpu/drm/xe/xe_bo_types.h struct xe_bo { · line 112
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo_types.h +2
-0@@ -108,6 +108,9 @@ struct xe_bo {* from default */ u64 min_align; + +/** @madv_purgeable: user space advise on BO purgeability */
, protected by BO's dma-resv lock.
Everything else LGTM.
Matt
Quoted 7 lines
drivers/gpu/drm/xe/xe_bo_types.h +1
-0+ u32 madv_purgeable; };#endif Signature · 1 line
2.43.0
Re: [PATCH v4 2/8] drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo
Quoted 125 lines · Matthew Brost
Quoted 117 lines · Arvind Yadav
Add infrastructure for tracking purgeable state of buffer objects. This includes:
Introduce enum
xe_madv_purgeable_statewith three states:
-XE_MADV_PURGEABLE_WILLNEED(0): BO is needed and should not be purged. This is the default state for all BOs.-
XE_MADV_PURGEABLE_DONTNEED(1): BO is not currently needed and can be purged by the kernel under memory pressure to reclaim resources. Only non-shared BOs can be marked as DONTNEED.-
XE_MADV_PURGEABLE_PURGED(2): BO has been purged by the kernel. Accessing a purged BO results in error. Follows i915 semantics where once purged, the BO remains permanently invalid ("once purged, always purged").Add
atomic_tmadv field to structxe_bofor state tracking
of purgeable state across concurrent access pathsv2:
- Addxe_bo_is_purged()helper, improve state documentationv3:
- Add the kernel doc(Matthew Brost)
- Add the new helpersxe_bo_madv_is_dontneed(Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.h | 56 ++++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_bo_types.h | 3 ++ 2 files changed, 59 insertions(+)drivers/gpu/drm/xe/xe_bo.h +56
-0@@ -86,6 +86,28 @@#define XE_PCI_BARRIER_MMAP_OFFSET (0x50 << XE_PTE_SHIFT) +/** +* enum xe_madv_purgeable_state - Buffer object purgeable state enumeration +* +* This enum defines the possible purgeable states for a buffer object, +* allowing userspace to provide memory usage hints to the kernel for +* better memory management under pressure. +* +* @XE_MADV_PURGEABLE_WILLNEED: The buffer object is needed and should not be purged. +* This is the default state. +* @XE_MADV_PURGEABLE_DONTNEED: The buffer object is not currently needed and can be +* purged by the kernel under memory pressure. +* @XE_MADV_PURGEABLE_PURGED: The buffer object has been purged by the kernel. +* +* Accessing a purged buffer will result in an error. Per i915 semantics, +* once purged, a BO remains permanently invalid and must be destroyed and recreated. +*/ +enum xe_madv_purgeable_state { +XE_MADV_PURGEABLE_WILLNEED , +XE_MADV_PURGEABLE_DONTNEED , +XE_MADV_PURGEABLE_PURGED , +}; +struct sg_table ;struct xe_bo *xe_bo_alloc (void );@@ -214,6 +236,40 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo)return bo->pxp_key_instance; } +/** +* xe_bo_is_purged() - Check if buffer object has been purged +* @bo: The buffer object to check +* +* Checks if the buffer object's backing store has been discarded by the +* kernel due to memory pressure after being marked as purgeable (DONTNEED). +* Once purged, the BO cannot be restored and any attempt to use it will fail. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO has been purged, false otherwise +*/ +static inline bool xe_bo_is_purged (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ; +} + +/** +* xe_bo_madv_is_dontneed() - Check if BO is marked as DONTNEED +* @bo: The buffer object to check +* +* Checks if userspace has marked this BO as DONTNEED (i.e., its contents +* are not currently needed and can be discarded under memory pressure). +* This is used internally to decide whether a BO is eligible for purging. +* +* Context: Caller must hold the BO's dma-resv lock +* Return: true if the BO is marked DONTNEED, false otherwise +*/ +static inline bool xe_bo_madv_is_dontneed (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; +} +static inline void xe_bo_unpin_map_no_vm (struct xe_bo *bo) {if (likely (bo)) {drivers/gpu/drm/xe/xe_bo_types.h +0
-0drivers/gpu/drm/xe/xe_bo_types.h struct xe_bo { · line 112
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo_types.h +2
-0@@ -108,6 +108,9 @@ struct xe_bo {* from default */ u64 min_align; + +/** @madv_purgeable: user space advise on BO purgeability */ , protected by BO's dma-resv lock.
Quoted 1 line
Everything else LGTM.
Noted, I will do the changes as per suggestion.
Thanks,
Arvind
Quoted 10 lines
Matt
Quoted 7 lines
drivers/gpu/drm/xe/xe_bo_types.h +1
-0+ u32 madv_purgeable; };#endif Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 2/8] drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo
In reply to Matthew BrostQuoted 147 lines · Matthew Brost
Quoted 137 lines · Arvind Yadav
Add infrastructure for tracking purgeable state of buffer objects. This includes:
Introduce enum
xe_madv_purgeable_statewith three states:
-XE_MADV_PURGEABLE_WILLNEED(0): BO is needed and should not be purged. This is the default state for all BOs.-
XE_MADV_PURGEABLE_DONTNEED(1): BO is not currently needed and can be purged by the kernel under memory pressure to reclaim resources. Only non-shared BOs can be marked as DONTNEED.-
XE_MADV_PURGEABLE_PURGED(2): BO has been purged by the kernel.
Accessing a purged BO results in error. Follows i915 semantics where once purged, the BO remains permanently invalid ("once purged, always purged").Add
atomic_tmadv field to structxe_bofor state tracking of purgeable state across concurrent access pathsv2:
- Addxe_bo_is_purged()helper, improve state documentationv3:
- Add the kernel doc(Matthew Brost)
- Add the new helpersxe_bo_madv_is_dontneed(Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_bo.h | 56
++++++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_bo_types.h | 3 ++2 files changed, 59 insertions(+)drivers/gpu/drm/xe/xe_bo.h +0
-0@@ -86,6 +86,28 @@
#defineXE_PCI_BARRIER_MMAP_OFFSET(0x50<<XE_PTE_SHIFT)
+/**
+ * enumxe_madv_purgeable_state- Buffer object purgeable state enumeration
+ *
+ * This enum defines the possible purgeable states for a buffer object,
+ * allowing userspace to provide memory usage hints to the kernel for
+ * better memory management under pressure.
+ *
+ * @XE_MADV_PURGEABLE_WILLNEED: The buffer object is needed and should not be purged.
+ * This is the default state.
+ * @XE_MADV_PURGEABLE_DONTNEED: The buffer object is not currently needed and can be
+ * purged by the kernel under memory pressure.
+ * @XE_MADV_PURGEABLE_PURGED: The buffer object has been purged by the kernel.
+ *
+ * Accessing a purged buffer will result in an error. Per i915 semantics,
+ * once purged, a BO remains permanently invalid and must be destroyed and recreated.
+ */drivers/gpu/drm/xe/xe_bo.h +6
-0+enum xe_madv_purgeable_state { +XE_MADV_PURGEABLE_WILLNEED , +XE_MADV_PURGEABLE_DONTNEED , +XE_MADV_PURGEABLE_PURGED , +}; +struct
sg_table;
structxe_bo*xe_bo_alloc(void);drivers/gpu/drm/xe/xe_bo.h +0
-0@@ -214,6 +236,40 @@ static inline bool xe_bo_is_protected(const struct xe_bo *bo) return bo->pxp_key_instance; }
+/**
+ *xe_bo_is_purged()- Check if buffer object has been purged
+ * @bo: The buffer object to check
+ *
+ * Checks if the buffer object's backing store has been discarded by the
+ * kernel due to memory pressure after being marked as purgeable (DONTNEED).
+ * Once purged, the BO cannot be restored and any attempt to use it will fail.
+ *
+ * Context: Caller must hold the BO's dma-resv lock
+ * Return: true if the BO has been purged, false otherwise
+ */drivers/gpu/drm/xe/xe_bo.h +11
-0+static inline bool xe_bo_is_purged (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ; +} + +/** +* xe_bo_madv_is_dontneed() - Check if BO is marked as DONTNEED +* @bo: The buffer object to check +* +* Checks if userspace has marked this BO as DONTNEED (i.e., its contents
+ * are not currently needed and can be discarded under memory pressure).
+ * This is used internally to decide whether a BO is eligible for purging.
+ *
+ * Context: Caller must hold the BO's dma-resv lock
+ * Return: true if the BO is marked DONTNEED, false otherwise
+ */drivers/gpu/drm/xe/xe_bo.h +6
-0+static inline bool xe_bo_madv_is_dontneed (struct xe_bo *bo) +{ +xe_bo_assert_held (bo); +return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; +} +static inline void
xe_bo_unpin_map_no_vm(struct xe_bo *bo){
if (likely(bo)) {drivers/gpu/drm/xe/xe_bo_types.h +0
-0drivers/gpu/drm/xe/xe_bo_types.h struct xe_bo { · line 113
Quoted 6 lines
drivers/gpu/drm/xe/xe_bo_types.h +2
-0@@ -108,6 +108,9 @@ struct xe_bo {* from default */ u64 min_align; + +/** @madv_purgeable: user space advise on BO purgeability */ , protected by BO's dma-resv lock.
Quoted 1 line
Everything else LGTM.
Agree, LGTM.
/Thomas
Quoted 10 lines
Matt
Quoted 7 lines
drivers/gpu/drm/xe/xe_bo_types.h +1
-0+u32 madv_purgeable; };
#endifSignature · 1 line
2.43.0
PATCH v4 3/8 drm/xe/madvise: Implement purgeable buffer object support
In reply to Arvind YadavThis allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalid
Design Rationale:
- Async TLB invalidation via trigger_rebind (no blocking xe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault mode
v2:
- Use xe_bo_trigger_rebind() for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Force clear_pt for non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale access
v3:
- Rebase as xe_gt_pagefault.c is gone upstream and replaced with xe_pagefault.c (Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers for madv_purgeable access(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Use xe_bo_assert_held instead of dma assert(Matthew Brost)
- Move the xe_bo_is_purged check under the dma-resv lock( by Matt)
- Drop is_purged from xe_pt_stage_bind_entry and just set is_null to true for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)
v4:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +49 -12
@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +* +* This function purges the backing store of a BO marked as DONTNEED and +* triggers rebind to invalidate stale GPU mappings. For fault-mode VMs, +* this zaps the PTEs. The next GPU access will trigger a page fault and +* perform NULL rebind (scratch pages or clear PTEs based on VM config). +*/ +static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) +{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); + +if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret); +if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =XE_MADV_PURGEABLE_PURGED ; + +/* +* Trigger rebind to invalidate stale GPU mappings. +* - Non-fault mode: Marks VMAs for rebind +* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault +* and NULL rebind with scratch/clear PTEs per VM config +*/ + ret =xe_bo_trigger_rebind (xe, bo, ctx); +XE_WARN_ON (ret); + } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *new_mem,
@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false;int ret =0 ; +/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace. +* The move_notify callback will handle invalidation asynchronously. +*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx); +return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) {if (new_mem->mem_type ==XE_PL_TT )
@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void xe_ttm_bo_swap_notify (struct ttm_buffer_object *ttm_bo) {struct ttm_operation_ctx ctx = {
@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,#endif INIT_LIST_HEAD (&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +drm_gem_private_object_init (&xe->drm, &bo->ttm.base, size);if (resv) {
drivers/gpu/drm/xe/xe_pagefault.c +12 -0
@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma,if (!bo)return 0 ; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs +* - Non-scratch VMs: FAIL the page fault (no scratch page available) +*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return need_vram_move ?xe_bo_migrate (bo, vram->placement,NULL , exec) :xe_bo_validate (bo, vm, true, exec); }
drivers/gpu/drm/xe/xe_pt.c +32 -6
@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,/* Is this a leaf entry ?*/ if (level ==0 ||xe_pt_hugepte_possible (addr, next, level, xe_walk)) {struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); +bool is_null_or_purged =xe_vma_is_null (xe_walk->vma) || + (bo &&xe_bo_is_purged (bo)); +bool is_vram = is_null_or_purged ? false :xe_res_is_vram (curs);XE_WARN_ON (xe_walk->va_curs_start != addr);if (xe_walk->clear_pt) { pte =0 ; }else { - pte = vm->pt_ops->pte_encode_vma (is_null ?0 : +/* +* For purged BOs, treat like null VMAs - pass address 0. +* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping. +*/ + pte = vm->pt_ops->pte_encode_vma (is_null_or_purged ?0 :xe_res_dma (curs) + xe_walk->dma_offset, xe_walk->vma, pat_index, level); -if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk->default_vram_pte : xe_walk->default_system_pte;
@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (unlikely (ret))return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)xe_res_next (curs, next - addr); xe_walk->va_curs_start = next; xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);
@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, };struct xe_pt *pt = vm->pt_root[tile->id];int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads +* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0 +* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely +* zero instead of creating a PRESENT mapping to physical address 0. +*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs +* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }if (range) {/* Move this entire thing to xe_svm.c? */
@@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,if (!range)xe_bo_assert_held (bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (xe_vma_is_userptr (vma))xe_res_first_dma (to_userptr_vma (vma)->userptr.pages.dma_addr,0 ,xe_vma_size (vma), &curs);
drivers/gpu/drm/xe/xe_vm.c +10 -1
@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,static u64 xelp_pte_encode_vma (u64 pte,struct xe_vma *vma,u16 pat_index,u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |=XE_PAGE_PRESENT ;if (likely (!xe_vma_read_only (vma)))
@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, pte |=pte_encode_pat_index (pat_index, pt_level); pte |=pte_encode_ps (pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read). +* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs. +* Never set NULL flag without scratch page - causes undefined behavior. +*/ +if (unlikely (xe_vma_is_null (vma) || + (bo &&xe_bo_is_purged (bo) &&xe_vm_has_scratch (vm)))) pte |=XE_PTE_NULL ;return pte;
drivers/gpu/drm/xe/xe_vm_madvise.c +88 -0
106 lines not on this page · raw message
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
Quoted 69 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_nullto true for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +5
-0@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +*
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 853
Quoted 10 lines
drivers/gpu/drm/xe/xe_bo.c +10
-0+* This function purges the backing store of a BO marked as DONTNEED and +* triggers rebind to invalidate stale GPU mappings. For fault-mode VMs, +* this zaps the PTEs. The next GPU access will trigger a page fault and +* perform NULL rebind (scratch pages or clear PTEs based on VM config). +*/ +static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) +{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); +
xe_bo_assert_held(bo);
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 858
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret);
I think since 'xe' in available here, you should use xe_assert in place of drm_WARN_ON.
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 861
Quoted 3 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =XE_MADV_PURGEABLE_PURGED ;
Helper to set madv_purgeable state /w lockdep assert?
Also perhaps assert valid state transitions in the helper (e.g., you cannot tranistion out of XE_MADV_PURGEABLE_PURGED.
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 870
Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +9
-0+ +/* +* Trigger rebind to invalidate stale GPU mappings. +* - Non-fault mode: Marks VMAs for rebind +* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault +* and NULL rebind with scratch/clear PTEs per VM config +*/ + ret =xe_bo_trigger_rebind (xe, bo, ctx); +XE_WARN_ON (ret);
I think xe_bo_trigger_rebind is allowed to fail if ctx->no_wait_gpu is set. In both the faulting fast path and certain parts of the shrinker we set this. So I think any error returned from xe_bo_trigger_rebind needs to propagte up the call stack.
Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+ } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *new_mem,
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · line 900
Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +6
-0@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false;int ret =0 ; +/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace. +* The move_notify callback will handle invalidation asynchronously. +*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx);
With above, we need to send errors from xe_ttm_bo_purge up the call stack.
Quoted 173 lines
drivers/gpu/drm/xe/xe_bo.c +6
-12+return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) {if (new_mem->mem_type ==XE_PL_TT )@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void xe_ttm_bo_swap_notify (struct ttm_buffer_object *ttm_bo) {struct ttm_operation_ctx ctx = {@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,#endif INIT_LIST_HEAD (&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +drm_gem_private_object_init (&xe->drm, &bo->ttm.base, size);if (resv) {drivers/gpu/drm/xe/xe_pagefault.c +12
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma,if (!bo)return 0 ; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs +* - Non-scratch VMs: FAIL the page fault (no scratch page available) +*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return need_vram_move ?xe_bo_migrate (bo, vram->placement,NULL , exec) :xe_bo_validate (bo, vm, true, exec); }drivers/gpu/drm/xe/xe_pt.c +32
-6@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,/* Is this a leaf entry ?*/ if (level ==0 ||xe_pt_hugepte_possible (addr, next, level, xe_walk)) {struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); +bool is_null_or_purged =xe_vma_is_null (xe_walk->vma) || + (bo &&xe_bo_is_purged (bo)); +bool is_vram = is_null_or_purged ? false :xe_res_is_vram (curs);XE_WARN_ON (xe_walk->va_curs_start != addr);if (xe_walk->clear_pt) { pte =0 ; }else { - pte = vm->pt_ops->pte_encode_vma (is_null ?0 : +/* +* For purged BOs, treat like null VMAs - pass address 0. +* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping. +*/ + pte = vm->pt_ops->pte_encode_vma (is_null_or_purged ?0 :xe_res_dma (curs) + xe_walk->dma_offset, xe_walk->vma, pat_index, level); -if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk->default_vram_pte : xe_walk->default_system_pte;@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (unlikely (ret))return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)xe_res_next (curs, next - addr); xe_walk->va_curs_start = next; xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, };struct xe_pt *pt = vm->pt_root[tile->id];int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads +* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0 +* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely +* zero instead of creating a PRESENT mapping to physical address 0. +*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs +* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }if (range) {/* Move this entire thing to xe_svm.c? */ @@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,if (!range)xe_bo_assert_held (bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (xe_vma_is_userptr (vma))xe_res_first_dma (to_userptr_vma (vma)->userptr.pages.dma_addr,0 ,xe_vma_size (vma), &curs);drivers/gpu/drm/xe/xe_vm.c +10
-1@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,static u64 xelp_pte_encode_vma (u64 pte,struct xe_vma *vma,u16 pat_index,u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |=XE_PAGE_PRESENT ;if (likely (!xe_vma_read_only (vma)))@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, pte |=pte_encode_pat_index (pat_index, pt_level); pte |=pte_encode_ps (pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read). +* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs. +* Never set NULL flag without scratch page - causes undefined behavior. +*/ +if (unlikely (xe_vma_is_null (vma) || + (bo &&xe_bo_is_purged (bo) &&xe_vm_has_scratch (vm)))) pte |=XE_PTE_NULL ;return pte;drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, }
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 189
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0} +/*: +* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. +* Returns true if any BO was purged, false otherwise. +* Caller must copy retained value to userspace after releasing locks. +*/ +static bool xe_vm_madvise_purgeable_bo (struct xe_device *xe,struct xe_vm *vm, +struct xe_vma **vmas,int num_vmas, +struct drm_xe_madvise *op)
Shouldn't this check be a vfunc in madvise_funcs?
Also I think you can hook into xe_madvise_details for the return value / final copy to user.
Quoted 20 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +20
-0+{ +bool has_purged_bo = false; +int i; + +xe_assert (vm->xe, op->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ); + +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */ +xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (xe_bo_is_purged (bo)) {
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 219
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+ has_purged_bo = true; +continue ; + } + +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ;
Use above suggested helper to set this state?
Quoted 33 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +22
-0+break ; +default : +drm_warn (&vm->xe->drm,"Invalid madvice value = %d\n" , + op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */ +return has_purged_bo; +} + typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm *vm, struct xe_vma **vmas, int num_vmas, struct drm_xe_madvise *op,@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvreturn false;break ; } +case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + +if (XE_IOCTL_DBG (xe, !(val ==DRM_XE_VMA_PURGEABLE_STATE_WILLNEED || + val ==DRM_XE_VMA_PURGEABLE_STATE_DONTNEED ))) +return false; + +break ; + }default :if (XE_IOCTL_DBG (xe,1 ))return false;
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 533
Quoted 9 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; } } +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo =xe_vm_madvise_purgeable_bo (xe, vm, madvise_range.vmas, + madvise_range.num_vmas, args); +
Again use the existing vfuncs here.
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); +
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 548
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+/* +* Set retained flag to indicate if backing store still exists. +* Matches i915: retained = 1 if not purged, 0 if purged. +* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency. +*/ +if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + +if (copy_to_user (u64_to_user_ptr (args->purge_state_val.retained), + &retained,sizeof (retained)))
I don't think remained needs to be a u64 - maybe a u16? Will comment on uAPI too.
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 549
Quoted 1 line
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+drm_warn (&vm->xe->drm,"Failed to copy retained value to user\n" );
See above, use xe_madvise_details_fini for the final copy to user.
Matt
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + } }if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
Quoted 415 lines · Matthew Brost
Quoted 69 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_nullto true for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +5
-0@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +* drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 853
Quoted 10 lines
drivers/gpu/drm/xe/xe_bo.c +10
-0+* This function purges the backing store of a BO marked as DONTNEED and +* triggers rebind to invalidate stale GPU mappings. For fault-mode VMs, +* this zaps the PTEs. The next GPU access will trigger a page fault and +* perform NULL rebind (scratch pages or clear PTEs based on VM config). +*/ +static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) +{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); +
xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 858
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret);I think since 'xe' in available here, you should use
xe_assertin place ofdrm_WARN_ON.drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 861
Quoted 3 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =XE_MADV_PURGEABLE_PURGED ;Helper to set
madv_purgeablestate /w lockdep assert?Also perhaps assert valid state transitions in the helper (e.g., you cannot tranistion out of
XE_MADV_PURGEABLE_PURGED.drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 870
Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +9
-0+ +/* +* Trigger rebind to invalidate stale GPU mappings. +* - Non-fault mode: Marks VMAs for rebind +* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault +* and NULL rebind with scratch/clear PTEs per VM config +*/ + ret =xe_bo_trigger_rebind (xe, bo, ctx); +XE_WARN_ON (ret);I think
xe_bo_trigger_rebindis allowed to fail ifctx->no_wait_gpuis set. In both the faulting fast path and certain parts of the shrinker we set this. So I think any error returned fromxe_bo_trigger_rebindneeds to propagte up the call stack.Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+ } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *new_mem,drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · line 900
Quoted 9 lines
drivers/gpu/drm/xe/xe_bo.c +6
-0@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false;int ret =0 ; +/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace. +* The move_notify callback will handle invalidation asynchronously. +*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx);With above, we need to send errors from
xe_ttm_bo_purgeup the call stack.Quoted 173 lines
drivers/gpu/drm/xe/xe_bo.c +6
-12+return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) {if (new_mem->mem_type ==XE_PL_TT )@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void xe_ttm_bo_swap_notify (struct ttm_buffer_object *ttm_bo) {struct ttm_operation_ctx ctx = {@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,#endif INIT_LIST_HEAD (&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +drm_gem_private_object_init (&xe->drm, &bo->ttm.base, size);if (resv) {drivers/gpu/drm/xe/xe_pagefault.c +12
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma,if (!bo)return 0 ; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs +* - Non-scratch VMs: FAIL the page fault (no scratch page available) +*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return need_vram_move ?xe_bo_migrate (bo, vram->placement,NULL , exec) :xe_bo_validate (bo, vm, true, exec); }drivers/gpu/drm/xe/xe_pt.c +32
-6@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,/* Is this a leaf entry ?*/ if (level ==0 ||xe_pt_hugepte_possible (addr, next, level, xe_walk)) {struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); +bool is_null_or_purged =xe_vma_is_null (xe_walk->vma) || + (bo &&xe_bo_is_purged (bo)); +bool is_vram = is_null_or_purged ? false :xe_res_is_vram (curs);XE_WARN_ON (xe_walk->va_curs_start != addr);if (xe_walk->clear_pt) { pte =0 ; }else { - pte = vm->pt_ops->pte_encode_vma (is_null ?0 : +/* +* For purged BOs, treat like null VMAs - pass address 0. +* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping. +*/ + pte = vm->pt_ops->pte_encode_vma (is_null_or_purged ?0 :xe_res_dma (curs) + xe_walk->dma_offset, xe_walk->vma, pat_index, level); -if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk->default_vram_pte : xe_walk->default_system_pte;@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (unlikely (ret))return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)xe_res_next (curs, next - addr); xe_walk->va_curs_start = next; xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, };struct xe_pt *pt = vm->pt_root[tile->id];int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads +* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0 +* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely +* zero instead of creating a PRESENT mapping to physical address 0. +*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs +* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }if (range) {/* Move this entire thing to xe_svm.c? */ @@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,if (!range)xe_bo_assert_held (bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (xe_vma_is_userptr (vma))xe_res_first_dma (to_userptr_vma (vma)->userptr.pages.dma_addr,0 ,xe_vma_size (vma), &curs);drivers/gpu/drm/xe/xe_vm.c +10
-1@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,static u64 xelp_pte_encode_vma (u64 pte,struct xe_vma *vma,u16 pat_index,u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |=XE_PAGE_PRESENT ;if (likely (!xe_vma_read_only (vma)))@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, pte |=pte_encode_pat_index (pat_index, pt_level); pte |=pte_encode_ps (pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read). +* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs. +* Never set NULL flag without scratch page - causes undefined behavior. +*/ +if (unlikely (xe_vma_is_null (vma) || + (bo &&xe_bo_is_purged (bo) &&xe_vm_has_scratch (vm)))) pte |=XE_PTE_NULL ;return pte;drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, }drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 189
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0} +/*: +* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. +* Returns true if any BO was purged, false otherwise. +* Caller must copy retained value to userspace after releasing locks. +*/ +static bool xe_vm_madvise_purgeable_bo (struct xe_device *xe,struct xe_vm *vm, +struct xe_vma **vmas,int num_vmas, +struct drm_xe_madvise *op)Shouldn't this check be a vfunc in
madvise_funcs?Also I think you can hook into
xe_madvise_detailsfor the return value / final copy to user.Quoted 20 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +20
-0+{ +bool has_purged_bo = false; +int i; + +xe_assert (vm->xe, op->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ); + +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */ +xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (xe_bo_is_purged (bo)) {drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 219
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+ has_purged_bo = true; +continue ; + } + +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ;Use above suggested helper to set this state?
Quoted 33 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +22
-0+break ; +default : +drm_warn (&vm->xe->drm,"Invalid madvice value = %d\n" , + op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */ +return has_purged_bo; +} + typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm *vm, struct xe_vma **vmas, int num_vmas, struct drm_xe_madvise *op,@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvreturn false;break ; } +case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + +if (XE_IOCTL_DBG (xe, !(val ==DRM_XE_VMA_PURGEABLE_STATE_WILLNEED || + val ==DRM_XE_VMA_PURGEABLE_STATE_DONTNEED ))) +return false; + +break ; + }default :if (XE_IOCTL_DBG (xe,1 ))return false;drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 533
Quoted 9 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; } } +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo =xe_vm_madvise_purgeable_bo (xe, vm, madvise_range.vmas, + madvise_range.num_vmas, args); +Again use the existing vfuncs here.
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); + +/* +* Set retained flag to indicate if backing store still exists. +* Matches i915: retained = 1 if not purged, 0 if purged. +* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency. +*/
Quoted 9 lines
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 548
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + +if (copy_to_user (u64_to_user_ptr (args->purge_state_val.retained), + &retained,sizeof (retained)))I don't think remained needs to be a u64 - maybe a u16? Will comment on uAPI too.
Ignore this, I forgot purge_state_val.retained is a userptr so u64 is correct. Let me follow on if we are allowed to change IOCTLs from IOW -> IOWR. I am really unclear on the rules that part of the uAPI.
Quoted 18 lines
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 549
Quoted 1 line
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+drm_warn (&vm->xe->drm,"Failed to copy retained value to user\n" );See above, use
xe_madvise_details_finifor the final copy to user.Matt
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + } }if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
Re: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
Quoted 66 lines · Matthew Brost
Quoted 65 lines · Matthew Brost
Quoted 64 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_nullto true for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; }
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c +8
-0+/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +* +* This function purges the backing store of a BO marked as DONTNEED and +* triggers rebind to invalidate stale GPU mappings. For fault-mode VMs, +* this zaps the PTEs. The next GPU access will trigger a page fault and Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 853
Quoted 7 lines
drivers/gpu/drm/xe/xe_bo.c +7
-0+* perform NULL rebind (scratch pages or clear PTEs based on VM config). +*/ +static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) +{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); +
xe_bo_assert_held(bo);
Noted, I will add xe_bo_assert_held() at the top of the purge helper
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 858
Quoted 6 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret);I think since 'xe' in available here, you should use
xe_assertin place ofdrm_WARN_ON.
Agreed. Switched this to xe_assert(xe, !ret) since this is Xe-specific and we already have xe.
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 861
Quoted 4 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =XE_MADV_PURGEABLE_PURGED ;Helper to set
madv_purgeablestate /w lockdep assert?Also perhaps assert valid state transitions in the helper (e.g., you cannot tranistion out of
XE_MADV_PURGEABLE_PURGED.
Noted, I will xe_bo_set_purgeable_state() which asserts the BO lock and enforces "once PURGED always PURGED" (reject transitions out of PURGED).
Quoted 14 lines
Quoted 14 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 870
Quoted 10 lines
drivers/gpu/drm/xe/xe_bo.c +9
-0+ +/* +* Trigger rebind to invalidate stale GPU mappings. +* - Non-fault mode: Marks VMAs for rebind +* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault +* and NULL rebind with scratch/clear PTEs per VM config +*/ + ret =xe_bo_trigger_rebind (xe, bo, ctx); +XE_WARN_ON (ret);I think
xe_bo_trigger_rebindis allowed to fail ifctx->no_wait_gpuis set. In both the faulting fast path and certain parts of the shrinker we set this. So I think any error returned fromxe_bo_trigger_rebindneeds to propagte up the call stack.
Agreed. I will changed xe_ttm_bo_purge() to return int and propagate errors. xe_bo_move() now returns the error when purge/rebind fails, rather than warning and continuing.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+ } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *new_mem,@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false;int ret =0 ;
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · line 900
Quoted 6 lines
drivers/gpu/drm/xe/xe_bo.c +6
-0+/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace. +* The move_notify callback will handle invalidation asynchronously. +*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx);With above, we need to send errors from
xe_ttm_bo_purgeup the call stack.
Noted.
Quoted 186 lines
Quoted 186 lines
Quoted 174 lines
drivers/gpu/drm/xe/xe_bo.c +6
-12+return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) {if (new_mem->mem_type ==XE_PL_TT )@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void xe_ttm_bo_swap_notify (struct ttm_buffer_object *ttm_bo) {struct ttm_operation_ctx ctx = {@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,#endif INIT_LIST_HEAD (&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +drm_gem_private_object_init (&xe->drm, &bo->ttm.base, size);if (resv) {drivers/gpu/drm/xe/xe_pagefault.c +12
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma,if (!bo)return 0 ; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs +* - Non-scratch VMs: FAIL the page fault (no scratch page available) +*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return need_vram_move ?xe_bo_migrate (bo, vram->placement,NULL , exec) :xe_bo_validate (bo, vm, true, exec); }drivers/gpu/drm/xe/xe_pt.c +32
-6@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,/* Is this a leaf entry ?*/ if (level ==0 ||xe_pt_hugepte_possible (addr, next, level, xe_walk)) {struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); +bool is_null_or_purged =xe_vma_is_null (xe_walk->vma) || + (bo &&xe_bo_is_purged (bo)); +bool is_vram = is_null_or_purged ? false :xe_res_is_vram (curs);XE_WARN_ON (xe_walk->va_curs_start != addr);if (xe_walk->clear_pt) { pte =0 ; }else { - pte = vm->pt_ops->pte_encode_vma (is_null ?0 : +/* +* For purged BOs, treat like null VMAs - pass address 0. +* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping. +*/ + pte = vm->pt_ops->pte_encode_vma (is_null_or_purged ?0 :xe_res_dma (curs) + xe_walk->dma_offset, xe_walk->vma, pat_index, level); -if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk->default_vram_pte : xe_walk->default_system_pte;@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (unlikely (ret))return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)xe_res_next (curs, next - addr); xe_walk->va_curs_start = next; xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, };struct xe_pt *pt = vm->pt_root[tile->id];int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads +* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0 +* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely +* zero instead of creating a PRESENT mapping to physical address 0. +*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs +* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }if (range) {/* Move this entire thing to xe_svm.c? */ @@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,if (!range)xe_bo_assert_held (bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (xe_vma_is_userptr (vma))xe_res_first_dma (to_userptr_vma (vma)->userptr.pages.dma_addr,0 ,xe_vma_size (vma), &curs);drivers/gpu/drm/xe/xe_vm.c +10
-1@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,static u64 xelp_pte_encode_vma (u64 pte,struct xe_vma *vma,u16 pat_index,u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |=XE_PAGE_PRESENT ;if (likely (!xe_vma_read_only (vma)))@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, pte |=pte_encode_pat_index (pat_index, pt_level); pte |=pte_encode_ps (pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read). +* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs. +* Never set NULL flag without scratch page - causes undefined behavior. +*/ +if (unlikely (xe_vma_is_null (vma) || + (bo &&xe_bo_is_purged (bo) &&xe_vm_has_scratch (vm)))) pte |=XE_PTE_NULL ;return pte;drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, }drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 189
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0} +/*: +* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. +* Returns true if any BO was purged, false otherwise. +* Caller must copy retained value to userspace after releasing locks. +*/ +static bool xe_vm_madvise_purgeable_bo (struct xe_device *xe,struct xe_vm *vm, +struct xe_vma **vmas,int num_vmas, +struct drm_xe_madvise *op)Shouldn't this check be a vfunc in
madvise_funcs?
Quoted 2 lines
Quoted 2 lines
Also I think you can hook into
xe_madvise_detailsfor the return value / final copy to user.
Yes. I will move purgeable handling into the madvise_funcs[] table as a proper madvise vfunc (madvise_purgeable). The retained return is now tracked via xe_madvise_details and copied back in xe_madvise_details_fini(), so we keep the "copy_to_user after dropping locks" rule without a special-case early return.
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +23
-0+{ +bool has_purged_bo = false; +int i; + +xe_assert (vm->xe, op->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ); + +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */ +xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (xe_bo_is_purged (bo)) { + has_purged_bo = true; +continue ; + }
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 219
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +7
-0+ +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ;Use above suggested helper to set this state?
Yes, converted to xe_bo_set_purgeable_state().
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +22
-0+break ; +default : +drm_warn (&vm->xe->drm,"Invalid madvice value = %d\n" , + op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */ +return has_purged_bo; +} + typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm *vm, struct xe_vma **vmas, int num_vmas, struct drm_xe_madvise *op,@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvreturn false;break ; } +case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + +if (XE_IOCTL_DBG (xe, !(val ==DRM_XE_VMA_PURGEABLE_STATE_WILLNEED || + val ==DRM_XE_VMA_PURGEABLE_STATE_DONTNEED ))) +return false; + +break ; + }default :if (XE_IOCTL_DBG (xe,1 ))return false;@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; }
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 533
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0} +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo =xe_vm_madvise_purgeable_bo (xe, vm, madvise_range.vmas, + madvise_range.num_vmas, args); +Again use the existing vfuncs here.
Noted,
Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +9
-0+/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); + +/* +* Set retained flag to indicate if backing store still exists. +* Matches i915: retained = 1 if not purged, 0 if purged. +* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency. Quoted 9 lines
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 548
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+*/ +if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + +if (copy_to_user (u64_to_user_ptr (args->purge_state_val.retained), + &retained,sizeof (retained)))I don't think remained needs to be a u64 - maybe a u16? Will comment on uAPI too.
Quoted 3 lines
Ignore this, I forgot purge_state_val.retained is a userptr so u64 is correct. Let me follow on if we are allowed to change IOCTLs from IOW -> IOWR. I am really unclear on the rules that part of the uAPI.
sure,
Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/xe/xe_vm_madvise.c int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · line 549
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+drm_warn (&vm->xe->drm,"Failed to copy retained value to user\n" );See above, use
xe_madvise_details_finifor the final copy to user.
Noted - The retained pointer is stored in details during init, and the final copy happens in xe_madvise_details_fini().
Thanks,
Arvind
Quoted 15 lines
Quoted 15 lines
Matt
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + } }if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
In reply to Matthew BrostQuoted 140 lines · Matthew Brost
Quoted 97 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory
reclamation of user-designated DONTNEED buffers during eviction.This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_nullto true
for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++-drivers/gpu/drm/xe/xe_vm_madvise.c| 88
++++++++++++++++++++++++++++++5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo, return 0; }
+/**
+ *xe_ttm_bo_purge()- Purge buffer object backing store
+ * @ttm_bo: The TTM buffer object to purge
+ * @ctx: TTM operation context
+ *
+ * This function purges the backing store of a BO marked as DONTNEED and
+ * triggers rebind to invalidate stale GPU mappings. For fault- mode VMs,
+ * this zaps the PTEs. The next GPU access will trigger a page fault and
+ * perform NULL rebind (scratch pages or clear PTEs based on VM config).
+ */
+static voidxe_ttm_bo_purge(struct ttm_buffer_object *ttm_bo, struct ttm_operation_ctx *ctx)drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 853
Quoted 4 lines
drivers/gpu/drm/xe/xe_bo.c +4
-0+{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); +
xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 858
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret);I think since 'xe' in available here, you should use
xe_assertin place
ofdrm_WARN_ON.Quoted 4 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =
XE_MADV_PURGEABLE_PURGED;Helper to set
madv_purgeablestate /w lockdep assert?Also perhaps assert valid state transitions in the helper (e.g., you cannot tranistion out of
XE_MADV_PURGEABLE_PURGED.Quoted 13 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+ +/* +* Trigger rebind to invalidate stale GPU mappings.
drivers/gpu/drm/xe/xe_bo.c +1
-0+* - Non-fault mode: Marks VMAs for rebind
drivers/gpu/drm/xe/xe_bo.c +1
-0+* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault
drivers/gpu/drm/xe/xe_bo.c +1
-0+ * and NULL rebind withscratch/clear PTEs per VM config
drivers/gpu/drm/xe/xe_bo.c +2
-0+*/ + ret = xe_bo_trigger_rebind(xe, bo,ctx);
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 870
Quoted 1 line
drivers/gpu/drm/xe/xe_bo.c +1
-0+XE_WARN_ON (ret);
Quoted 7 lines
I think
xe_bo_trigger_rebindis allowed to fail ifctx->no_wait_gpuis
set. In both the faulting fast path and certain parts of the shrinker we
set this. So I think any error returned fromxe_bo_trigger_rebindneeds
to propagte up the call stack.
If possible, I think we should call xe_bo_move_notify(), which will in turn call xe_bo_trigger_rebind() rather than call xe_bo_trigger_rebind(), since xe_bo_move_notify() is intended to unbind / unmap everything needed before a bo move / purge. In this case xe_bo_trigger_rebind() may be sufficient, but perhaps not in the future.
Quoted 387 lines
Quoted 21 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+ } + } + } +} +static int xe_bo_move(struct
ttm_buffer_object*ttm_bo, bool evict,
structttm_operation_ctx*ctx, structttm_resource*new_mem,drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false; int ret = 0;drivers/gpu/drm/xe/xe_bo.c +2
-0+/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace.
drivers/gpu/drm/xe/xe_bo.c +1
-0+* The move_notify callback will handle invalidation asynchronously.
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · line 900
Quoted 3 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx);With above, we need to send errors from
xe_ttm_bo_purgeup the call stack.Quoted 232 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) { if (
new_mem->mem_type==XE_PL_TT)drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } }
-static voidxe_ttm_bo_purge(struct ttm_buffer_object *ttm_bo, struct ttm_operation_ctx *ctx)drivers/gpu/drm/xe/xe_bo.c +0
-11-{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void
xe_ttm_bo_swap_notify(struct ttm_buffer_object *ttm_bo)
{
structttm_operation_ctxctx = {drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, #endif INIT_LIST_HEAD(&bo->vram_userfault_link);drivers/gpu/drm/xe/xe_bo.c +3
-0+/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +
drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size);
if (resv) {drivers/gpu/drm/xe/xe_pagefault.c +0
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma, if (!bo) return 0;drivers/gpu/drm/xe/xe_pagefault.c +4
-0+/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs
drivers/gpu/drm/xe/xe_pagefault.c +1
-0+* - Non-scratch VMs: FAIL the page fault (no scratch page available)
drivers/gpu/drm/xe/xe_pagefault.c +7
-0+*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return
need_vram_move?xe_bo_migrate(bo, vram->placement, NULL, exec):
xe_bo_validate(bo, vm, true, exec);
}drivers/gpu/drm/xe/xe_pt.c +2
-2@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,pgoff_t offset, /* Is this a leaf entry ?*/ if (level == 0 || xe_pt_hugepte_possible(addr, next, level, xe_walk)) { struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); + bool is_null_or_purged = xe_vma_is_null(xe_walk-Quoted 1 line
vma) ||
drivers/gpu/drm/xe/xe_pt.c +1
-0+ (bo &&
xe_bo_is_purged(bo));drivers/gpu/drm/xe/xe_pt.c +1
-0+ bool is_vram = is_null_or_purged ? false :
xe_res_is_vram(curs);
XE_WARN_ON(xe_walk->va_curs_start != addr);
if (xe_walk->clear_pt) {
pte = 0;
} else {drivers/gpu/drm/xe/xe_pt.c +0
-1- pte = vm->pt_ops->pte_encode_vma(is_null ?0 :
drivers/gpu/drm/xe/xe_pt.c +2
-0+/* +* For purged BOs, treat like null VMAs - pass address 0.
drivers/gpu/drm/xe/xe_pt.c +1
-0+* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping.
drivers/gpu/drm/xe/xe_pt.c +2
-0+*/ + pte = vm->pt_ops-Quoted 1 line
pte_encode_vma(is_null_or_purged ? 0 :
xe_res_dma(curs)+
xe_walk-Quoted 1 line
dma_offset,xe_walk-
Quoted 1 line
vma,
pat_index, level);drivers/gpu/drm/xe/xe_pt.c +1
-1-if (!is_null) +if (!is_null_or_purged)pte |=
is_vram? xe_walk-Quoted 1 line
default_vram_pte:xe_walk-
Quoted 1 line
default_system_pte;drivers/gpu/drm/xe/xe_pt.c +0
-0@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,pgoff_t offset, if (unlikely(ret)) return ret;drivers/gpu/drm/xe/xe_pt.c +1
-1-if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)
xe_res_next(curs, next - addr);xe_walk->va_curs_start= next;xe_walk->vma->gpuva.flags|= (XE_VMA_PTE_4K<< level);drivers/gpu/drm/xe/xe_pt.c +5
-0@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, structxe_vma *vma, }; struct xe_pt *pt = vm->pt_root[tile->id]; int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads
drivers/gpu/drm/xe/xe_pt.c +1
-0+* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0
drivers/gpu/drm/xe/xe_pt.c +2
-0+* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely
drivers/gpu/drm/xe/xe_pt.c +1
-0+* zero instead of creating a PRESENT mapping to physical address 0.
drivers/gpu/drm/xe/xe_pt.c +6
-0+*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs
drivers/gpu/drm/xe/xe_pt.c +5
-0+* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }
if (range) {
/* Move this entire thing toxe_svm.c? */drivers/gpu/drm/xe/xe_pt.c +0
-0@@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, if (!range) xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_pt.c +1
-1-if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (
xe_vma_is_userptr(vma)) xe_res_first_dma(to_userptr_vma(vma)-Quoted 1 line
userptr.pages.dma_addr, 0,
xe_vma_size(vma), &curs);drivers/gpu/drm/xe/xe_vm.c +3
-0@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset, static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, u16 pat_index, u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |= XE_PAGE_PRESENT;
if (likely(!xe_vma_read_only(vma)))drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte,struct xe_vma *vma, pte |= pte_encode_pat_index(pat_index, pt_level); pte |= pte_encode_ps(pt_level);drivers/gpu/drm/xe/xe_vm.c +2
-1-if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read).
drivers/gpu/drm/xe/xe_vm.c +1
-0+* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs.
drivers/gpu/drm/xe/xe_vm.c +1
-0+* Never set NULL flag without scratch page - causes undefined behavior.
drivers/gpu/drm/xe/xe_vm.c +3
-0+*/ + if (unlikely(xe_vma_is_null(vma) || + (bo && xe_bo_is_purged(bo) &&
xe_vm_has_scratch(vm))))
pte |=XE_PTE_NULL;
return pte;drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } }
+/*:
+ * Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED.
+ * Returns true if any BO was purged, false otherwise.
+ * Caller must copy retained value to userspace after releasing locks.
+ */
+static bool xe_vm_madvise_purgeable_bo(structxe_device*xe, structxe_vm*vm,drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ struct xe_vma **vmas, int
num_vmas,drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 189
Quoted 1 line
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ struct drm_xe_madvise *op)Shouldn't this check be a vfunc in
madvise_funcs?Also I think you can hook into
xe_madvise_detailsfor the return value /
final copy to user.Quoted 35 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+{ +bool has_purged_bo = false; +int i; + + xe_assert(vm->xe, op->type ==
DRM_XE_VMA_ATTR_PURGEABLE_STATE);drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0+ +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */
drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* This matches i915 semantics where purged BOs are permanently invalid.
drivers/gpu/drm/xe/xe_vm_madvise.c +9
-0+*/ +if (xe_bo_is_purged (bo)) { + has_purged_bo = true; +continue ; + } + +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =
XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =Quoted 1 line
XE_MADV_PURGEABLE_DONTNEED;Use above suggested helper to set this state?
Quoted 50 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+break ; +default : + drm_warn(&vm->xe->drm, "Invalid madvicevalue = %d\n",
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+return has_purged_bo; +} +typedef void (*madvise_func)(struct
xe_device*xe, structxe_vm*vm,
structxe_vma**vmas, intnum_vmas, structdrm_xe_madvise*op,drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct
xe_device*xe, const structdrm_xe_madv
return false;
break;
}drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + + if (XE_IOCTL_DBG(xe, !(val ==
DRM_XE_VMA_PURGEABLE_STATE_WILLNEED||drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ val ==
DRM_XE_VMA_PURGEABLE_STATE_DONTNEED)))drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+return false; + +break ; + }default:
if (XE_IOCTL_DBG(xe, 1))
return false;drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; } } +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo = xe_vm_madvise_purgeable_bo(xe, vm, madvise_range.vmas, +Quoted 2 lines
madvise_range.num_vmas, args);
+Again use the existing vfuncs here.
Quoted 19 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +7
-0+/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); + +/* +* Set retained flag to indicate if backing store still exists.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* Matches i915: retained = 1 if not purged, 0 if purged.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency.
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+*/ +if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + + if(copy_to_user(u64_to_user_ptr(args->purge_state_val.retained),
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ &retained,Quoted 1 line
sizeof(retained)))
I don't think remained needs to be a u64 - maybe a u16? Will comment on
uAPI too.
Quoted 4 lines
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ drm_warn(&vm->xe->drm,"Failed to copy retained value to user\n");
See above, use
xe_madvise_details_finifor the final copy to user.
Can we use put_user() rather than copy_to_user() ?
Also, should the IOCTL return a failure in this case?
Another option is ofc to assert that retained is set to false on IOCTL call, so that if put_user() fails, UMD will not try to reuse a bo whose retained state is unclear.
Thanks,
Thomas
Quoted 15 lines
Matt
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + }}
if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
Re: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
Quoted 144 lines · Thomas Hellström
Quoted 135 lines · Matthew Brost
Quoted 97 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory
reclamation of user-designated DONTNEED buffers during eviction.This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged"
semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault
modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas
Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas
Hellström)
- Implement i915-compatible retained field logic (Thomas
Hellström)
- Skip BO validation for purged BOs in page fault handler (crash
fix)
- Add scratch VM check in page fault path (non-scratch VMs fail
fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping
(review fix)
- Add !is_purged check to resource cursor setup to prevent stale
accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by
Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_null
to true
for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)v4:
- @madv_purgeableatomic_t→ u32 change across all relevant
patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +6
-0@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo, return 0; } +/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +* +* This function purges the backing store of a BO marked as DONTNEED and
+ * triggers rebind to invalidate stale GPU mappings. For fault- mode VMs,
+ * this zaps the PTEs. The next GPU access will trigger a page fault and
+ * perform NULL rebind (scratch pages or clear PTEs based on VM config).
+ */
+static voidxe_ttm_bo_purge(struct ttm_buffer_object *ttm_bo, struct ttm_operation_ctx *ctx)drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 853
Quoted 4 lines
drivers/gpu/drm/xe/xe_bo.c +4
-0+{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); +
xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 858
Quoted 5 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret);I think since 'xe' in available here, you should use
xe_assertin place
ofdrm_WARN_ON.Quoted 4 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =
XE_MADV_PURGEABLE_PURGED;Helper to set
madv_purgeablestate /w lockdep assert?Also perhaps assert valid state transitions in the helper (e.g., you cannot tranistion out of
XE_MADV_PURGEABLE_PURGED.Quoted 13 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+ +/* +* Trigger rebind to invalidate stale GPU mappings.
drivers/gpu/drm/xe/xe_bo.c +1
-0+* - Non-fault mode: Marks VMAs for rebind
drivers/gpu/drm/xe/xe_bo.c +1
-0+* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault
drivers/gpu/drm/xe/xe_bo.c +1
-0+ * and NULL rebind withscratch/clear PTEs per VM config
drivers/gpu/drm/xe/xe_bo.c +2
-0+*/ + ret = xe_bo_trigger_rebind(xe, bo,ctx);
Quoted 8 lines
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move_notify(struct xe_bo *bo, · line 870
Quoted 1 line
drivers/gpu/drm/xe/xe_bo.c +1
-0+XE_WARN_ON (ret);I think
xe_bo_trigger_rebindis allowed to fail ifctx->no_wait_gpuis
set. In both the faulting fast path and certain parts of the shrinker we
set this. So I think any error returned fromxe_bo_trigger_rebindneeds
to propagte up the call stack.
Quoted 6 lines
If possible, I think we should call
xe_bo_move_notify(), which will in turn callxe_bo_trigger_rebind()rather than callxe_bo_trigger_rebind(), sincexe_bo_move_notify()is intended to unbind / unmap everything needed before a bo move / purge. In this casexe_bo_trigger_rebind()may be sufficient, but perhaps not in the future.
Agree with both points. Will do the change.
Quoted 381 lines
Quoted 381 lines
Quoted 20 lines
drivers/gpu/drm/xe/xe_bo.c +5
-0+ } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, boolevict,
structttm_operation_ctx*ctx, structttm_resource*new_mem,drivers/gpu/drm/xe/xe_bo.c +2
-0@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false; int ret = 0; +/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace.
drivers/gpu/drm/xe/xe_bo.c +1
-0+* The move_notify callback will handle invalidation asynchronously.
drivers/gpu/drm/xe/xe_bo.c static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, · line 900
Quoted 3 lines
drivers/gpu/drm/xe/xe_bo.c +3
-0+*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx);With above, we need to send errors from
xe_ttm_bo_purgeup the call stack.Quoted 232 lines
drivers/gpu/drm/xe/xe_bo.c +3
-12+return 0 ; + } + /* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) { if (new_mem->mem_type == XE_PL_TT)@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo, struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} - static void xe_ttm_bo_swap_notify(struct ttm_buffer_object
*ttm_bo)
{
structttm_operation_ctxctx = {drivers/gpu/drm/xe/xe_bo.c +3
-0@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo, #endif INIT_LIST_HEAD(&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; + drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size);if (resv) {
drivers/gpu/drm/xe/xe_pagefault.c +4
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma, if (!bo) return 0; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs
drivers/gpu/drm/xe/xe_pagefault.c +1
-0+* - Non-scratch VMs: FAIL the page fault (no scratch page available)
drivers/gpu/drm/xe/xe_pagefault.c +7
-0+*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } + return need_vram_move ? xe_bo_migrate(bo, vram->placement,NULL, exec) :
xe_bo_validate(bo, vm, true, exec); }drivers/gpu/drm/xe/xe_pt.c +2
-2@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,pgoff_t offset, /* Is this a leaf entry ?*/ if (level == 0 || xe_pt_hugepte_possible(addr, next, level, xe_walk)) { struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); + bool is_null_or_purged = xe_vma_is_null(xe_walk-Quoted 1 line
vma) ||
drivers/gpu/drm/xe/xe_pt.c +1
-0+ (bo &&
xe_bo_is_purged(bo));drivers/gpu/drm/xe/xe_pt.c +1
-0+ bool is_vram = is_null_or_purged ? false :
xe_res_is_vram(curs);
XE_WARN_ON(xe_walk->va_curs_start != addr);if (
xe_walk->clear_pt) {
pte = 0;
} else {drivers/gpu/drm/xe/xe_pt.c +0
-1- pte = vm->pt_ops->pte_encode_vma(is_null ?0 :
drivers/gpu/drm/xe/xe_pt.c +2
-0+/* +* For purged BOs, treat like null VMAs - pass address 0.
drivers/gpu/drm/xe/xe_pt.c +1
-0+* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping.
drivers/gpu/drm/xe/xe_pt.c +2
-0+*/ + pte = vm->pt_ops-Quoted 1 line
pte_encode_vma(is_null_or_purged ? 0 :
xe_res_dma(curs)+
xe_walk-Quoted 1 line
dma_offset,xe_walk-
Quoted 1 line
vma,
pat_index, level);drivers/gpu/drm/xe/xe_pt.c +1
-1-if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk-Quoted 1 line
default_vram_pte:xe_walk-
Quoted 1 line
default_system_pte;drivers/gpu/drm/xe/xe_pt.c +1
-1@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent,pgoff_t offset, if (unlikely(ret)) return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt) xe_res_next(curs, next - addr); xe_walk->va_curs_start = next;
xe_walk->vma->gpuva.flags|= (XE_VMA_PTE_4K<<
level);drivers/gpu/drm/xe/xe_pt.c +5
-0@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, structxe_vma *vma, }; struct xe_pt *pt = vm->pt_root[tile->id]; int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads
drivers/gpu/drm/xe/xe_pt.c +1
-0+* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0
drivers/gpu/drm/xe/xe_pt.c +2
-0+* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely
drivers/gpu/drm/xe/xe_pt.c +1
-0+* zero instead of creating a PRESENT mapping to physical address 0.
drivers/gpu/drm/xe/xe_pt.c +6
-0+*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs
drivers/gpu/drm/xe/xe_pt.c +6
-1+* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + } if (range) { /* Move this entire thing to xe_svm.c? */@@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, if (!range) xe_bo_assert_held(bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) { if (xe_vma_is_userptr(vma)) xe_res_first_dma(to_userptr_vma(vma)-Quoted 1 line
userptr.pages.dma_addr, 0,
xe_vma_size(vma), &curs);drivers/gpu/drm/xe/xe_vm.c +3
-0@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset, static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, u16 pat_index, u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |= XE_PAGE_PRESENT;if (likely(!xe_vma_read_only(vma)))
drivers/gpu/drm/xe/xe_vm.c +2
-1@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte,struct xe_vma *vma, pte |= pte_encode_pat_index(pat_index, pt_level); pte |= pte_encode_ps(pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read).
drivers/gpu/drm/xe/xe_vm.c +1
-0+* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs.
drivers/gpu/drm/xe/xe_vm.c +1
-0+* Never set NULL flag without scratch page - causes undefined behavior.
drivers/gpu/drm/xe/xe_vm.c +3
-0+*/ + if (unlikely(xe_vma_is_null(vma) || + (bo && xe_bo_is_purged(bo) &&
xe_vm_has_scratch(vm))))
pte |=XE_PTE_NULL;return pte;
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/*: +* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED.
+ * Returns true if any BO was purged, false otherwise.
+ * Caller must copy retained value to userspace after releasing locks.
+ */
+static bool xe_vm_madvise_purgeable_bo(structxe_device*xe, structxe_vm*vm,drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ struct xe_vma **vmas, int
num_vmas,drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 189
Quoted 1 line
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ struct drm_xe_madvise *op)Shouldn't this check be a vfunc in
madvise_funcs?Also I think you can hook into
xe_madvise_detailsfor the return value /
final copy to user.Quoted 35 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+{ +bool has_purged_bo = false; +int i; + + xe_assert(vm->xe, op->type ==
DRM_XE_VMA_ATTR_PURGEABLE_STATE);drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0+ +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */
drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* This matches i915 semantics where purged BOs are permanently invalid.
drivers/gpu/drm/xe/xe_vm_madvise.c +9
-0+*/ +if (xe_bo_is_purged (bo)) { + has_purged_bo = true; +continue ; + } + +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =
XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =Quoted 1 line
XE_MADV_PURGEABLE_DONTNEED;Use above suggested helper to set this state?
Quoted 50 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+break ; +default : + drm_warn(&vm->xe->drm, "Invalid madvicevalue = %d\n",
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+return has_purged_bo; +} + typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm*vm,
structxe_vma**vmas, intnum_vmas, structdrm_xe_madvise*op,drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct
xe_device*xe, const structdrm_xe_madv
return false;
break;
}drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + + if (XE_IOCTL_DBG(xe, !(val ==
DRM_XE_VMA_PURGEABLE_STATE_WILLNEED||drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ val ==
DRM_XE_VMA_PURGEABLE_STATE_DONTNEED)))drivers/gpu/drm/xe/xe_vm_madvise.c +9
-0+return false; + +break ; + } default: if (XE_IOCTL_DBG(xe, 1)) return false;@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; } } +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo = xe_vm_madvise_purgeable_bo(xe, vm, madvise_range.vmas, +Quoted 2 lines
madvise_range.num_vmas, args);
+Again use the existing vfuncs here.
Quoted 19 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +7
-0+/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); + +/* +* Set retained flag to indicate if backing store still exists.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* Matches i915: retained = 1 if not purged, 0 if purged.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency.
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+*/ +if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + + if(copy_to_user(u64_to_user_ptr(args->purge_state_val.retained),
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ &retained,Quoted 1 line
sizeof(retained)))
I don't think remained needs to be a u64 - maybe a u16? Will comment on
uAPI too.
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ drm_warn(&vm->xe->drm,"Failed to copy retained value to user\n");
See above, use
xe_madvise_details_finifor the final copy to user.Can we use
put_user()rather thancopy_to_user()?
Agreed, Will change to put_user().
Quoted 2 lines
Also, should the IOCTL return a failure in this case?
yes. If we can't communicate the retained state to userspace, the IOCTL should fail. Will return -EFAULT.
Quoted 4 lines
Another option is ofc to assert that retained is set to false on IOCTL call, so that if
put_user()fails, UMD will not try to reuse a bo whose retained state is unclear.
Agreed, I will add a check that retained == 0 on IOCTL entry and reject the call otherwise. Combined with returning -EFAULT on put_user() failure, this guarantees userspace never observes or relies on an uncertain retained state.
Thanks,
~Arvind
Quoted 20 lines
Thanks,
ThomasQuoted 14 lines
Matt
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+ } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + } } if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 3/8] drm/xe/madvise: Implement purgeable buffer object support
In reply to Arvind YadavQuoted 41 lines · Arvind Yadav
This allows userspace applications to provide memory usage hints to the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from
xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalidDesign Rationale:
- Async TLB invalidation viatrigger_rebind(no blockingxe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged" semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault modev2:
- Usexe_bo_trigger_rebind()for async TLB invalidation (Thomas Hellström)
- Add NULL rebind with scratch PTEs for fault mode (Thomas Hellström)
- Implement i915-compatible retained field logic (Thomas Hellström)
- Skip BO validation for purged BOs in page fault handler (crash fix)
- Add scratch VM check in page fault path (non-scratch VMs fail fault)
- Forceclear_ptfor non-scratch VMs to avoid phys addr 0 mapping (review fix)
- Add !is_purged check to resource cursor setup to prevent stale accessv3:
- Rebase asxe_gt_pagefault.cis gone upstream and replaced withxe_pagefault.c(Matthew Brost)
- Xe specific warn on (Matthew Brost)
- Call helpers formadv_purgeableaccess(Matthew Brost)
- Remove bo NULL check(Matthew Brost)
- Usexe_bo_assert_heldinstead of dma assert(Matthew Brost)
- Move thexe_bo_is_purgedcheck under the dma-resv lock( by Matt)
- Dropis_purgedfromxe_pt_stage_bind_entryand just setis_nullto true for purged BO rename s/is_null/is_null_or_purged (by Matt)
- UAPI rule should not be changed.(Matthew Brost)
- Make 'retained' a userptr (Matthew Brost)
Quoted 4 lines
v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>
One last nit here - it is fine you want to implement parts of the IOCTL eariler in the series to make this eaiser to review but please don't flip on functionality of the IOCTL until all parts are in place so we can't biscet the tree and get half of the IOCTLs functionality.
Matt
Quoted 352 lines
Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 61 +++++++++++++++++---- drivers/gpu/drm/xe/xe_pagefault.c | 12 ++++ drivers/gpu/drm/xe/xe_pt.c | 38 +++++++++++-- drivers/gpu/drm/xe/xe_vm.c | 11 +++- drivers/gpu/drm/xe/xe_vm_madvise.c | 88 ++++++++++++++++++++++++++++++ 5 files changed, 191 insertions(+), 19 deletions(-)drivers/gpu/drm/xe/xe_bo.c +49
-12@@ -836,6 +836,43 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_ttm_bo_purge() - Purge buffer object backing store +* @ttm_bo: The TTM buffer object to purge +* @ctx: TTM operation context +* +* This function purges the backing store of a BO marked as DONTNEED and +* triggers rebind to invalidate stale GPU mappings. For fault-mode VMs, +* this zaps the PTEs. The next GPU access will trigger a page fault and +* perform NULL rebind (scratch pages or clear PTEs based on VM config). +*/ +static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) +{ +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_bo *bo =ttm_to_xe_bo (ttm_bo); + +if (ttm_bo->ttm) { +struct ttm_placement place = {}; +int ret =ttm_bo_validate (ttm_bo, &place, ctx); + +drm_WARN_ON (&xe->drm, ret); +if (!ret) { +if (xe_bo_madv_is_dontneed (bo)) { + bo->madv_purgeable =XE_MADV_PURGEABLE_PURGED ; + +/* +* Trigger rebind to invalidate stale GPU mappings. +* - Non-fault mode: Marks VMAs for rebind +* - Fault mode: Zaps PTEs (sets to 0), next access triggers fault +* and NULL rebind with scratch/clear PTEs per VM config +*/ + ret =xe_bo_trigger_rebind (xe, bo, ctx); +XE_WARN_ON (ret); + } + } + } +} + static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, struct ttm_operation_ctx *ctx, struct ttm_resource *new_mem,@@ -855,6 +892,15 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict, ttm && ttm_tt_is_populated(ttm)) ? true : false;int ret =0 ; +/* +* Purge only non-shared BOs explicitly marked DONTNEED by userspace. +* The move_notify callback will handle invalidation asynchronously. +*/ +if (evict &&xe_bo_madv_is_dontneed (bo)) { +xe_ttm_bo_purge (ttm_bo, ctx); +return 0 ; + } +/* Bo creation path, moving to system or TT. */ if ((!old_mem && ttm) && !handle_system_ccs) {if (new_mem->mem_type ==XE_PL_TT )@@ -1604,18 +1650,6 @@ static void xe_ttm_bo_delete_mem_notify(struct ttm_buffer_object *ttm_bo) } } -static void xe_ttm_bo_purge (struct ttm_buffer_object *ttm_bo,struct ttm_operation_ctx *ctx) -{ -struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); - -if (ttm_bo->ttm) { -struct ttm_placement place = {}; -int ret =ttm_bo_validate (ttm_bo, &place, ctx); - -drm_WARN_ON (&xe->drm, ret); - } -} -static void xe_ttm_bo_swap_notify (struct ttm_buffer_object *ttm_bo) {struct ttm_operation_ctx ctx = {@@ -2196,6 +2230,9 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,#endif INIT_LIST_HEAD (&bo->vram_userfault_link); +/* Initialize purge advisory state */ + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +drm_gem_private_object_init (&xe->drm, &bo->ttm.base, size);if (resv) {drivers/gpu/drm/xe/xe_pagefault.c +12
-0@@ -59,6 +59,18 @@ static int xe_pagefault_begin(struct drm_exec *exec, struct xe_vma *vma,if (!bo)return 0 ; +/* +* Check if BO is purged (under dma-resv lock). +* For purged BOs: +* - Scratch VMs: Skip validation, rebind will use scratch PTEs +* - Non-scratch VMs: FAIL the page fault (no scratch page available) +*/ +if (unlikely (xe_bo_is_purged (bo))) { +if (!xe_vm_has_scratch (vm)) +return -EACCES ; +return 0 ; + } +return need_vram_move ?xe_bo_migrate (bo, vram->placement,NULL , exec) :xe_bo_validate (bo, vm, true, exec); }drivers/gpu/drm/xe/xe_pt.c +32
-6@@ -533,20 +533,26 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,/* Is this a leaf entry ?*/ if (level ==0 ||xe_pt_hugepte_possible (addr, next, level, xe_walk)) {struct xe_res_cursor *curs = xe_walk->curs; -bool is_null =xe_vma_is_null (xe_walk->vma); -bool is_vram = is_null ? false :xe_res_is_vram (curs); +struct xe_bo *bo =xe_vma_bo (xe_walk->vma); +bool is_null_or_purged =xe_vma_is_null (xe_walk->vma) || + (bo &&xe_bo_is_purged (bo)); +bool is_vram = is_null_or_purged ? false :xe_res_is_vram (curs);XE_WARN_ON (xe_walk->va_curs_start != addr);if (xe_walk->clear_pt) { pte =0 ; }else { - pte = vm->pt_ops->pte_encode_vma (is_null ?0 : +/* +* For purged BOs, treat like null VMAs - pass address 0. +* The pte_encode_vma will set XE_PTE_NULL flag for scratch mapping. +*/ + pte = vm->pt_ops->pte_encode_vma (is_null_or_purged ?0 :xe_res_dma (curs) + xe_walk->dma_offset, xe_walk->vma, pat_index, level); -if (!is_null) +if (!is_null_or_purged) pte |= is_vram ? xe_walk->default_vram_pte : xe_walk->default_system_pte;@@ -570,7 +576,7 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (unlikely (ret))return ret; -if (!is_null && !xe_walk->clear_pt) +if (!is_null_or_purged && !xe_walk->clear_pt)xe_res_next (curs, next - addr); xe_walk->va_curs_start = next; xe_walk->vma->gpuva.flags |= (XE_VMA_PTE_4K << level);@@ -723,6 +729,26 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma, };struct xe_pt *pt = vm->pt_root[tile->id];int ret; +bool is_purged = false; + +/* +* Check if BO is purged: +* - Scratch VMs: Use scratch PTEs (XE_PTE_NULL) for safe zero reads +* - Non-scratch VMs: Clear PTEs to zero (non-present) to avoid mapping to phys addr 0 +* +* For non-scratch VMs, we force clear_pt=true so leaf PTEs become completely +* zero instead of creating a PRESENT mapping to physical address 0. +*/ +if (bo &&xe_bo_is_purged (bo)) { + is_purged = true; + +/* +* For non-scratch VMs, a NULL rebind should use zero PTEs +* (non-present), not a present PTE to phys 0. +*/ +if (!xe_vm_has_scratch (vm)) + xe_walk.clear_pt = true; + }if (range) {/* Move this entire thing to xe_svm.c? */ @@ -762,7 +788,7 @@ xe_pt_stage_bind(struct xe_tile *tile, struct xe_vma *vma,if (!range)xe_bo_assert_held (bo); -if (!xe_vma_is_null (vma) && !range) { +if (!xe_vma_is_null (vma) && !range && !is_purged) {if (xe_vma_is_userptr (vma))xe_res_first_dma (to_userptr_vma (vma)->userptr.pages.dma_addr,0 ,xe_vma_size (vma), &curs);drivers/gpu/drm/xe/xe_vm.c +10
-1@@ -1359,6 +1359,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,static u64 xelp_pte_encode_vma (u64 pte,struct xe_vma *vma,u16 pat_index,u32 pt_level) { +struct xe_bo *bo =xe_vma_bo (vma); +struct xe_vm *vm =xe_vma_vm (vma); + pte |=XE_PAGE_PRESENT ;if (likely (!xe_vma_read_only (vma)))@@ -1367,7 +1370,13 @@ static u64 xelp_pte_encode_vma(u64 pte, struct xe_vma *vma, pte |=pte_encode_pat_index (pat_index, pt_level); pte |=pte_encode_ps (pt_level); -if (unlikely (xe_vma_is_null (vma))) +/* +* NULL PTEs redirect to scratch page (return zeros on read). +* Set for: 1) explicit null VMAs, 2) purged BOs on scratch VMs. +* Never set NULL flag without scratch page - causes undefined behavior. +*/ +if (unlikely (xe_vma_is_null (vma) || + (bo &&xe_bo_is_purged (bo) &&xe_vm_has_scratch (vm)))) pte |=XE_PTE_NULL ;return pte;drivers/gpu/drm/xe/xe_vm_madvise.c +88
-0@@ -179,6 +179,56 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/* +* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. +* Returns true if any BO was purged, false otherwise. +* Caller must copy retained value to userspace after releasing locks. +*/ +static bool xe_vm_madvise_purgeable_bo (struct xe_device *xe,struct xe_vm *vm, +struct xe_vma **vmas,int num_vmas, +struct drm_xe_madvise *op) +{ +bool has_purged_bo = false; +int i; + +xe_assert (vm->xe, op->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ); + +for (i =0 ; i < num_vmas; i++) { +struct xe_bo *bo =xe_vma_bo (vmas[i]); + +if (!bo) +continue ; + +/* BO must be locked before modifying madv state */ +xe_bo_assert_held (bo); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (xe_bo_is_purged (bo)) { + has_purged_bo = true; +continue ; + } + +switch (op->purge_state_val.val) { +case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +break ; +case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; +break ; +default : +drm_warn (&vm->xe->drm,"Invalid madvice value = %d\n" , + op->purge_state_val.val); +return false; + } + } + +/* Return whether any BO was purged; caller will copy to user after unlocking */ +return has_purged_bo; +} + typedef void (*madvise_func)(struct xe_device *xe, struct xe_vm *vm, struct xe_vma **vmas, int num_vmas, struct drm_xe_madvise *op,@@ -306,6 +356,16 @@ static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvreturn false;break ; } +case DRM_XE_VMA_ATTR_PURGEABLE_STATE : + { +u32 val = args->purge_state_val.val; + +if (XE_IOCTL_DBG (xe, !(val ==DRM_XE_VMA_PURGEABLE_STATE_WILLNEED || + val ==DRM_XE_VMA_PURGEABLE_STATE_DONTNEED ))) +return false; + +break ; + }default :if (XE_IOCTL_DBG (xe,1 ))return false;@@ -465,6 +525,34 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil goto err_fini; } } +if (args->type ==DRM_XE_VMA_ATTR_PURGEABLE_STATE ) { +bool has_purged_bo; + + has_purged_bo =xe_vm_madvise_purgeable_bo (xe, vm, madvise_range.vmas, + madvise_range.num_vmas, args); + +/* Release BO locks */ +drm_exec_fini (&exec); +kfree (madvise_range.vmas); +up_write (&vm->lock); + +/* +* Set retained flag to indicate if backing store still exists. +* Matches i915: retained = 1 if not purged, 0 if purged. +* Must copy_to_user AFTER releasing ALL locks to avoid circular dependency. +*/ +if (args->purge_state_val.retained) { +u32 retained = !has_purged_bo; + +if (copy_to_user (u64_to_user_ptr (args->purge_state_val.retained), + &retained,sizeof (retained))) +drm_warn (&vm->xe->drm,"Failed to copy retained value to user\n" ); + } + +/* Final cleanup for early return */ +xe_vm_put (vm); +return 0 ; + } }if (madvise_range.has_svm_userptr_vmas) {Signature · 1 line
2.43.0
PATCH v4 4/8 drm/xe/bo: Handle CPU faults on purged buffer objects
In reply to Arvind YadavReturn error when CPU attempts to access a purged buffer object. Purged BOs have their backing store reclaimed by the kernel, making CPU access invalid. The fault handler returns SIGBUS to userspace, matching i915 semantics.
The purged check is added to both CPU fault paths:
- Fastpath (xe_bo_cpu_fault_fastpath): Returns error immediately under dma-resv lock, preventing attempts to migrate/validate freed pages
- Slowpath (xe_bo_cpu_fault): Returns -EFAULT under drm_exec lock, converted to VM_FAULT_SIGBUS
Without the fastpath check, accessing existing mmap mappings of purged BOs would trigger xe_bo_fault_migrate() on freed backing store, causing kernel hangs or crashes.
v2:
- Added xe_bo_is_purged(bo) instead of atomic_read.
- Avoids leaks and keeps drm_dev_exit() while returning.
v3:
- Move xe_bo_is_purged check under a dma-resv lock (Matthew Brost)
v4:
- Add purged check to fastpath (xe_bo_cpu_fault_fastpath) to prevent hang when accessing existing mmap of purged BO
--- drivers/gpu/drm/xe/xe_bo.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
drivers/gpu/drm/xe/xe_bo.c +12 -0
@@ -1934,6 +1934,12 @@ static vm_fault_t xe_bo_cpu_fault_fastpath(struct vm_fault *vmf, struct xe_devicif (!dma_resv_trylock (tbo->base.resv)) goto out_validation; +/* Purged BOs have no backing store - fault to userspace */ +if (xe_bo_is_purged (bo)) { + ret =VM_FAULT_SIGBUS ; + goto out_unlock; + } +if (xe_ttm_bo_is_imported (tbo)) { ret =VM_FAULT_SIGBUS ;drm_dbg (&xe->drm,"CPU trying to access an imported buffer object.\n" );
@@ -2024,6 +2030,12 @@ static vm_fault_t xe_bo_cpu_fault(struct vm_fault *vmf)if (err)break ; +/* Purged BOs have no backing store - fault to userspace */ +if (xe_bo_is_purged (bo)) { + err = -EFAULT ; +break ; + } +if (xe_ttm_bo_is_imported (tbo)) { err = -EFAULT ;drm_dbg (&xe->drm,"CPU trying to access an imported buffer object.\n" );
Signature · 1 line
2.43.0
Re: [PATCH v4 4/8] drm/xe/bo: Handle CPU faults on purged buffer objects
Quoted 19 lines · Arvind Yadav
Return error when CPU attempts to access a purged buffer object. Purged BOs have their backing store reclaimed by the kernel, making CPU access invalid. The fault handler returns SIGBUS to userspace, matching i915 semantics.
The purged check is added to both CPU fault paths:
- Fastpath (xe_bo_cpu_fault_fastpath): Returns error immediately under dma-resv lock, preventing attempts to migrate/validate freed pages
- Slowpath (xe_bo_cpu_fault): Returns-EFAULTunderdrm_execlock, converted toVM_FAULT_SIGBUSWithout the fastpath check, accessing existing mmap mappings of purged BOs would trigger
xe_bo_fault_migrate()on freed backing store, causing kernel hangs or crashes.v2:
- Addedxe_bo_is_purged(bo)instead ofatomic_read.
- Avoids leaks and keepsdrm_dev_exit()while returning.
Quoted 8 lines
v3:
- Movexe_bo_is_purgedcheck under a dma-resv lock (Matthew Brost)v4:
- Add purged check to fastpath (xe_bo_cpu_fault_fastpath) to prevent hang when accessing existing mmap of purged BOCc Matthew Brost <[email protected]>
Quoted 40 lines
Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)drivers/gpu/drm/xe/xe_bo.c +12
-0@@ -1934,6 +1934,12 @@ static vm_fault_t xe_bo_cpu_fault_fastpath(struct vm_fault *vmf, struct xe_devicif (!dma_resv_trylock (tbo->base.resv)) goto out_validation; +/* Purged BOs have no backing store - fault to userspace */ +if (xe_bo_is_purged (bo)) { + ret =VM_FAULT_SIGBUS ; + goto out_unlock; + } +if (xe_ttm_bo_is_imported (tbo)) { ret =VM_FAULT_SIGBUS ;drm_dbg (&xe->drm,"CPU trying to access an imported buffer object.\n" );@@ -2024,6 +2030,12 @@ static vm_fault_t xe_bo_cpu_fault(struct vm_fault *vmf)if (err)break ; +/* Purged BOs have no backing store - fault to userspace */ +if (xe_bo_is_purged (bo)) { + err = -EFAULT ; +break ; + } +if (xe_ttm_bo_is_imported (tbo)) { err = -EFAULT ;drm_dbg (&xe->drm,"CPU trying to access an imported buffer object.\n" );Signature · 1 line
2.43.0
Re: [PATCH v4 4/8] drm/xe/bo: Handle CPU faults on purged buffer objects
In reply to Arvind YadavQuoted 30 lines · Arvind Yadav
Return error when CPU attempts to access a purged buffer object. Purged BOs have their backing store reclaimed by the kernel, making CPU access invalid. The fault handler returns SIGBUS to userspace, matching i915 semantics.
The purged check is added to both CPU fault paths:
- Fastpath (xe_bo_cpu_fault_fastpath): Returns error immediately under dma-resv lock, preventing attempts to migrate/validate freed pages
- Slowpath (xe_bo_cpu_fault): Returns-EFAULTunderdrm_execlock, converted toVM_FAULT_SIGBUSWithout the fastpath check, accessing existing mmap mappings of purged BOs
would triggerxe_bo_fault_migrate()on freed backing store, causing kernel
hangs or crashes.v2:
- Addedxe_bo_is_purged(bo)instead ofatomic_read.
- Avoids leaks and keepsdrm_dev_exit()while returning.v3:
- Movexe_bo_is_purgedcheck under a dma-resv lock (Matthew Brost)v4:
- Add purged check to fastpath (xe_bo_cpu_fault_fastpath) to prevent
hang when accessing existing mmap of purged BO
Quoted 4 lines
Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>
Quoted 39 lines
---
drivers/gpu/drm/xe/xe_bo.c | 12 ++++++++++++1 file changed, 12 insertions(+)drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -1934,6 +1934,12 @@ static vm_fault_txe_bo_cpu_fault_fastpath(struct
vm_fault*vmf, structxe_devicif (!dma_resv_trylock(tbo->base.resv))
gotoout_validation;drivers/gpu/drm/xe/xe_bo.c +6
-0+/* Purged BOs have no backing store - fault to userspace */ +if (xe_bo_is_purged (bo)) { + ret =VM_FAULT_SIGBUS ; + goto out_unlock; + } +if (
xe_ttm_bo_is_imported(tbo)) {
ret =VM_FAULT_SIGBUS;
drm_dbg(&xe->drm, "CPU trying to access an imported buffer object.\n");drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -2024,6 +2030,12 @@ static vm_fault_t xe_bo_cpu_fault(struct vm_fault *vmf) if (err) break;drivers/gpu/drm/xe/xe_bo.c +1
-0+/* Purged BOs have no backing store - fault to userspace */
drivers/gpu/drm/xe/xe_bo.c +5
-0+if (xe_bo_is_purged (bo)) { + err = -EFAULT ; +break ; + } +if (
xe_ttm_bo_is_imported(tbo)) {
err =-EFAULT;drm_dbg(&xe->drm, "CPU trying to access an imported buffer object.\n");
PATCH v4 5/8 drm/xe/vm: Prevent binding of purged buffer objects
In reply to Arvind YadavAdd check_purged parameter to vma_lock_and_validate() to block new mapping operations on purged BOs while allowing cleanup operations to proceed.
Purged BOs have their backing pages freed by the kernel. New mapping operations (MAP, PREFETCH, REMAP) must be rejected with -EINVAL to prevent GPU access to invalid memory. Cleanup operations (UNMAP) must be allowed so applications can release resources after detecting purge via the retained field.
REMAP operations require mixed handling - reject new prev/next VMAs if the BO is purged, but allow the unmap portion to proceed for cleanup.
The check_purged parameter distinguishes between these cases:
true for new mappings (must reject), false for cleanup (allow).
v2:
- Clarify that purged BOs are permanently invalid (i915 semantics)
- Remove incorrect claim about madvise(WILLNEED) restoring purged BOs
v3:
- Move xe_bo_is_purged check under vma_lock_and_validate (Matthew Brost)
- Add check_purged parameter to distinguish new mappings from cleanup
- Allow UNMAP operations to prevent resource leaks
- Handle REMAP operation's dual nature (cleanup + new mappings)
--- drivers/gpu/drm/xe/xe_vm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +13 -7
@@ -2883,7 +2883,7 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, }static int vma_lock_and_validate (struct drm_exec *exec,struct xe_vma *vma, -bool res_evict,bool validate) +bool res_evict,bool validate,bool check_purged) {struct xe_bo *bo =xe_vma_bo (vma);struct xe_vm *vm =xe_vma_vm (vma);
@@ -2892,6 +2892,11 @@ static int vma_lock_and_validate(struct drm_exec *exec, struct xe_vma *vma,if (bo) {if (!bo->vm) err =drm_exec_lock_obj (exec, &bo->ttm.base); + +/* Reject new mappings to purged BOs; allow cleanup operations */ +if (!err && check_purged &&xe_bo_is_purged (bo)) + err = -EINVAL ; +if (!err && validate) err = xe_bo_validate(bo, vm, !xe_vm_in_preempt_fence_mode(vm) &&
@@ -2990,7 +2995,8 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec, op->map.vma, res_evict, !xe_vm_in_fault_mode (vm) || - op->map.immediate); + op->map.immediate, + true);break ;case DRM_GPUVA_OP_REMAP : err =check_ufence (gpuva_to_vma (op->base.remap.unmap->va));
@@ -2999,13 +3005,13 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.remap.unmap->va), - res_evict, false); + res_evict, false, false);if (!err && op->remap.prev) err =vma_lock_and_validate (exec, op->remap.prev, - res_evict, true); + res_evict, true, true);if (!err && op->remap.next) err =vma_lock_and_validate (exec, op->remap.next, - res_evict, true); + res_evict, true, true);break ;case DRM_GPUVA_OP_UNMAP : err =check_ufence (gpuva_to_vma (op->base.unmap.va));
@@ -3014,7 +3020,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.unmap.va), - res_evict, false); + res_evict, false, false);break ;case DRM_GPUVA_OP_PREFETCH : {
@@ -3029,7 +3035,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.prefetch.va), - res_evict, false); + res_evict, false, true);if (!err && !xe_vma_has_no_bo (vma)) err = xe_bo_migrate(xe_vma_bo(vma), region_to_mem_type[region],
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 5/8] drm/xe/vm: Prevent binding of purged buffer objects
Quoted 40 lines · Arvind Yadav
Add
check_purgedparameter tovma_lock_and_validate()to block new mapping operations on purged BOs while allowing cleanup operations to proceed.Purged BOs have their backing pages freed by the kernel. New mapping operations (MAP, PREFETCH, REMAP) must be rejected with
-EINVALto prevent GPU access to invalid memory. Cleanup operations (UNMAP) must be allowed so applications can release resources after detecting purge via the retained field.REMAP operations require mixed handling - reject new prev/next VMAs if the BO is purged, but allow the unmap portion to proceed for cleanup.
The
check_purgedparameter distinguishes between these cases:
true for new mappings (must reject), false for cleanup (allow).v2:
- Clarify that purged BOs are permanently invalid (i915 semantics)
- Remove incorrect claim about madvise(WILLNEED) restoring purged BOsv3:
- Movexe_bo_is_purgedcheck undervma_lock_and_validate(Matthew Brost)
- Addcheck_purgedparameter to distinguish new mappings from cleanup
- Allow UNMAP operations to prevent resource leaks
- Handle REMAP operation's dual nature (cleanup + new mappings)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_vm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)drivers/gpu/drm/xe/xe_vm.c +0
-0
drivers/gpu/drm/xe/xe_vm.c static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, · line 2886
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -2883,7 +2883,7 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, }static int vma_lock_and_validate (struct drm_exec *exec,struct xe_vma *vma, -bool res_evict,bool validate) +bool res_evict,bool validate,bool check_purged)
It probably time to add something like this to avoid transposing arguments.
struct lock_and_validate_flags {
bool res_evict;
bool validate;
bool check_purged;
};Logic in the patch looks correct though.
Matt
Quoted 63 lines
drivers/gpu/drm/xe/xe_vm.c +12
-6{struct xe_bo *bo =xe_vma_bo (vma);struct xe_vm *vm =xe_vma_vm (vma);@@ -2892,6 +2892,11 @@ static int vma_lock_and_validate(struct drm_exec *exec, struct xe_vma *vma,if (bo) {if (!bo->vm) err =drm_exec_lock_obj (exec, &bo->ttm.base); + +/* Reject new mappings to purged BOs; allow cleanup operations */ +if (!err && check_purged &&xe_bo_is_purged (bo)) + err = -EINVAL ; +if (!err && validate) err = xe_bo_validate(bo, vm, !xe_vm_in_preempt_fence_mode(vm) &&@@ -2990,7 +2995,8 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec, op->map.vma, res_evict, !xe_vm_in_fault_mode (vm) || - op->map.immediate); + op->map.immediate, + true);break ;case DRM_GPUVA_OP_REMAP : err =check_ufence (gpuva_to_vma (op->base.remap.unmap->va));@@ -2999,13 +3005,13 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.remap.unmap->va), - res_evict, false); + res_evict, false, false);if (!err && op->remap.prev) err =vma_lock_and_validate (exec, op->remap.prev, - res_evict, true); + res_evict, true, true);if (!err && op->remap.next) err =vma_lock_and_validate (exec, op->remap.next, - res_evict, true); + res_evict, true, true);break ;case DRM_GPUVA_OP_UNMAP : err =check_ufence (gpuva_to_vma (op->base.unmap.va));@@ -3014,7 +3020,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.unmap.va), - res_evict, false); + res_evict, false, false);break ;case DRM_GPUVA_OP_PREFETCH : {@@ -3029,7 +3035,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.prefetch.va), - res_evict, false); + res_evict, false, true);if (!err && !xe_vma_has_no_bo (vma)) err = xe_bo_migrate(xe_vma_bo(vma), region_to_mem_type[region],Signature · 1 line
2.43.0
Re: [PATCH v4 5/8] drm/xe/vm: Prevent binding of purged buffer objects
Quoted 48 lines · Matthew Brost
Quoted 40 lines · Arvind Yadav
Add
check_purgedparameter tovma_lock_and_validate()to block new mapping operations on purged BOs while allowing cleanup operations to proceed.Purged BOs have their backing pages freed by the kernel. New mapping operations (MAP, PREFETCH, REMAP) must be rejected with
-EINVALto prevent GPU access to invalid memory. Cleanup operations (UNMAP) must be allowed so applications can release resources after detecting purge via the retained field.REMAP operations require mixed handling - reject new prev/next VMAs if the BO is purged, but allow the unmap portion to proceed for cleanup.
The
check_purgedparameter distinguishes between these cases:
true for new mappings (must reject), false for cleanup (allow).v2:
- Clarify that purged BOs are permanently invalid (i915 semantics)
- Remove incorrect claim about madvise(WILLNEED) restoring purged BOsv3:
- Movexe_bo_is_purgedcheck undervma_lock_and_validate(Matthew Brost)
- Addcheck_purgedparameter to distinguish new mappings from cleanup
- Allow UNMAP operations to prevent resource leaks
- Handle REMAP operation's dual nature (cleanup + new mappings)Cc Matthew Brost<[email protected]>Cc Thomas Hellström<[email protected]>Cc Himal Prasad Ghimiray<[email protected]>Signed-off-by Arvind Yadav<[email protected]>--- drivers/gpu/drm/xe/xe_vm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)drivers/gpu/drm/xe/xe_vm.c +0
-0drivers/gpu/drm/xe/xe_vm.c static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, · line 2886
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -2883,7 +2883,7 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, }static int vma_lock_and_validate (struct drm_exec *exec,struct xe_vma *vma, -bool res_evict,bool validate) +bool res_evict,bool validate,bool check_purged)It probably time to add something like this to avoid transposing arguments.
Quoted 7 lines
struct lock_and_validate_flags { bool res_evict; bool validate; bool check_purged; };Logic in the patch looks correct though.
Noted, I will add "struct xe_lock_and_validate_flags" Thanks, Arvind
Quoted 66 lines
Matt
Quoted 63 lines
drivers/gpu/drm/xe/xe_vm.c +12
-6{struct xe_bo *bo =xe_vma_bo (vma);struct xe_vm *vm =xe_vma_vm (vma);@@ -2892,6 +2892,11 @@ static int vma_lock_and_validate(struct drm_exec *exec, struct xe_vma *vma,if (bo) {if (!bo->vm) err =drm_exec_lock_obj (exec, &bo->ttm.base); + +/* Reject new mappings to purged BOs; allow cleanup operations */ +if (!err && check_purged &&xe_bo_is_purged (bo)) + err = -EINVAL ; +if (!err && validate) err = xe_bo_validate(bo, vm, !xe_vm_in_preempt_fence_mode(vm) &&@@ -2990,7 +2995,8 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec, op->map.vma, res_evict, !xe_vm_in_fault_mode (vm) || - op->map.immediate); + op->map.immediate, + true);break ;case DRM_GPUVA_OP_REMAP : err =check_ufence (gpuva_to_vma (op->base.remap.unmap->va));@@ -2999,13 +3005,13 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.remap.unmap->va), - res_evict, false); + res_evict, false, false);if (!err && op->remap.prev) err =vma_lock_and_validate (exec, op->remap.prev, - res_evict, true); + res_evict, true, true);if (!err && op->remap.next) err =vma_lock_and_validate (exec, op->remap.next, - res_evict, true); + res_evict, true, true);break ;case DRM_GPUVA_OP_UNMAP : err =check_ufence (gpuva_to_vma (op->base.unmap.va));@@ -3014,7 +3020,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.unmap.va), - res_evict, false); + res_evict, false, false);break ;case DRM_GPUVA_OP_PREFETCH : {@@ -3029,7 +3035,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock_and_validate (exec,gpuva_to_vma (op->base.prefetch.va), - res_evict, false); + res_evict, false, true);if (!err && !xe_vma_has_no_bo (vma)) err = xe_bo_migrate(xe_vma_bo(vma), region_to_mem_type[region],Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 5/8] drm/xe/vm: Prevent binding of purged buffer objects
Quoted 66 lines · Arvind
Quoted 64 lines · Matthew Brost
Quoted 53 lines · Arvind Yadav
Add
check_purgedparameter tovma_lock_and_validate()to block new mapping operations on purged BOs while allowing cleanup operations to proceed.Purged BOs have their backing pages freed by the kernel. New mapping operations (MAP, PREFETCH, REMAP) must be rejected with
-EINVALto prevent GPU access to invalid memory. Cleanup operations (UNMAP) must be allowed so applications can release resources after detecting purge via the retained field.REMAP operations require mixed handling - reject new prev/next VMAs if the BO is purged, but allow the unmap portion to proceed for cleanup.
The
check_purgedparameter distinguishes between these cases:
true for new mappings (must reject), false for cleanup (allow).v2:
- Clarify that purged BOs are permanently invalid (i915 semantics)
- Remove incorrect claim about madvise(WILLNEED) restoring purged BOsv3:
- Movexe_bo_is_purgedcheck undervma_lock_and_validate(Matthew Brost)
- Addcheck_purgedparameter to distinguish new mappings from cleanup
- Allow UNMAP operations to prevent resource leaks
- Handle REMAP operation's dual nature (cleanup + new mappings)Cc Matthew Brost<[email protected]>Cc Thomas Hellström<[email protected]>Cc Himal Prasad Ghimiray<[email protected]>Signed-off-by Arvind Yadav<[email protected]>---
drivers/gpu/drm/xe/xe_vm.c| 20 +++++++++++++-------1 file changed, 13 insertions(+), 7 deletions(-)drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -2883,7 +2883,7 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, }static int vma_lock_and_validate(struct
drm_exec*exec, structxe_vma*vma,drivers/gpu/drm/xe/xe_vm.c +1
-1- bool res_evict, bool validate) + bool res_evict, bool validate,Quoted 1 line
bool
check_purged)It probably time to add something like this to avoid transposing arguments.
struct lock_and_validate_flags { bool res_evict; bool validate; bool check_purged; };
Quoted 4 lines
Quoted 1 line
Logic in the patch looks correct though.
Noted, I will add "struct
xe_lock_and_validate_flags" Thanks, Arvind
Note that if you follow the pattern of struct xe_bo_shink_flags, passing the struct as a const and using
struct lock_and_validate_flags {
u32 res_evict : 1;
u32 validate : 1;
u32 check_purged : 1;
};This will be more or less equivalent to passing a bit-field with type- checking.
Some reviewers frown on using "bool" in compound types although we accept that in the xe driver.
Otherwise patch LGTM as well.
/Thomas
Quoted 93 lines
Quoted 93 lines
Matt
Quoted 89 lines
{
structxe_bo*bo =xe_vma_bo(vma);
structxe_vm*vm =xe_vma_vm(vma);drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -2892,6 +2892,11 @@ static int vma_lock_and_validate(struct drm_exec *exec, struct xe_vma *vma, if (bo) { if (!bo->vm)err = drm_exec_lock_obj(exec, &bo-
Quoted 1 line
ttm.base);
drivers/gpu/drm/xe/xe_vm.c +2
-0+ +/* Reject new mappings to purged BOs; allow cleanup operations */
drivers/gpu/drm/xe/xe_vm.c +3
-0+if (!err && check_purged &&xe_bo_is_purged (bo)) + err = -EINVAL ; +if (!err && validate)
err = xe_bo_validate(bo, vm,
!xe_vm_in_preempt_fence_mode(vm) &&drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -2990,7 +2995,8 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,err = vma_lock_and_validate(exec, op-
Quoted 1 line
map.vma,
res_evict,
!xe_vm_in_fault_mode(vm) ||drivers/gpu/drm/xe/xe_vm.c +0
-1- op-Quoted 1 line
map.immediate);
drivers/gpu/drm/xe/xe_vm.c +1
-0+ op-Quoted 1 line
map.immediate,
drivers/gpu/drm/xe/xe_vm.c +1
-0+ true);break;
caseDRM_GPUVA_OP_REMAP:
err = check_ufence(gpuva_to_vma(op-Quoted 1 line
base.remap.unmap->va));drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -2999,13 +3005,13 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
err = vma_lock_and_validate(exec, gpuva_to_vma(op-Quoted 1 line
base.remap.unmap->va),drivers/gpu/drm/xe/xe_vm.c +1
-1- res_evict, false); + res_evict, false,false);
if (!err &&op->remap.prev)
err = vma_lock_and_validate(exec, op-Quoted 1 line
remap.prev,
drivers/gpu/drm/xe/xe_vm.c +0
-1- res_evict,true);
drivers/gpu/drm/xe/xe_vm.c +1
-0+ res_evict,true, true);
if (!err &&op->remap.next)
err = vma_lock_and_validate(exec, op-Quoted 1 line
remap.next,
drivers/gpu/drm/xe/xe_vm.c +0
-1- res_evict,true);
drivers/gpu/drm/xe/xe_vm.c +1
-0+ res_evict,true, true);
break;
caseDRM_GPUVA_OP_UNMAP:
err = check_ufence(gpuva_to_vma(op-Quoted 1 line
base.unmap.va));
drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -3014,7 +3020,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
err = vma_lock_and_validate(exec, gpuva_to_vma(op-Quoted 1 line
base.unmap.va),
drivers/gpu/drm/xe/xe_vm.c +1
-1- res_evict, false); + res_evict, false,false);
break;
caseDRM_GPUVA_OP_PREFETCH:
{drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -3029,7 +3035,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm,
err = vma_lock_and_validate(exec, gpuva_to_vma(op-Quoted 1 line
base.prefetch.va),
drivers/gpu/drm/xe/xe_vm.c +1
-1- res_evict, false); + res_evict, false,true);
if (!err && !xe_vma_has_no_bo(vma)) err = xe_bo_migrate(xe_vma_bo(vma),
region_to_mem_type[region],Signature · 1 line
2.43.0
Re: [PATCH v4 5/8] drm/xe/vm: Prevent binding of purged buffer objects
Quoted 78 lines · Thomas Hellström
Quoted 65 lines · Arvind
Quoted 64 lines · Matthew Brost
Quoted 46 lines · Arvind Yadav
Add
check_purgedparameter tovma_lock_and_validate()to block new mapping operations on purged BOs while allowing cleanup operations to proceed.Purged BOs have their backing pages freed by the kernel. New mapping operations (MAP, PREFETCH, REMAP) must be rejected with
-EINVALto prevent GPU access to invalid memory. Cleanup operations (UNMAP) must be allowed so applications can release resources after detecting purge via the retained field.REMAP operations require mixed handling - reject new prev/next VMAs if the BO is purged, but allow the unmap portion to proceed for cleanup.
The
check_purgedparameter distinguishes between these cases:
true for new mappings (must reject), false for cleanup (allow).v2:
- Clarify that purged BOs are permanently invalid (i915
semantics)
- Remove incorrect claim about madvise(WILLNEED) restoring
purged BOsv3:
- Movexe_bo_is_purgedcheck undervma_lock_and_validate
(Matthew Brost)
- Addcheck_purgedparameter to distinguish new mappings from
cleanup
- Allow UNMAP operations to prevent resource leaks
- Handle REMAP operation's dual nature (cleanup + new
mappings)Cc Matthew Brost<[email protected]>Cc Thomas Hellström<[email protected]>Cc Himal Prasad Ghimiray<[email protected]>Signed-off-by Arvind Yadav<[email protected]>--- drivers/gpu/drm/xe/xe_vm.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)drivers/gpu/drm/xe/xe_vm.c +0
-0drivers/gpu/drm/xe/xe_vm.c static void vm_bind_ioctl_ops_unwind(struct · line 2889
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -2883,7 +2883,7 @@ static void vm_bind_ioctl_ops_unwind(structxe_vm *vm, } static int vma_lock_and_validate(struct drm_exec *exec, structxe_vma *vma, - bool res_evict, bool validate) +bool res_evict,bool validate,bool check_purged)It probably time to add something like this to avoid transposing arguments.
struct lock_and_validate_flags { bool res_evict; bool validate; bool check_purged; };Quoted 3 lines
Quoted 1 line
Logic in the patch looks correct though.
Noted, I will add "struct
xe_lock_and_validate_flags" Thanks, ArvindNote that if you follow the pattern of struct
xe_bo_shink_flags, passing the struct as a const and usingstruct lock_and_validate_flags { u32 res_evict : 1; u32 validate : 1; u32 check_purged : 1; };
Quoted 5 lines
This will be more or less equivalent to passing a bit-field with type- checking.
Some reviewers frown on using "bool" in compound types although we accept that in the xe driver.
Noted, I will do the changes as per suggestion..
Thanks,
Arvind
Quoted 96 lines
Otherwise patch LGTM as well.
/Thomas
Quoted 91 lines
Quoted 91 lines
Matt
Quoted 89 lines
{
structxe_bo*bo =xe_vma_bo(vma);
structxe_vm*vm =xe_vma_vm(vma);drivers/gpu/drm/xe/xe_vm.c +2
-0@@ -2892,6 +2892,11 @@ static int vma_lock_and_validate(struct drm_exec *exec, struct xe_vma *vma, if (bo) { if (!bo->vm) err = drm_exec_lock_obj(exec, &bo- > ttm.base); + +/* Reject new mappings to purged BOs; allow cleanup operations */
drivers/gpu/drm/xe/xe_vm.c +3
-0+if (!err && check_purged &&xe_bo_is_purged (bo)) + err = -EINVAL ; + if (!err && validate) err = xe_bo_validate(bo, vm,!xe_vm_in_preempt_fence_mode(vm) &&
drivers/gpu/drm/xe/xe_vm.c +0
-0@@ -2990,7 +2995,8 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err = vma_lock_and_validate(exec, op- > map.vma, res_evict,!xe_vm_in_fault_mode(vm) ||
drivers/gpu/drm/xe/xe_vm.c +0
-1- op-Quoted 1 line
map.immediate);
drivers/gpu/drm/xe/xe_vm.c +1
-0+ op-Quoted 1 line
map.immediate,
drivers/gpu/drm/xe/xe_vm.c +1
-0+ true); break; case DRM_GPUVA_OP_REMAP: err = check_ufence(gpuva_to_vma(op-Quoted 1 line
base.remap.unmap->va));drivers/gpu/drm/xe/xe_vm.c +2
-2@@ -2999,13 +3005,13 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err = vma_lock_and_validate(exec, gpuva_to_vma(op- > base.remap.unmap->va), - res_evict, false); + res_evict, false, false); if (!err && op->remap.prev) err = vma_lock_and_validate(exec, op- > remap.prev, - res_evict, true); + res_evict, true, true);if (!err &&
op->remap.next)
err = vma_lock_and_validate(exec, op-Quoted 1 line
remap.next,
drivers/gpu/drm/xe/xe_vm.c +0
-1- res_evict,true);
drivers/gpu/drm/xe/xe_vm.c +1
-0+ res_evict,true, true);
break;
caseDRM_GPUVA_OP_UNMAP:
err = check_ufence(gpuva_to_vma(op-Quoted 1 line
base.unmap.va));
drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -3014,7 +3020,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err = vma_lock_and_validate(exec, gpuva_to_vma(op- > base.unmap.va), - res_evict, false); + res_evict, false, false);break;
caseDRM_GPUVA_OP_PREFETCH:
{drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -3029,7 +3035,7 @@ static int op_lock_and_prep(struct drm_exec *exec, struct xe_vm *vm, err = vma_lock_and_validate(exec, gpuva_to_vma(op- > base.prefetch.va), - res_evict, false); + res_evict, false, true);if (!err && !xe_vma_has_no_bo(vma)) err = xe_bo_migrate(xe_vma_bo(vma),
region_to_mem_type[region],
Signature · 1 line
2.43.0
PATCH v4 6/8 drm/xe/madvise: Implement per-VMA purgeable state tracking
In reply to Arvind YadavTrack purgeable state per-VMA instead of using a coarse shared BO check. This prevents purging shared BOs until all VMAs across all VMs are marked DONTNEED.
Add xe_bo_all_vmas_dontneed() to check all VMAs before marking a BO purgeable. Add xe_bo_recheck_purgeable_on_vma_unbind() to handle state transitions when VMAs are destroyed - if all remaining VMAs are DONTNEED the BO can become purgeable, or if no VMAs remain it transitions to WILLNEED.
The per-VMA purgeable_state field stores the madvise hint for each mapping. Shared BOs can only be purged when all VMAs unanimously indicate DONTNEED.
v3:
- This addresses Thomas Hellström's feedback: "loop over all vmas attached to the bo and check that they all say WONTNEED. This will also need a check at VMA unbinding"
v4:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
--- drivers/gpu/drm/xe/xe_vm.c | 15 +++++- drivers/gpu/drm/xe/xe_vm_madvise.c | 84 +++++++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_vm_madvise.h | 3 ++ drivers/gpu/drm/xe/xe_vm_types.h | 11 ++++ 4 files changed, 111 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +14 -1
@@ -40,6 +40,7 @@#include "xe_tile.h" #include "xe_tlb_inval.h" #include "xe_trace_bo.h" +#include "xe_vm_madvise.h" #include "xe_wa.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm)
@@ -1079,12 +1080,18 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,static void xe_vma_destroy_late (struct xe_vma *vma) {struct xe_vm *vm =xe_vma_vm (vma); +struct xe_bo *bo =NULL ;if (vma->ufence) {xe_sync_ufence_put (vma->ufence); vma->ufence =NULL ; } +/* Get BO reference for purgeable state re-check */ +if (!xe_vma_is_userptr (vma) && !xe_vma_is_null (vma) && + !xe_vma_is_cpu_addr_mirror (vma)) + bo =xe_vma_bo (vma); +if (xe_vma_is_userptr (vma)) {struct xe_userptr_vma *uvma =to_userptr_vma (vma);
@@ -1093,7 +1100,13 @@ static void xe_vma_destroy_late(struct xe_vma *vma) }else if (xe_vma_is_null (vma) ||xe_vma_is_cpu_addr_mirror (vma)) {xe_vm_put (vm); }else { -xe_bo_put (xe_vma_bo (vma)); +/* Trylock safe for async context; madvise corrects failures */ +if (bo &&dma_resv_trylock (bo->ttm.base.resv)) { +xe_bo_recheck_purgeable_on_vma_unbind (bo); +dma_resv_unlock (bo->ttm.base.resv); + } + +xe_bo_put (bo); }xe_vma_free (vma);
drivers/gpu/drm/xe/xe_vm_madvise.c +83 -1
105 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm_madvise.h +3 -0
@@ -8,8 +8,11 @@struct drm_device ;struct drm_file ; +struct xe_bo ;int xe_vm_madvise_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo); +#endif
drivers/gpu/drm/xe/xe_vm_types.h +11 -0
@@ -150,6 +150,17 @@ struct xe_vma {*/ bool skip_invalidation; +/** +* @purgeable_state: Purgeable hint for this VMA mapping +* +* Per-VMA purgeable state from madvise. Valid states are WILLNEED (0) +* or DONTNEED (1). Shared BOs require all VMAs to be DONTNEED before +* the BO can be purged. PURGED state exists only at BO level. +* +* Protected by BO dma-resv lock. Set via DRM_IOCTL_XE_MADVISE. +*/ +u32 purgeable_state; +/** * @ufence: The user fence that was provided with MAP. * Needs to be signalled before UNMAP can be processed.
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 6/8] drm/xe/madvise: Implement per-VMA purgeable state tracking
Quoted 50 lines · Arvind Yadav
Track purgeable state per-VMA instead of using a coarse shared BO check. This prevents purging shared BOs until all VMAs across all VMs are marked DONTNEED.
Add
xe_bo_all_vmas_dontneed()to check all VMAs before marking a BO purgeable. Addxe_bo_recheck_purgeable_on_vma_unbind()to handle state transitions when VMAs are destroyed - if all remaining VMAs are DONTNEED the BO can become purgeable, or if no VMAs remain it transitions to WILLNEED.The per-VMA
purgeable_statefield stores the madvise hint for each mapping. Shared BOs can only be purged when all VMAs unanimously indicate DONTNEED.v3:
- This addresses Thomas Hellström's feedback: "loop over all vmas attached to the bo and check that they all say WONTNEED. This will also need a check at VMA unbinding"v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_vm.c | 15 +++++- drivers/gpu/drm/xe/xe_vm_madvise.c | 84 +++++++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_vm_madvise.h | 3 ++ drivers/gpu/drm/xe/xe_vm_types.h | 11 ++++ 4 files changed, 111 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_vm.c +2
-0@@ -40,6 +40,7 @@#include "xe_tile.h" #include "xe_tlb_inval.h" #include "xe_trace_bo.h" +#include "xe_vm_madvise.h" #include "xe_wa.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm)@@ -1079,12 +1080,18 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,static void xe_vma_destroy_late (struct xe_vma *vma) {struct xe_vm *vm =xe_vma_vm (vma); +struct xe_bo *bo =NULL ;
drivers/gpu/drm/xe/xe_vm.c static struct xe_vma *xe_vma_create(struct xe_vm *vm, · line 1093
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0if (vma->ufence) {xe_sync_ufence_put (vma->ufence); vma->ufence =NULL ; } +/* Get BO reference for purgeable state re-check */ +if (!xe_vma_is_userptr (vma) && !xe_vma_is_null (vma) && + !xe_vma_is_cpu_addr_mirror (vma)) + bo =xe_vma_bo (vma);
I think xe_vma_bo just returns NULL if any of the above conditions are met, so I believe is ok to just blindly call xe_vma_bo as you have a NULL check on the BO below.
drivers/gpu/drm/xe/xe_vm.c static void xe_vma_destroy_late(struct xe_vma *vma) · line 1105
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm.c +4
-1+if (xe_vma_is_userptr (vma)) {struct xe_userptr_vma *uvma =to_userptr_vma (vma);@@ -1093,7 +1100,13 @@ static void xe_vma_destroy_late(struct xe_vma *vma) }else if (xe_vma_is_null (vma) ||xe_vma_is_cpu_addr_mirror (vma)) {xe_vm_put (vm); }else { -xe_bo_put (xe_vma_bo (vma)); +/* Trylock safe for async context; madvise corrects failures */ +if (bo &&dma_resv_trylock (bo->ttm.base.resv)) { +xe_bo_recheck_purgeable_on_vma_unbind (bo);
Also I don't think the correct place to call this.
I believe you can call this function in xe_vma_destroy after drm_gpuva_unlink. You also have BO lock there too so no need from this trylock path.
Quoted 41 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0+dma_resv_unlock (bo->ttm.base.resv); + } + +xe_bo_put (bo); }xe_vma_free (vma);drivers/gpu/drm/xe/xe_vm_madvise.c +19
-0@@ -12,6 +12,7 @@#include "xe_pat.h" #include "xe_pt.h" #include "xe_svm.h" +#include "xe_vm.h" struct xe_vmas_in_madvise_range {u64 addr;@@ -179,6 +180,80 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/** +* xe_bo_all_vmas_dontneed() - Check if all VMAs of a BO are marked DONTNEED +* @bo: Buffer object +* +* Check all VMAs across all VMs to determine if BO can be purged. +* Shared BOs require unanimous DONTNEED state from all mappings. +* +* Caller must hold BO dma-resv lock. +* +* Return: true if all VMAs are DONTNEED, false otherwise +*/ +static bool xe_bo_all_vmas_dontneed (struct xe_bo *bo) +{ +struct drm_gpuvm_bo *vm_bo; +struct drm_gpuva *gpuva; +struct drm_gem_object *obj = &bo->ttm.base; +bool has_vmas = false; +
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 210
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+dma_resv_assert_held (bo->ttm.base.resv); + +drm_gem_for_each_gpuvm_bo (vm_bo, obj) { +drm_gpuvm_bo_for_each_va (gpuva, vm_bo) { +struct xe_vma *vma =gpuva_to_vma (gpuva); + + has_vmas = true; + +/* Any non-DONTNEED VMA prevents purging */ +if (READ_ONCE (vma->purgeable_state) !=XE_MADV_PURGEABLE_DONTNEED )
You don't need the READ_ONCE as purgeable_state is only accessed under the dma-resv lock, also there isn't a WRITE_ONCE anywhere.
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 215
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+return false; + } + } + +/* No VMAs means not purgeable */
No VMAs means it is purgeable, right?
Quoted 88 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +49
-1+if (!has_vmas) +return false; + +return true; +} + +/** +* xe_bo_recheck_purgeable_on_vma_unbind() - Re-evaluate BO purgeable state after VMA unbind +* @bo: Buffer object +* +* When a VMA is unbound, re-check if the BO's purgeable state should change. +* Destroyed VMAs may allow the BO to become purgeable if all remaining VMAs +* are DONTNEED, or require transition to WILLNEED if no VMAs remain. +* +* Called from VMA destruction path with BO dma-resv lock held. +*/ +void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo) +{ +if (!bo) +return ; + +dma_resv_assert_held (bo->ttm.base.resv); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ) +return ; + +if (xe_bo_all_vmas_dontneed (bo)) { +/* All VMAs are DONTNEED - mark BO purgeable */ +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; + }else { +/* At least one VMA is WILLNEED - BO must not be purgeable */ +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; + } +} +/* * Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. * Returns true if any BO was purged, false otherwise. @@ -213,10 +288,17 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,switch (op->purge_state_val.val) {case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + vmas[i]->purgeable_state =XE_MADV_PURGEABLE_WILLNEED ; + +/* Mark VMA WILLNEED - BO becomes non-purgeable immediately */ bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ;break ;case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : - bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; + vmas[i]->purgeable_state =XE_MADV_PURGEABLE_DONTNEED ; + +/* Mark BO purgeable only if all VMAs are DONTNEED */ +if (xe_bo_all_vmas_dontneed (bo)) + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ;break ;default : drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",drivers/gpu/drm/xe/xe_vm_madvise.h +3
-0@@ -8,8 +8,11 @@struct drm_device ;struct drm_file ; +struct xe_bo ;int xe_vm_madvise_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo); +#endif drivers/gpu/drm/xe/xe_vm_types.h +1
-0@@ -150,6 +150,17 @@ struct xe_vma {*/ bool skip_invalidation; +/**
drivers/gpu/drm/xe/xe_vm_types.h struct xe_vma { · line 163
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_types.h +10
-0+* @purgeable_state: Purgeable hint for this VMA mapping +* +* Per-VMA purgeable state from madvise. Valid states are WILLNEED (0) +* or DONTNEED (1). Shared BOs require all VMAs to be DONTNEED before +* the BO can be purged. PURGED state exists only at BO level. +* +* Protected by BO dma-resv lock. Set via DRM_IOCTL_XE_MADVISE. +*/ +u32 purgeable_state; +
I think xe_vma_mem_attr is a better place for this field.
Matt
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_types.h +0
-0/** * @ufence: The user fence that was provided with MAP. * Needs to be signalled before UNMAP can be processed. Signature · 1 line
2.43.0
Re: [PATCH v4 6/8] drm/xe/madvise: Implement per-VMA purgeable state tracking
Quoted 57 lines · Matthew Brost
Quoted 56 lines · Arvind Yadav
Track purgeable state per-VMA instead of using a coarse shared BO check. This prevents purging shared BOs until all VMAs across all VMs are marked DONTNEED.
Add
xe_bo_all_vmas_dontneed()to check all VMAs before marking a BO purgeable. Addxe_bo_recheck_purgeable_on_vma_unbind()to handle state transitions when VMAs are destroyed - if all remaining VMAs are DONTNEED the BO can become purgeable, or if no VMAs remain it transitions to WILLNEED.The per-VMA
purgeable_statefield stores the madvise hint for each mapping. Shared BOs can only be purged when all VMAs unanimously indicate DONTNEED.v3:
- This addresses Thomas Hellström's feedback: "loop over all vmas attached to the bo and check that they all say WONTNEED. This will also need a check at VMA unbinding"v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_vm.c | 15 +++++- drivers/gpu/drm/xe/xe_vm_madvise.c | 84 +++++++++++++++++++++++++++++- drivers/gpu/drm/xe/xe_vm_madvise.h | 3 ++ drivers/gpu/drm/xe/xe_vm_types.h | 11 ++++ 4 files changed, 111 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_vm.c +2
-0@@ -40,6 +40,7 @@#include "xe_tile.h" #include "xe_tlb_inval.h" #include "xe_trace_bo.h" +#include "xe_vm_madvise.h" #include "xe_wa.h" static struct drm_gem_object *xe_vm_obj (struct xe_vm *vm)@@ -1079,12 +1080,18 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm,static void xe_vma_destroy_late (struct xe_vma *vma) {struct xe_vm *vm =xe_vma_vm (vma); +struct xe_bo *bo =NULL ;if (vma->ufence) {xe_sync_ufence_put (vma->ufence); vma->ufence =NULL ; }
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm.c static struct xe_vma *xe_vma_create(struct xe_vm *vm, · line 1093
Quoted 4 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0+/* Get BO reference for purgeable state re-check */ +if (!xe_vma_is_userptr (vma) && !xe_vma_is_null (vma) && + !xe_vma_is_cpu_addr_mirror (vma)) + bo =xe_vma_bo (vma);I think
xe_vma_bojust returns NULL if any of the above conditions are met, so I believe is ok to just blindly callxe_vma_boas you have a NULL check on the BO below.
Right, xe_vma_bo() already returns NULL for those cases. Will simplify to just bo = xe_vma_bo(vma).
Quoted 15 lines
drivers/gpu/drm/xe/xe_vm.c static void xe_vma_destroy_late(struct xe_vma *vma) · line 1105
Quoted 13 lines
drivers/gpu/drm/xe/xe_vm.c +4
-1+if (xe_vma_is_userptr (vma)) {struct xe_userptr_vma *uvma =to_userptr_vma (vma);@@ -1093,7 +1100,13 @@ static void xe_vma_destroy_late(struct xe_vma *vma) }else if (xe_vma_is_null (vma) ||xe_vma_is_cpu_addr_mirror (vma)) {xe_vm_put (vm); }else { -xe_bo_put (xe_vma_bo (vma)); +/* Trylock safe for async context; madvise corrects failures */ +if (bo &&dma_resv_trylock (bo->ttm.base.resv)) { +xe_bo_recheck_purgeable_on_vma_unbind (bo);Also I don't think the correct place to call this.
Quoted 3 lines
I believe you can call this function in
xe_vma_destroyafterdrm_gpuva_unlink. You also have BO lock there too so no need from this trylock path.
Good catch! Will move to xe_vma_destroy() after drm_gpuva_unlink() where we already have the BO lock. No trylock needed there.
Quoted 45 lines
Quoted 45 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0+dma_resv_unlock (bo->ttm.base.resv); + } + +xe_bo_put (bo); }xe_vma_free (vma);drivers/gpu/drm/xe/xe_vm_madvise.c +23
-0@@ -12,6 +12,7 @@#include "xe_pat.h" #include "xe_pt.h" #include "xe_svm.h" +#include "xe_vm.h" struct xe_vmas_in_madvise_range {u64 addr;@@ -179,6 +180,80 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/** +* xe_bo_all_vmas_dontneed() - Check if all VMAs of a BO are marked DONTNEED +* @bo: Buffer object +* +* Check all VMAs across all VMs to determine if BO can be purged. +* Shared BOs require unanimous DONTNEED state from all mappings. +* +* Caller must hold BO dma-resv lock. +* +* Return: true if all VMAs are DONTNEED, false otherwise +*/ +static bool xe_bo_all_vmas_dontneed (struct xe_bo *bo) +{ +struct drm_gpuvm_bo *vm_bo; +struct drm_gpuva *gpuva; +struct drm_gem_object *obj = &bo->ttm.base; +bool has_vmas = false; + +dma_resv_assert_held (bo->ttm.base.resv); + +drm_gem_for_each_gpuvm_bo (vm_bo, obj) { +drm_gpuvm_bo_for_each_va (gpuva, vm_bo) {
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 210
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +6
-0+struct xe_vma *vma =gpuva_to_vma (gpuva); + + has_vmas = true; + +/* Any non-DONTNEED VMA prevents purging */ +if (READ_ONCE (vma->purgeable_state) !=XE_MADV_PURGEABLE_DONTNEED )You don't need the
READ_ONCEaspurgeable_stateis only accessed under the dma-resv lock, also there isn't aWRITE_ONCEanywhere.
Noted - I will remove READ_ONCE() - the dma-resv lock provides the necessary protection.
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 215
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+return false; + } + } + +/* No VMAs means not purgeable */ No VMAs means it is purgeable, right?
I kept “no VMAs => not purgeable” since otherwise we can mark BOs purgeable without any userspace DONTNEED hint (No VMAs means no user hint at all).
Quoted 87 lines
Quoted 87 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +49
-1+if (!has_vmas) +return false; + +return true; +} + +/** +* xe_bo_recheck_purgeable_on_vma_unbind() - Re-evaluate BO purgeable state after VMA unbind +* @bo: Buffer object +* +* When a VMA is unbound, re-check if the BO's purgeable state should change. +* Destroyed VMAs may allow the BO to become purgeable if all remaining VMAs +* are DONTNEED, or require transition to WILLNEED if no VMAs remain. +* +* Called from VMA destruction path with BO dma-resv lock held. +*/ +void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo) +{ +if (!bo) +return ; + +dma_resv_assert_held (bo->ttm.base.resv); + +/* +* Once purged, always purged. Cannot transition back to WILLNEED. +* This matches i915 semantics where purged BOs are permanently invalid. +*/ +if (bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ) +return ; + +if (xe_bo_all_vmas_dontneed (bo)) { +/* All VMAs are DONTNEED - mark BO purgeable */ +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; + }else { +/* At least one VMA is WILLNEED - BO must not be purgeable */ +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) + bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; + } +} +/* * Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED. * Returns true if any BO was purged, false otherwise. @@ -213,10 +288,17 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,switch (op->purge_state_val.val) {case DRM_XE_VMA_PURGEABLE_STATE_WILLNEED : + vmas[i]->purgeable_state =XE_MADV_PURGEABLE_WILLNEED ; + +/* Mark VMA WILLNEED - BO becomes non-purgeable immediately */ bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ;break ;case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : - bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; + vmas[i]->purgeable_state =XE_MADV_PURGEABLE_DONTNEED ; + +/* Mark BO purgeable only if all VMAs are DONTNEED */ +if (xe_bo_all_vmas_dontneed (bo)) + bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ;break ;default : drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",drivers/gpu/drm/xe/xe_vm_madvise.h +3
-0@@ -8,8 +8,11 @@struct drm_device ;struct drm_file ; +struct xe_bo ;int xe_vm_madvise_ioctl (struct drm_device *dev,void *data,struct drm_file *file); +void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo); +#endif drivers/gpu/drm/xe/xe_vm_types.h +0
-0@@ -150,6 +150,17 @@ struct xe_vma {*/ bool skip_invalidation;
Quoted 12 lines
drivers/gpu/drm/xe/xe_vm_types.h struct xe_vma { · line 163
Quoted 11 lines
drivers/gpu/drm/xe/xe_vm_types.h +11
-0+/** +* @purgeable_state: Purgeable hint for this VMA mapping +* +* Per-VMA purgeable state from madvise. Valid states are WILLNEED (0) +* or DONTNEED (1). Shared BOs require all VMAs to be DONTNEED before +* the BO can be purged. PURGED state exists only at BO level. +* +* Protected by BO dma-resv lock. Set via DRM_IOCTL_XE_MADVISE. +*/ +u32 purgeable_state; +I think
xe_vma_mem_attris a better place for this field.
Noted - I will move purgeable_state into xe_vma_mem_attr for better organization.
Thanks,
Arvind
Quoted 9 lines
Matt
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_types.h +0
-0/** * @ufence: The user fence that was provided with MAP. * Needs to be signalled before UNMAP can be processed. Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 6/8] drm/xe/madvise: Implement per-VMA purgeable state tracking
In reply to Matthew BrostQuoted 156 lines · Matthew Brost
Quoted 61 lines · Arvind Yadav
Track purgeable state per-VMA instead of using a coarse shared BO check. This prevents purging shared BOs until all VMAs across all VMs are marked DONTNEED.
Add
xe_bo_all_vmas_dontneed()to check all VMAs before marking a BO purgeable. Addxe_bo_recheck_purgeable_on_vma_unbind()to handle state transitions when VMAs are destroyed - if all remaining VMAs are DONTNEED the BO can become purgeable, or if no VMAs remain it transitions to WILLNEED.The per-VMA
purgeable_statefield stores the madvise hint for each mapping. Shared BOs can only be purged when all VMAs unanimously indicate DONTNEED.v3:
- This addresses Thomas Hellström's feedback: "loop over all vmas attached to the bo and check that they all say WONTNEED. This will
also need a check at VMA unbinding"v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_vm.c | 15 +++++-
drivers/gpu/drm/xe/xe_vm_madvise.c | 84
+++++++++++++++++++++++++++++-
drivers/gpu/drm/xe/xe_vm_madvise.h | 3 ++
drivers/gpu/drm/xe/xe_vm_types.h | 11 ++++4 files changed, 111 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_vm.c +1
-0@@ -40,6 +40,7 @@ #include "xe_tile.h" #include "xe_tlb_inval.h" #include "xe_trace_bo.h" +#include "xe_vm_madvise.h" #include "xe_wa.h"
static structdrm_gem_object*xe_vm_obj(structxe_vm*vm)drivers/gpu/drm/xe/xe_vm.c +1
-0@@ -1079,12 +1080,18 @@ static struct xe_vma *xe_vma_create(structxe_vm *vm, static void xe_vma_destroy_late(struct xe_vma *vma) { struct xe_vm *vm = xe_vma_vm(vma); +struct xe_bo *bo =NULL ;
if (vma->ufence) {
xe_sync_ufence_put(vma->ufence);
vma->ufence= NULL;
}drivers/gpu/drm/xe/xe_vm.c static struct xe_vma *xe_vma_create(struct xe_vm *vm, · line 1093
Quoted 4 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0+/* Get BO reference for purgeable state re-check */ +if (!xe_vma_is_userptr (vma) && !xe_vma_is_null (vma) && + !xe_vma_is_cpu_addr_mirror(vma)) + bo =xe_vma_bo (vma);I think
xe_vma_bojust returns NULL if any of the above conditions are
met, so I believe is ok to just blindly callxe_vma_boas you have a NULL check on the BO below.Quoted 13 lines
+
if (xe_vma_is_userptr(vma)) {
structxe_userptr_vma*uvma =to_userptr_vma(vma);drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -1093,7 +1100,13 @@ static void xe_vma_destroy_late(struct xe_vma *vma) } else if (xe_vma_is_null(vma) ||xe_vma_is_cpu_addr_mirror (vma)) { xe_vm_put(vm); } else { -xe_bo_put (xe_vma_bo (vma)); +/* Trylock safe for async context; madvise corrects failures */
drivers/gpu/drm/xe/xe_vm.c static void xe_vma_destroy_late(struct xe_vma *vma) · line 1105
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm.c +2
-0+if (bo &&dma_resv_trylock (bo->ttm.base.resv)) { +xe_bo_recheck_purgeable_on_vma_unbind (bo);Also I don't think the correct place to call this.
I believe you can call this function in
xe_vma_destroyafterdrm_gpuva_unlink. You also have BO lock there too so no need from this
trylock path.Quoted 55 lines
drivers/gpu/drm/xe/xe_vm.c +4
-0+dma_resv_unlock (bo->ttm.base.resv); + } + +xe_bo_put (bo);}
xe_vma_free(vma);drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0@@ -12,6 +12,7 @@ #include "xe_pat.h" #include "xe_pt.h" #include "xe_svm.h" +#include "xe_vm.h"
structxe_vmas_in_madvise_range{
u64 addr;drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -179,6 +180,80 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } }
+/**
+ *xe_bo_all_vmas_dontneed()- Check if all VMAs of a BO are marked DONTNEED
+ * @bo: Buffer object
+ *
+ * Check all VMAs across all VMs to determine if BO can be purged.
+ * Shared BOs require unanimous DONTNEED state from all mappings.
+ *
+ * Caller must hold BO dma-resv lock.
+ *
+ * Return: true if all VMAs are DONTNEED, false otherwise
+ */drivers/gpu/drm/xe/xe_vm_madvise.c +16
-0+static bool xe_bo_all_vmas_dontneed (struct xe_bo *bo) +{ +struct drm_gpuvm_bo *vm_bo; +struct drm_gpuva *gpuva; +struct drm_gem_object *obj = &bo->ttm.base; +bool has_vmas = false; + +dma_resv_assert_held (bo->ttm.base.resv); + +drm_gem_for_each_gpuvm_bo (vm_bo, obj) { +drm_gpuvm_bo_for_each_va (gpuva, vm_bo) { +struct xe_vma *vma =gpuva_to_vma (gpuva); + + has_vmas = true; + +/* Any non-DONTNEED VMA prevents purging */
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ if (READ_ONCE(vma->purgeable_state) !=Quoted 1 line
XE_MADV_PURGEABLE_DONTNEED)You don't need the
READ_ONCEaspurgeable_stateis only accessed under
the dma-resv lock, also there isn't aWRITE_ONCEanywhere.
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 215
Quoted 5 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +5
-0+return false; + } + } + +/* No VMAs means not purgeable */ No VMAs means it is purgeable, right?
Then it would be purgeable just after creation, even if CPU is accessing. However I think we should consider the case where a bo is purgeable and it's single VMA goes away. Then I'd say it's still purgeable.
I guess this is our last chance to, after discussing for years, revert to a bo madvise UAPI before the UAPI is set in stone, given the above corner-cases. I must admit I'm not 100% sure myself, given CPU madvises are always based on the virtual address map, and we try to mimic that as much as possible.
Quoted 28 lines
Quoted 28 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +8
-0+if (!has_vmas) +return false; + +return true; +} + +/** +* xe_bo_recheck_purgeable_on_vma_unbind() - Re-evaluate BO purgeable state after VMA unbind
+ * @bo: Buffer object
+ *
+ * When a VMA is unbound, re-check if the BO's purgeable state should change.
+ * Destroyed VMAs may allow the BO to become purgeable if all remaining VMAs
+ * are DONTNEED, or require transition to WILLNEED if no VMAs remain.
+ *
+ * Called from VMA destruction path with BO dma-resv lock held.
+ */drivers/gpu/drm/xe/xe_vm_madvise.c +7
-0+void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo) +{ +if (!bo) +return ; + +dma_resv_assert_held (bo->ttm.base.resv); +
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 244
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+/* +* Once purged, always purged. Cannot transition back to WILLNEED.
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+* This matches i915 semantics where purged BOs are permanently invalid.
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+*/ +if (bo->madv_purgeable ==XE_MADV_PURGEABLE_PURGED ) +return ;
Perhaps replace the below with bo->madv_purgeable = xe_bo_all_vmas_dontneed(bo) ? XE_MADV_PURGEABLE_DONTNEED : XE_MADV_PURGEABLE_WILLNEED;
Quoted 41 lines
Quoted 41 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+ +if (xe_bo_all_vmas_dontneed (bo)) { +/* All VMAs are DONTNEED - mark BO purgeable */ + if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_DONTNEED)drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ bo->madv_purgeable =
XE_MADV_PURGEABLE_DONTNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+ }else { +/* At least one VMA is WILLNEED - BO must not be purgeable */
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_WILLNEED)drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ bo->madv_purgeable =
XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+ } +} +/*
* Handle purgeable buffer object advice for DONTNEED/WILLNEED/PURGED.
* Returns true if any BO was purged, false otherwise.drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -213,10 +288,17 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,
switch (op->purge_state_val.val) { caseDRM_XE_VMA_PURGEABLE_STATE_WILLNEED:drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ vmas[i]->purgeable_state =
XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+ +/* Mark VMA WILLNEED - BO becomes non- purgeable immediately */
bo->madv_purgeable=XE_MADV_PURGEABLE_WILLNEED;
break;
caseDRM_XE_VMA_PURGEABLE_STATE_DONTNEED:drivers/gpu/drm/xe/xe_vm_madvise.c +0
-1- bo->madv_purgeable =
XE_MADV_PURGEABLE_DONTNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ vmas[i]->purgeable_state =
XE_MADV_PURGEABLE_DONTNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+ +/* Mark BO purgeable only if all VMAs are
Quoted 8 lines
Quoted 8 lines
DONTNEED */
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+if (xe_bo_all_vmas_dontneed (bo)) + bo->madv_purgeable =
XE_MADV_PURGEABLE_DONTNEED;
break;
default:
drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",
This is part of the IOCTL processing, right? Then we should use XE_IOCTL_DBG() on invalid values, or xe_assert() if input values are already checked elsewhere.
Quoted 35 lines
Quoted 35 lines
drivers/gpu/drm/xe/xe_vm_madvise.h +0
-0@@ -8,8 +8,11 @@
structdrm_device;
structdrm_file;
+structxe_bo;
intxe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *file);drivers/gpu/drm/xe/xe_vm_madvise.h +2
-0+void xe_bo_recheck_purgeable_on_vma_unbind (struct xe_bo *bo); +#endif
drivers/gpu/drm/xe/xe_vm_types.h +0
-0@@ -150,6 +150,17 @@ struct xe_vma {*/ bool skip_invalidation;drivers/gpu/drm/xe/xe_vm_types.h +4
-0+/** +* @purgeable_state: Purgeable hint for this VMA mapping +* +* Per-VMA purgeable state from madvise. Valid states are WILLNEED (0)
drivers/gpu/drm/xe/xe_vm_types.h +1
-0+* or DONTNEED (1). Shared BOs require all VMAs to be DONTNEED before
drivers/gpu/drm/xe/xe_vm_types.h +1
-0+* the BO can be purged. PURGED state exists only at BO level.
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_types.h struct xe_vma { · line 163
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_types.h +2
-0+* +* Protected by BO dma-resv lock. Set via
DRM_IOCTL_XE_MADVISE.drivers/gpu/drm/xe/xe_vm_types.h +3
-0+*/ +u32 purgeable_state; +I think
xe_vma_mem_attris a better place for this field.
+1.
/Thomas
Quoted 9 lines
Matt
Quoted 6 lines
/**
* @ufence: The user fence that was provided with MAP.
* Needs to be signalled before UNMAP can be processed.Signature · 1 line
2.43.0
PATCH v4 7/8 drm/xe/madvise: Block imported and exported dma-bufs
In reply to Arvind YadavPrevent marking imported or exported dma-bufs as purgeable. External devices may be accessing these buffers without our knowledge, making purging unsafe.
Check drm_gem_is_imported() for buffers created by other drivers and obj->dma_buf for buffers exported to other drivers. Silently skip these BOs during madvise processing.
This follows drm_gem_shmem's purgeable implementation and prevents data corruption from purging actively-used shared buffers.
v3:
- Addresses review feedback from Matt Roper about handling imported/exported BOs correctly in the purgeable BO implementation.
v4:
- Check should be add to xe_vm_madvise_purgeable_bo.
--- drivers/gpu/drm/xe/xe_vm_madvise.c | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
drivers/gpu/drm/xe/xe_vm_madvise.c +33 -0
@@ -180,6 +180,31 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/** +* xe_bo_is_external_dmabuf() - Check if BO is imported or exported dma-buf +* @bo: Buffer object +* +* Prevent marking imported or exported dma-bufs as purgeable. +* External devices may be accessing these buffers without our +* knowledge, making purging unsafe. +* +* Return: true if BO is imported or exported, false otherwise +*/ +static bool xe_bo_is_external_dmabuf (struct xe_bo *bo) +{ +struct drm_gem_object *obj = &bo->ttm.base; + +/* Imported from another driver */ +if (drm_gem_is_imported (obj)) +return true; + +/* Exported to another driver */ +if (obj->dma_buf) +return true; + +return false; +} +/** * xe_bo_all_vmas_dontneed() - Check if all VMAs of a BO are marked DONTNEED * @bo: Buffer object
@@ -200,6 +225,10 @@ static bool xe_bo_all_vmas_dontneed(struct xe_bo *bo)dma_resv_assert_held (bo->ttm.base.resv); +/* External dma-bufs cannot be purgeable */ +if (xe_bo_is_external_dmabuf (bo)) +return false; +drm_gem_for_each_gpuvm_bo (vm_bo, obj) {drm_gpuvm_bo_for_each_va (gpuva, vm_bo) {struct xe_vma *vma =gpuva_to_vma (gpuva);
@@ -277,6 +306,10 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,/* BO must be locked before modifying madv state */ xe_bo_assert_held (bo); +/* Skip external dma-bufs */ +if (xe_bo_is_external_dmabuf (bo)) +continue ; +/* * Once purged, always purged. Cannot transition back to WILLNEED. * This matches i915 semantics where purged BOs are permanently invalid.
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 7/8] drm/xe/madvise: Block imported and exported dma-bufs
Quoted 17 lines · Arvind Yadav
Prevent marking imported or exported dma-bufs as purgeable. External devices may be accessing these buffers without our knowledge, making purging unsafe.
Check
drm_gem_is_imported()for buffers created by other drivers andobj->dma_buffor buffers exported to other drivers. Silently skip these BOs during madvise processing.This follows drm_gem_shmem's purgeable implementation and prevents data corruption from purging actively-used shared buffers.
v3:
- Addresses review feedback from Matt Roper about handling imported/exported BOs correctly in the purgeable BO implementation.
Quoted 5 lines
v4:
- Check should be add toxe_vm_madvise_purgeable_bo.Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>
@Thomas - couple questions below here I need a 2nd opinion on.
Quoted 16 lines
Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_vm_madvise.c | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0@@ -180,6 +180,31 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/** +* xe_bo_is_external_dmabuf() - Check if BO is imported or exported dma-buf
drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 194
Quoted 10 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +10
-0+* @bo: Buffer object +* +* Prevent marking imported or exported dma-bufs as purgeable. +* External devices may be accessing these buffers without our +* knowledge, making purging unsafe. +* +* Return: true if BO is imported or exported, false otherwise +*/ +static bool xe_bo_is_external_dmabuf (struct xe_bo *bo) +{
@Thomas
Should we have this check more generic? e.g., if a BO is not tied to a VM, we don't allow purablity to be set?
Quoted 28 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +17
-0+struct drm_gem_object *obj = &bo->ttm.base; + +/* Imported from another driver */ +if (drm_gem_is_imported (obj)) +return true; + +/* Exported to another driver */ +if (obj->dma_buf) +return true; + +return false; +} +/** * xe_bo_all_vmas_dontneed() - Check if all VMAs of a BO are marked DONTNEED * @bo: Buffer object @@ -200,6 +225,10 @@ static bool xe_bo_all_vmas_dontneed(struct xe_bo *bo)dma_resv_assert_held (bo->ttm.base.resv); +/* External dma-bufs cannot be purgeable */ +if (xe_bo_is_external_dmabuf (bo)) +return false; +drm_gem_for_each_gpuvm_bo (vm_bo, obj) {drm_gpuvm_bo_for_each_va (gpuva, vm_bo) {struct xe_vma *vma =gpuva_to_vma (gpuva);
drivers/gpu/drm/xe/xe_vm_madvise.c static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm, · line 311
Quoted 6 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0@@ -277,6 +306,10 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,/* BO must be locked before modifying madv state */ xe_bo_assert_held (bo); +/* Skip external dma-bufs */ +if (xe_bo_is_external_dmabuf (bo)) +continue ;
@Thomas
I think instead of silently continuing here we fail the IOCTL with -EINVAL?
What do you think?
Matt
Quoted 7 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+/* * Once purged, always purged. Cannot transition back to WILLNEED. * This matches i915 semantics where purged BOs are permanently invalid. Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 7/8] drm/xe/madvise: Block imported and exported dma-bufs
Quoted 59 lines · Matthew Brost
Quoted 17 lines · Arvind Yadav
Prevent marking imported or exported dma-bufs as purgeable. External devices may be accessing these buffers without our knowledge, making purging unsafe.
Check
drm_gem_is_imported()for buffers created by other drivers andobj->dma_buffor buffers exported to other drivers. Silently skip these BOs during madvise processing.This follows drm_gem_shmem's purgeable implementation and prevents data corruption from purging actively-used shared buffers.
v3:
- Addresses review feedback from Matt Roper about handling imported/exported BOs correctly in the purgeable BO implementation.Quoted 5 lines
v4:
- Check should be add toxe_vm_madvise_purgeable_bo.Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>@Thomas - couple questions below here I need a 2nd opinion on.
Quoted 28 lines
Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_vm_madvise.c | 33
++++++++++++++++++++++++++++++1 file changed, 33 insertions(+)drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -180,6 +180,31 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } }
+/**
+ *xe_bo_is_external_dmabuf()- Check if BO is imported or exported dma-buf
+ * @bo: Buffer object
+ *
+ * Prevent marking imported or exported dma-bufs as purgeable.
+ * External devices may be accessing these buffers without our
+ * knowledge, making purging unsafe.
+ *
+ * Return: true if BO is imported or exported, false otherwise
+ */drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 194
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+static bool xe_bo_is_external_dmabuf (struct xe_bo *bo) +{@Thomas
Quoted 3 lines
Should we have this check more generic? e.g., if a BO is not tied to a
VM, we don't allow purablity to be set?
I think if NEO is going to implement this in one form or another, we need to support also external bos as long as they're not exported, since NEO refuses to use local bos.
Quoted 42 lines
Quoted 36 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +13
-0+struct drm_gem_object *obj = &bo->ttm.base; + +/* Imported from another driver */ +if (drm_gem_is_imported (obj)) +return true; + +/* Exported to another driver */ +if (obj->dma_buf) +return true; + +return false; +} +/**
*xe_bo_all_vmas_dontneed()- Check if all VMAs of a BO are marked DONTNEED
* @bo: Buffer objectdrivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -200,6 +225,10 @@ static bool xe_bo_all_vmas_dontneed(struct xe_bo *bo)
dma_resv_assert_held(bo->ttm.base.resv);drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+/* External dma-bufs cannot be purgeable */ +if (xe_bo_is_external_dmabuf (bo)) +return false; +
drm_gem_for_each_gpuvm_bo(vm_bo, obj){drm_gpuvm_bo_for_each_va(gpuva, vm_bo){ structxe_vma*vma =gpuva_to_vma(gpuva);drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -277,6 +306,10 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm, /* BO must be locked before modifying madv state */ xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_vm_madvise.c static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm, · line 311
Quoted 3 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+/* Skip external dma-bufs */ +if (xe_bo_is_external_dmabuf (bo)) +continue ;@Thomas
Quoted 4 lines
I think instead of silently continuing here we fail the IOCTL with
-EINVAL?What do you think?
Hmm. If we view the export as yet another map, then IMO silently continuing would probably be the best option and consistent with sharing across VMs.
Finally I wonder if we should update the purgeable state on final dma- buf attach.
Floating an idea here:
Could the code be simplified if we maintain a map_count and a purgeable_count on each bo? Bo is purgeable iff map_count == purgeable_count. vmas and dma-buf attachments are included in map_count.
Thanks,
Thomas
Quoted 12 lines
Matt
Quoted 9 lines
+
/*
* Once purged, always purged. Cannot transition back to WILLNEED.
* This matches i915 semantics where purged BOs are permanently invalid.Signature · 1 line
2.43.0
Re: [PATCH v4 7/8] drm/xe/madvise: Block imported and exported dma-bufs
Quoted 58 lines · Thomas Hellström
Quoted 57 lines · Matthew Brost
Quoted 17 lines · Arvind Yadav
Prevent marking imported or exported dma-bufs as purgeable. External devices may be accessing these buffers without our knowledge, making purging unsafe.
Check
drm_gem_is_imported()for buffers created by other drivers andobj->dma_buffor buffers exported to other drivers. Silently skip these BOs during madvise processing.This follows drm_gem_shmem's purgeable implementation and prevents data corruption from purging actively-used shared buffers.
v3:
- Addresses review feedback from Matt Roper about handling imported/exported BOs correctly in the purgeable BO implementation.Quoted 5 lines
v4:
- Check should be add toxe_vm_madvise_purgeable_bo.Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>@Thomas - couple questions below here I need a 2nd opinion on.
Quoted 28 lines
Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_vm_madvise.c | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0@@ -180,6 +180,31 @@ static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, } } +/** +* xe_bo_is_external_dmabuf() - Check if BO is imported or exported dma-buf
+ * @bo: Buffer object
+ *
+ * Prevent marking imported or exported dma-bufs as purgeable.
+ * External devices may be accessing these buffers without our
+ * knowledge, making purging unsafe.
+ *
+ * Return: true if BO is imported or exported, false otherwise
+ */drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 194
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+static bool xe_bo_is_external_dmabuf (struct xe_bo *bo) +{@Thomas
Quoted 6 lines
Quoted 3 lines
Should we have this check more generic? e.g., if a BO is not tied to a
VM, we don't allow purablity to be set?I think if NEO is going to implement this in one form or another, we need to support also external bos as long as they're not exported, since NEO refuses to use local bos.
You're right, and I believe the current implementation should support that.
To clarify: "external" here refers only to dma-buf sharing (imported or exported), not to placement. System-memory BOs that NEO typically uses remain fully eligible for purging.
The xe_bo_is_dmabuf_shared() check specifically targets:
- Imported dma-bufs (we don't own backing store)
- Exported dma-bufs (external devices may have active mappings)
Regular Xe-owned system BOs can be marked DONTNEED and purged under memory pressure. We only skip cases where the BO is shared via dma-buf, following the same pattern as drm_gem_shmem_is_purgeable().
Please let me know if I've misunderstood the NEO requirements.
Quoted 40 lines
Quoted 40 lines
Quoted 30 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +13
-0+struct drm_gem_object *obj = &bo->ttm.base; + +/* Imported from another driver */ +if (drm_gem_is_imported (obj)) +return true; + +/* Exported to another driver */ +if (obj->dma_buf) +return true; + +return false; +} + /*** xe_bo_all_vmas_dontneed() - Check if all VMAs of a BO are marked DONTNEED
* @bo: Buffer objectdrivers/gpu/drm/xe/xe_vm_madvise.c +4
-0@@ -200,6 +225,10 @@ static bool xe_bo_all_vmas_dontneed(struct xe_bo *bo) dma_resv_assert_held(bo->ttm.base.resv); +/* External dma-bufs cannot be purgeable */ +if (xe_bo_is_external_dmabuf (bo)) +return false; + drm_gem_for_each_gpuvm_bo(vm_bo, obj) { drm_gpuvm_bo_for_each_va(gpuva, vm_bo) {struct
xe_vma*vma =gpuva_to_vma(gpuva);drivers/gpu/drm/xe/xe_vm_madvise.c static bool xe_vm_madvise_purgeable_bo(struct · line 313
Quoted 8 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0@@ -277,6 +306,10 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm, /* BO must be locked before modifying madv state */ xe_bo_assert_held(bo); +/* Skip external dma-bufs */ +if (xe_bo_is_external_dmabuf (bo)) +continue ;@Thomas
Quoted 7 lines
Quoted 4 lines
I think instead of silently continuing here we fail the IOCTL with
-EINVAL?What do you think?
Hmm. If we view the export as yet another map, then IMO silently continuing would probably be the best option and consistent with sharing across VMs.
Noted, I'll keep the silent skip behavior for dma-buf shared BOs to match the "unanimous DONTNEED" model.
Quoted 3 lines
Finally I wonder if we should update the purgeable state on final dma- buf attach.
Good point. Currently the check only happens at madvise time, so there's a window where a BO marked DONTNEED could later be exported. Would it make sense to hook into the export path to force transition to WILLNEED? That would make the "export as implicit mapping" model complete, though it adds complexity. Let me know your preference.
Quoted 7 lines
Floating an idea here:
Could the code be simplified if we maintain a
map_countand apurgeable_counton each bo? Bo is purgeable iffmap_count==purgeable_count. vmas and dma-buf attachments are included inmap_count.
That's good idea and would avoid VMA iteration. The tradeoff is maintaining correct counts across all VMA and dma-buf lifecycle paths.
For this series, would you prefer I pursue the counter-based refactor now, or is the current iteration approach acceptable with counters as a future optimization? I'm happy to go either direction based on your feedback.
Thanks,
Arvind
Quoted 16 lines
Thanks,
ThomasQuoted 11 lines
Matt
Quoted 9 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ /** Once purged, always purged. Cannot transition
back to WILLNEED.
* This matches i915 semantics where purged BOs
are permanently invalid.Signature · 1 line
2.43.0
Thomas Hellström
maintainerRe: [PATCH v4 7/8] drm/xe/madvise: Block imported and exported dma-bufs
Quoted 84 lines · Arvind
Quoted 59 lines · Thomas Hellström
Quoted 58 lines · Matthew Brost
Quoted 17 lines · Arvind Yadav
Prevent marking imported or exported dma-bufs as purgeable. External devices may be accessing these buffers without our knowledge, making purging unsafe.
Check
drm_gem_is_imported()for buffers created by other drivers andobj->dma_buffor buffers exported to other drivers. Silently skip these BOs during madvise processing.This follows drm_gem_shmem's purgeable implementation and prevents data corruption from purging actively-used shared buffers.
v3:
- Addresses review feedback from Matt Roper about handling imported/exported BOs correctly in the purgeable BO implementation.Quoted 5 lines
v4:
- Check should be add toxe_vm_madvise_purgeable_bo.Cc Matthew Brost <[email protected]>Cc Thomas Hellström <[email protected]>@Thomas - couple questions below here I need a 2nd opinion on.
Quoted 29 lines
Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_vm_madvise.c | 33
++++++++++++++++++++++++++++++1 file changed, 33 insertions(+)drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -180,6 +180,31 @@ static void madvise_pat_index(struct
xe_device
*xe, structxe_vm*vm,
}
}
+/**
+ *xe_bo_is_external_dmabuf()- Check if BO is imported or exported dma-buf
+ * @bo: Buffer object
+ *
+ * Prevent marking imported or exported dma-bufs as purgeable.
+ * External devices may be accessing these buffers without our
+ * knowledge, making purging unsafe.
+ *
+ * Return: true if BO is imported or exported, false otherwise
+ */drivers/gpu/drm/xe/xe_vm_madvise.c static void madvise_pat_index(struct xe_device *xe, struct xe_vm *vm, · line 194
Quoted 2 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+static bool xe_bo_is_external_dmabuf (struct xe_bo *bo) +{@Thomas
Quoted 8 lines
Quoted 4 lines
Should we have this check more generic? e.g., if a BO is not tied to
a
VM, we don't allow purablity to be set?I think if NEO is going to implement this in one form or another, we
need to support also external bos as long as they're not exported, since NEO refuses to use local bos.You're right, and I believe the current implementation should support that.
To clarify: "external" here refers only to dma-buf sharing (imported or
exported), not to placement. System-memory BOs that NEO typically uses
remain fully eligible for purging.The
xe_bo_is_dmabuf_shared()check specifically targets:
- Imported dma-bufs (we don't own backing store)
- Exported dma-bufs (external devices may have active mappings)
Quoted 7 lines
Regular Xe-owned system BOs can be marked DONTNEED and purged under memory
pressure. We only skip cases where the BO is shared via dma-buf, following
the same pattern asdrm_gem_shmem_is_purgeable().Please let me know if I've misunderstood the NEO requirements.
No it sounds right. We have
*local BOs, those that Matt refers to as tied to a VM. They are not sharable.
*external BOs. Those may or may not be exported as dma-bufs. NEO uses only these, and they need to be purgeable as well, as long as they are not exported or imported dma-bufs.
Thanks,
Thomas
Quoted 111 lines
Quoted 45 lines
Quoted 45 lines
Quoted 40 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +13
-0+struct drm_gem_object *obj = &bo->ttm.base; + +/* Imported from another driver */ +if (drm_gem_is_imported (obj)) +return true; + +/* Exported to another driver */ +if (obj->dma_buf) +return true; + +return false; +} +/**
*xe_bo_all_vmas_dontneed()- Check if all VMAs of a BO are marked DONTNEED
* @bo: Buffer objectdrivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -200,6 +225,10 @@ static bool xe_bo_all_vmas_dontneed(struct xe_bo *bo)
dma_resv_assert_held(bo->ttm.base.resv);drivers/gpu/drm/xe/xe_vm_madvise.c +4
-0+/* External dma-bufs cannot be purgeable */ +if (xe_bo_is_external_dmabuf (bo)) +return false; +
drm_gem_for_each_gpuvm_bo(vm_bo, obj){drm_gpuvm_bo_for_each_va(gpuva, vm_bo){ structxe_vma*vma =gpuva_to_vma(gpuva);drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -277,6 +306,10 @@ static boolxe_vm_madvise_purgeable_bo(struct
xe_device*xe, structxe_vm*vm,
/* BO must be locked before modifying madv state
*/
xe_bo_assert_held(bo);drivers/gpu/drm/xe/xe_vm_madvise.c static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm, · line 311
Quoted 3 lines
drivers/gpu/drm/xe/xe_vm_madvise.c +3
-0+/* Skip external dma-bufs */ +if (xe_bo_is_external_dmabuf (bo)) +continue ;@Thomas
Quoted 8 lines
Quoted 5 lines
I think instead of silently continuing here we fail the IOCTL with
-EINVAL?What do you think?
Hmm. If we view the export as yet another map, then IMO silently continuing would probably be the best option and consistent with sharing across VMs.
Noted, I'll keep the silent skip behavior for dma-buf shared BOs to match the "unanimous DONTNEED" model.
Quoted 4 lines
Finally I wonder if we should update the purgeable state on final dma-
buf attach.Good point. Currently the check only happens at madvise time, so there's
a window where a BO marked DONTNEED could later be exported. Would it make sense to hook into the export path to force transition to
WILLNEED? That would make the "export as implicit mapping" model complete,
though it adds complexity. Let me know your preference.Quoted 7 lines
Floating an idea here:
Could the code be simplified if we maintain a
map_countand apurgeable_counton each bo? Bo is purgeable iffmap_count==purgeable_count. vmas and dma-buf attachments are included inmap_count.That's good idea and would avoid VMA iteration. The tradeoff is maintaining correct counts across all VMA and dma-buf lifecycle paths.
For this series, would you prefer I pursue the counter-based refactor now, or is the current iteration approach acceptable
with counters as a future optimization? I'm happy to go either direction
based on your feedback.Thanks,
ArvindQuoted 18 lines
Thanks,
ThomasQuoted 13 lines
Matt
Quoted 11 lines
+
/*
* Once purged, always purged. Cannot transition
back to WILLNEED.
* This matches i915 semantics where purged BOs
are permanently invalid.Signature · 1 line
2.43.0
PATCH v4 8/8 drm/xe/bo: Add purgeable shrinker state helpers
In reply to Arvind YadavEncapsulate TTM purgeable flag updates and shrinker page accounting into helper functions. This prevents desynchronization between the TTM tt->purgeable flag and the shrinker's page bucket counters.
Without these helpers, direct manipulation of xe_ttm_tt->purgeable risks forgetting to update the corresponding shrinker counters, leading to incorrect memory pressure calculations.
Add xe_bo_set_purgeable_shrinker() and xe_bo_clear_purgeable_shrinker() which atomically update both the TTM flag and transfer pages between the shrinkable and purgeable buckets.
v4:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
--- drivers/gpu/drm/xe/xe_bo.c | 60 ++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_bo.h | 3 ++ drivers/gpu/drm/xe/xe_vm_madvise.c | 13 +++++-- 3 files changed, 73 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +60 -0
@@ -836,6 +836,66 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_bo_set_purgeable_shrinker() - Mark BO purgeable and update shrinker +* @bo: Buffer object +* +* Transfers pages from shrinkable to purgeable bucket. Shrinker can now +* discard pages immediately without swapping. Caller holds BO lock. +*/ +void xe_bo_set_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (!xe_tt->purgeable) { + xe_tt->purgeable = true; +/* Transfer pages from shrinkable to purgeable count */ +xe_shrinker_mod_pages (xe->mem.shrinker, + -(long )tt->num_pages, + tt->num_pages); + } +} + +/** +* xe_bo_clear_purgeable_shrinker() - Mark BO non-purgeable and update shrinker +* @bo: Buffer object +* +* Transfers pages from purgeable to shrinkable bucket. Shrinker must now +* swap pages instead of discarding. Caller holds BO lock. +*/ +void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (xe_tt->purgeable) { + xe_tt->purgeable = false; +/* Transfer pages from purgeable to shrinkable count */ +xe_shrinker_mod_pages (xe->mem.shrinker, + tt->num_pages, + -(long )tt->num_pages); + } +} +/** * xe_ttm_bo_purge() - Purge buffer object backing store * @ttm_bo: The TTM buffer object to purge
drivers/gpu/drm/xe/xe_bo.h +3 -0
@@ -270,6 +270,9 @@ static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo)return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; } +void xe_bo_set_purgeable_shrinker (struct xe_bo *bo); +void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo); +static inline void xe_bo_unpin_map_no_vm (struct xe_bo *bo) {if (likely (bo)) {
drivers/gpu/drm/xe/xe_vm_madvise.c +10 -3
@@ -274,12 +274,16 @@ void xe_bo_recheck_purgeable_on_vma_unbind(struct xe_bo *bo)if (xe_bo_all_vmas_dontneed (bo)) {/* All VMAs are DONTNEED - mark BO purgeable */ -if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) { bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; +xe_bo_set_purgeable_shrinker (bo); + } }else {/* At least one VMA is WILLNEED - BO must not be purgeable */ -if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) { bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +xe_bo_clear_purgeable_shrinker (bo); + } } }
@@ -325,13 +329,16 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,/* Mark VMA WILLNEED - BO becomes non-purgeable immediately */ bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +xe_bo_clear_purgeable_shrinker (bo);break ;case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : vmas[i]->purgeable_state =XE_MADV_PURGEABLE_DONTNEED ;/* Mark BO purgeable only if all VMAs are DONTNEED */ -if (xe_bo_all_vmas_dontneed (bo)) +if (xe_bo_all_vmas_dontneed (bo)) { bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; +xe_bo_set_purgeable_shrinker (bo); + }break ;default : drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",
Signature · 1 line
2.43.0
Matthew Brost
maintainerRe: [PATCH v4 8/8] drm/xe/bo: Add purgeable shrinker state helpers
Quoted 8 lines · Arvind Yadav
Encapsulate TTM purgeable flag updates and shrinker page accounting into helper functions. This prevents desynchronization between the TTM
tt->purgeableflag and the shrinker's page bucket counters.Without these helpers, direct manipulation of
xe_ttm_tt->purgeablerisks forgetting to update the corresponding shrinker counters, leading to incorrect memory pressure calculations.
Quoted 8 lines
Add
xe_bo_set_purgeable_shrinker()andxe_bo_clear_purgeable_shrinker()which atomically update both the TTM flag and transfer pages between the shrinkable and purgeable buckets.v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)Cc Matthew Brost <[email protected]>
I think this patch is right but best to double check with Thomas as he wrote the shrinker logic and is the expert here.
Matt
Quoted 138 lines
Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>--- drivers/gpu/drm/xe/xe_bo.c | 60 ++++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_bo.h | 3 ++ drivers/gpu/drm/xe/xe_vm_madvise.c | 13 +++++-- 3 files changed, 73 insertions(+), 3 deletions(-)drivers/gpu/drm/xe/xe_bo.c +60
-0@@ -836,6 +836,66 @@ static int xe_bo_move_notify(struct xe_bo *bo,return 0 ; } +/** +* xe_bo_set_purgeable_shrinker() - Mark BO purgeable and update shrinker +* @bo: Buffer object +* +* Transfers pages from shrinkable to purgeable bucket. Shrinker can now +* discard pages immediately without swapping. Caller holds BO lock. +*/ +void xe_bo_set_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (!xe_tt->purgeable) { + xe_tt->purgeable = true; +/* Transfer pages from shrinkable to purgeable count */ +xe_shrinker_mod_pages (xe->mem.shrinker, + -(long )tt->num_pages, + tt->num_pages); + } +} + +/** +* xe_bo_clear_purgeable_shrinker() - Mark BO non-purgeable and update shrinker +* @bo: Buffer object +* +* Transfers pages from purgeable to shrinkable bucket. Shrinker must now +* swap pages instead of discarding. Caller holds BO lock. +*/ +void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (xe_tt->purgeable) { + xe_tt->purgeable = false; +/* Transfer pages from purgeable to shrinkable count */ +xe_shrinker_mod_pages (xe->mem.shrinker, + tt->num_pages, + -(long )tt->num_pages); + } +} +/** * xe_ttm_bo_purge() - Purge buffer object backing store * @ttm_bo: The TTM buffer object to purge drivers/gpu/drm/xe/xe_bo.h +3
-0@@ -270,6 +270,9 @@ static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo)return bo->madv_purgeable ==XE_MADV_PURGEABLE_DONTNEED ; } +void xe_bo_set_purgeable_shrinker (struct xe_bo *bo); +void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo); +static inline void xe_bo_unpin_map_no_vm (struct xe_bo *bo) {if (likely (bo)) {drivers/gpu/drm/xe/xe_vm_madvise.c +10
-3@@ -274,12 +274,16 @@ void xe_bo_recheck_purgeable_on_vma_unbind(struct xe_bo *bo)if (xe_bo_all_vmas_dontneed (bo)) {/* All VMAs are DONTNEED - mark BO purgeable */ -if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_DONTNEED ) { bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; +xe_bo_set_purgeable_shrinker (bo); + } }else {/* At least one VMA is WILLNEED - BO must not be purgeable */ -if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) +if (bo->madv_purgeable !=XE_MADV_PURGEABLE_WILLNEED ) { bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +xe_bo_clear_purgeable_shrinker (bo); + } } }@@ -325,13 +329,16 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,/* Mark VMA WILLNEED - BO becomes non-purgeable immediately */ bo->madv_purgeable =XE_MADV_PURGEABLE_WILLNEED ; +xe_bo_clear_purgeable_shrinker (bo);break ;case DRM_XE_VMA_PURGEABLE_STATE_DONTNEED : vmas[i]->purgeable_state =XE_MADV_PURGEABLE_DONTNEED ;/* Mark BO purgeable only if all VMAs are DONTNEED */ -if (xe_bo_all_vmas_dontneed (bo)) +if (xe_bo_all_vmas_dontneed (bo)) { bo->madv_purgeable =XE_MADV_PURGEABLE_DONTNEED ; +xe_bo_set_purgeable_shrinker (bo); + }break ;default : drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",Signature · 1 line
2.43.0
Re: [PATCH v4 8/8] drm/xe/bo: Add purgeable shrinker state helpers
Quoted 19 lines · Matthew Brost
Quoted 18 lines · Arvind Yadav
Encapsulate TTM purgeable flag updates and shrinker page accounting into helper functions. This prevents desynchronization between the TTM
tt->purgeableflag and the shrinker's page bucket counters.Without these helpers, direct manipulation of
xe_ttm_tt->purgeablerisks forgetting to update the corresponding shrinker counters, leading to incorrect memory pressure calculations.Add
xe_bo_set_purgeable_shrinker()andxe_bo_clear_purgeable_shrinker()
which atomically update both the TTM flag and transfer pages between
the shrinkable and purgeable buckets.v4:
- @madv_purgeableatomic_t→ u32 change across all relevant patches. (Matt)
Quoted 7 lines
Quoted 1 line
Cc Matthew Brost <[email protected]>I think this patch is right but best to double check with Thomas as he
wrote the shrinker logic and is the expert here.Matt
Looks correct to me.
@Arvind: Did you look at removing the xe_tt purgeable field (which I always thought of as an intermediate until we had purgeable bos) and replace it with the bo->madv_purgeable field in the shrinker code. I'm not sure how complicated it would be, TBH but feel free to leave that as a follow-up patch.
Also make sure the xe_bo_shrink xe_live_ktest subtest still works since it relies heavily on shrinkable bos. I don't think CI runs it because of the execution time and insufficient memory allocation amount heuristics. We should also file a jira to rewrite that test to a user- space IGT test now that we support shrinkable bos. I'll take a look at that.
Quoted 166 lines
Quoted 166 lines
Cc Thomas Hellström <[email protected]>Cc Himal Prasad Ghimiray <[email protected]>Signed-off-by Arvind Yadav <[email protected]>---
drivers/gpu/drm/xe/xe_bo.c | 60
++++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_bo.h | 3 ++
drivers/gpu/drm/xe/xe_vm_madvise.c | 13 +++++--3 files changed, 73 insertions(+), 3 deletions(-)drivers/gpu/drm/xe/xe_bo.c +0
-0@@ -836,6 +836,66 @@ static int xe_bo_move_notify(struct xe_bo *bo, return 0; }
+/**
+ *xe_bo_set_purgeable_shrinker()- Mark BO purgeable and update shrinker
+ * @bo: Buffer object
+ *
+ * Transfers pages from shrinkable to purgeable bucket. Shrinker can now
+ * discard pages immediately without swapping. Caller holds BO lock.
+ */drivers/gpu/drm/xe/xe_bo.c +17
-0+void xe_bo_set_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (!xe_tt->purgeable) { + xe_tt->purgeable = true; +/* Transfer pages from shrinkable to purgeable count */
drivers/gpu/drm/xe/xe_bo.c +8
-0+xe_shrinker_mod_pages (xe->mem.shrinker, + -(long)tt->num_pages, + tt->num_pages); + } +} + +/** +* xe_bo_clear_purgeable_shrinker() - Mark BO non-purgeable and update shrinker
+ * @bo: Buffer object
+ *
+ * Transfers pages from purgeable to shrinkable bucket. Shrinker must now
+ * swap pages instead of discarding. Caller holds BO lock.
+ */drivers/gpu/drm/xe/xe_bo.c +17
-0+void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo) +{ +struct ttm_buffer_object *ttm_bo = &bo->ttm; +struct ttm_tt *tt = ttm_bo->ttm; +struct xe_device *xe =ttm_to_xe_device (ttm_bo->bdev); +struct xe_ttm_tt *xe_tt; + +xe_bo_assert_held (bo); + +if (!tt || !ttm_tt_is_populated (tt)) +return ; + + xe_tt = container_of(tt, struct xe_ttm_tt, ttm); + +if (xe_tt->purgeable) { + xe_tt->purgeable = false; +/* Transfer pages from purgeable to shrinkable count */
drivers/gpu/drm/xe/xe_bo.c +6
-0+xe_shrinker_mod_pages (xe->mem.shrinker, + tt->num_pages, + -(long)tt->num_pages); + } +} +/**
*xe_ttm_bo_purge()- Purge buffer object backing store
* @ttm_bo: The TTM buffer object to purgedrivers/gpu/drm/xe/xe_bo.h +0
-0@@ -270,6 +270,9 @@ static inline bool xe_bo_madv_is_dontneed(struct xe_bo *bo) return bo->madv_purgeable == XE_MADV_PURGEABLE_DONTNEED; }drivers/gpu/drm/xe/xe_bo.h +3
-0+void xe_bo_set_purgeable_shrinker (struct xe_bo *bo); +void xe_bo_clear_purgeable_shrinker (struct xe_bo *bo); +static inline void
xe_bo_unpin_map_no_vm(struct xe_bo *bo){
if (likely(bo)) {drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -274,12 +274,16 @@ void xe_bo_recheck_purgeable_on_vma_unbind(struct xe_bo *bo)
if (xe_bo_all_vmas_dontneed(bo)) {
/* All VMAs are DONTNEED - mark BO purgeable */drivers/gpu/drm/xe/xe_vm_madvise.c +0
-1- if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_DONTNEED)drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_DONTNEED) {
bo->madv_purgeable=XE_MADV_PURGEABLE_DONTNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+xe_bo_set_purgeable_shrinker (bo); + }} else {
/* At least one VMA is WILLNEED - BO must not be purgeable */drivers/gpu/drm/xe/xe_vm_madvise.c +0
-1- if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_WILLNEED)drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+ if (bo->madv_purgeable !=
XE_MADV_PURGEABLE_WILLNEED) {
bo->madv_purgeable=XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+xe_bo_clear_purgeable_shrinker (bo); + }}
}drivers/gpu/drm/xe/xe_vm_madvise.c +0
-0@@ -325,13 +329,16 @@ static bool xe_vm_madvise_purgeable_bo(struct xe_device *xe, struct xe_vm *vm,
/* Mark VMA WILLNEED - BO becomes non- purgeable immediately */
bo->madv_purgeable=XE_MADV_PURGEABLE_WILLNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +1
-0+xe_bo_clear_purgeable_shrinker (bo);break;
caseDRM_XE_VMA_PURGEABLE_STATE_DONTNEED:
vmas[i]->purgeable_state =XE_MADV_PURGEABLE_DONTNEED;
/* Mark BO purgeable only if all VMAs are DONTNEED */drivers/gpu/drm/xe/xe_vm_madvise.c +1
-1-if (xe_bo_all_vmas_dontneed (bo)) +if (xe_bo_all_vmas_dontneed (bo)) {
bo->madv_purgeable=XE_MADV_PURGEABLE_DONTNEED;drivers/gpu/drm/xe/xe_vm_madvise.c +2
-0+xe_bo_set_purgeable_shrinker (bo); + }break;
default:
drm_warn(&vm->xe->drm, "Invalid madvice value = %d\n",Signature · 1 line
2.43.0
Patchwork
CI✗ CI.checkpatch: warning for drm/xe/madvise: Add support for purgeable buffer objects (rev5)
In reply to Arvind Yadav== Series Details ==
== Series Details ==
Series: drm/xe/madvise: Add support for purgeable buffer objects (rev5)
URL : https://patchwork.freedesktop.org/series/156651/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master ee83616c430ce70bd254bd2774d143a5733c8666
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit b8e43aa90f01e0196366622cb23da4c031bb10bb
Author: Arvind Yadav <[email protected]> Date: Tue Jan 20 11:38:54 2026 +0530
drm/xe/bo: Add purgeable shrinker state helpers
Encapsulate TTM purgeable flag updates and shrinker page accounting into helper functions. This prevents desynchronization between the TTM tt->purgeable flag and the shrinker's page bucket counters.
Without these helpers, direct manipulation of xe_ttm_tt->purgeable risks forgetting to update the corresponding shrinker counters, leading to incorrect memory pressure calculations.
Add xe_bo_set_purgeable_shrinker() and xe_bo_clear_purgeable_shrinker() which atomically update both the TTM flag and transfer pages between the shrinkable and purgeable buckets.
v4:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
+ /mt/dim checkpatch 3319a649c1a1d4389f48c561537b9915b1789800 drm-intel6540a04e9e19 drm/xe/uapi: Add UAPI support for purgeable buffer objectsbd39a2b12369 drm/xe/bo: Add purgeable bo state tracking and field madv to xe_bo
-:37: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #37:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
total: 0 errors, 1 warnings, 0 checks, 77 lines checkedaa339e71bdc8 drm/xe/madvise: Implement purgeable buffer object support
-:23: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #23:
- Async TLB invalidation via trigger_rebind (no blocking xe_vm_invalidate_vma)
total: 0 errors, 1 warnings, 0 checks, 304 lines checked1b71ab7b8171 drm/xe/bo: Handle CPU faults on purged buffer objectseb146765e124 drm/xe/vm: Prevent binding of purged buffer objects277c31862829 drm/xe/madvise: Implement per-VMA purgeable state tracking
-:29: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #29:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
total: 0 errors, 1 warnings, 0 checks, 172 lines checked37e2d276b275 drm/xe/madvise: Block imported and exported dma-bufsb8e43aa90f01 drm/xe/bo: Add purgeable shrinker state helpers
-:22: WARNING:COMMIT_LOG_LONG_LINE: Prefer a maximum 75 chars per line (possible unwrapped commit description?) #22:
- @madv_purgeable atomic_t → u32 change across all relevant patches. (Matt)
total: 0 errors, 1 warnings, 0 checks, 110 lines checked
Patchwork
CI✗ CI.KUnit: failure for drm/xe/madvise: Add support for purgeable buffer objects (rev5)
In reply to Arvind Yadav== Series Details ==
== Series Details ==
Series: drm/xe/madvise: Add support for purgeable buffer objects (rev5)
URL : https://patchwork.freedesktop.org/series/156651/
State : failure
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig [06:14:58] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig [06:15:02] Building KUnit Kernel ... Populating config with: $ make ARCH=um O=.kunit olddefconfig Building with: $ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48 [06:15:34] Starting KUnit Kernel (1/1)... [06:15:34] ============================================================ Running tests with: $ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt [06:15:34] ================== guc_buf (11 subtests) =================== [06:15:34] [PASSED] test_smallest [06:15:34] [PASSED] test_largest [06:15:34] [PASSED] test_granular [06:15:34] [PASSED] test_unique [06:15:34] [PASSED] test_overlap [06:15:34] [PASSED] test_reusable [06:15:34] [PASSED] test_too_big [06:15:34] [PASSED] test_flush [06:15:34] [PASSED] test_lookup [06:15:34] [PASSED] test_data [06:15:34] [PASSED] test_class [06:15:34] ===================== [PASSED] guc_buf ===================== [06:15:34] =================== guc_dbm (7 subtests) =================== [06:15:34] [PASSED] test_empty [06:15:34] [PASSED] test_default [06:15:34] ======================== test_size ======================== [06:15:34] [PASSED] 4 [06:15:34] [PASSED] 8 [06:15:34] [PASSED] 32 [06:15:34] [PASSED] 256 [06:15:34] ==================== [PASSED] test_size ==================== [06:15:34] ======================= test_reuse ======================== [06:15:34] [PASSED] 4 [06:15:34] [PASSED] 8 [06:15:34] [PASSED] 32 [06:15:34] [PASSED] 256 [06:15:34] =================== [PASSED] test_reuse ==================== [06:15:34] =================== test_range_overlap ==================== [06:15:34] [PASSED] 4 [06:15:34] [PASSED] 8 [06:15:34] [PASSED] 32 [06:15:34] [PASSED] 256 [06:15:34] =============== [PASSED] test_range_overlap ================ [06:15:34] =================== test_range_compact ==================== [06:15:34] [PASSED] 4 [06:15:34] [PASSED] 8 [06:15:34] [PASSED] 32 [06:15:34] [PASSED] 256 [06:15:34] =============== [PASSED] test_range_compact ================ [06:15:34] ==================== test_range_spare ===================== [06:15:34] [PASSED] 4 [06:15:34] [PASSED] 8 [06:15:34] [PASSED] 32 [06:15:34] [PASSED] 256 [06:15:34] ================ [PASSED] test_range_spare ================= [06:15:34] ===================== [PASSED] guc_dbm ===================== [06:15:34] =================== guc_idm (6 subtests) =================== [06:15:34] [PASSED] bad_init [06:15:34] [PASSED] no_init [06:15:34] [PASSED] init_fini [06:15:34] [PASSED] check_used [06:15:34] [PASSED] check_quota [06:15:34] [PASSED] check_all [06:15:34] ===================== [PASSED] guc_idm ===================== [06:15:34] ================== no_relay (3 subtests) =================== [06:15:34] [PASSED] xe_drops_guc2pf_if_not_ready [06:15:34] [PASSED] xe_drops_guc2vf_if_not_ready [06:15:34] [PASSED] xe_rejects_send_if_not_ready [06:15:34] ==================== [PASSED] no_relay ===================== [06:15:34] ================== pf_relay (14 subtests) ================== [06:15:34] [PASSED] pf_rejects_guc2pf_too_short [06:15:34] [PASSED] pf_rejects_guc2pf_too_long [06:15:34] [PASSED] pf_rejects_guc2pf_no_payload [06:15:34] [PASSED] pf_fails_no_payload [06:15:34] [PASSED] pf_fails_bad_origin [06:15:34] [PASSED] pf_fails_bad_type [06:15:34] [PASSED] pf_txn_reports_error [06:15:34] [PASSED] pf_txn_sends_pf2guc [06:15:34] [PASSED] pf_sends_pf2guc [06:15:34] [SKIPPED] pf_loopback_nop [06:15:34] [SKIPPED] pf_loopback_echo [06:15:34] [SKIPPED] pf_loopback_fail [06:15:34] [SKIPPED] pf_loopback_busy [06:15:34] [SKIPPED] pf_loopback_retry [06:15:34] ==================== [PASSED] pf_relay ===================== [06:15:34] ================== vf_relay (3 subtests) =================== [06:15:34] [PASSED] vf_rejects_guc2vf_too_short [06:15:34] [PASSED] vf_rejects_guc2vf_too_long [06:15:34] [PASSED] vf_rejects_guc2vf_no_payload [06:15:34] ==================== [PASSED] vf_relay ===================== [06:15:34] ================ pf_gt_config (6 subtests) ================= [06:15:34] [PASSED] fair_contexts_1vf [06:15:34] [PASSED] fair_doorbells_1vf [06:15:34] [PASSED] fair_ggtt_1vf [06:15:34] ====================== fair_contexts ====================== [06:15:34] [PASSED] 1 VF [06:15:34] [PASSED] 2 VFs [06:15:34] [PASSED] 3 VFs [06:15:34] [PASSED] 4 VFs [06:15:34] [PASSED] 5 VFs [06:15:34] [PASSED] 6 VFs [06:15:34] [PASSED] 7 VFs [06:15:34] [PASSED] 8 VFs [06:15:34] [PASSED] 9 VFs [06:15:34] [PASSED] 10 VFs [06:15:34] [PASSED] 11 VFs [06:15:34] [PASSED] 12 VFs [06:15:34] [PASSED] 13 VFs [06:15:34] [PASSED] 14 VFs [06:15:34] [PASSED] 15 VFs [06:15:34] [PASSED] 16 VFs [06:15:34] [PASSED] 17 VFs [06:15:34] [PASSED] 18 VFs [06:15:34] [PASSED] 19 VFs [06:15:34] [PASSED] 20 VFs [06:15:34] [PASSED] 21 VFs [06:15:34] [PASSED] 22 VFs [06:15:34] [PASSED] 23 VFs [06:15:34] [PASSED] 24 VFs [06:15:34] [PASSED] 25 VFs [06:15:34] [PASSED] 26 VFs [06:15:34] [PASSED] 27 VFs [06:15:34] [PASSED] 28 VFs [06:15:34] [PASSED] 29 VFs [06:15:34] [PASSED] 30 VFs [06:15:34] [PASSED] 31 VFs [06:15:34] [PASSED] 32 VFs [06:15:34] [PASSED] 33 VFs [06:15:34] [PASSED] 34 VFs [06:15:34] [PASSED] 35 VFs [06:15:34] [PASSED] 36 VFs [06:15:34] [PASSED] 37 VFs [06:15:34] [PASSED] 38 VFs [06:15:34] [PASSED] 39 VFs [06:15:34] [PASSED] 40 VFs [06:15:34] [PASSED] 41 VFs [06:15:34] [PASSED] 42 VFs [06:15:34] [PASSED] 43 VFs [06:15:34] [PASSED] 44 VFs [06:15:34] [PASSED] 45 VFs [06:15:34] [PASSED] 46 VFs [06:15:34] [PASSED] 47 VFs [06:15:34] [PASSED] 48 VFs [06:15:34] [PASSED] 49 VFs [06:15:34] [PASSED] 50 VFs [06:15:34] [PASSED] 51 VFs [06:15:34] [PASSED] 52 VFs [06:15:34] [PASSED] 53 VFs [06:15:34] [PASSED] 54 VFs [06:15:34] [PASSED] 55 VFs [06:15:34] [PASSED] 56 VFs [06:15:34] [PASSED] 57 VFs [06:15:34] [PASSED] 58 VFs [06:15:34] [PASSED] 59 VFs [06:15:34] [PASSED] 60 VFs [06:15:34] [PASSED] 61 VFs [06:15:34] [PASSED] 62 VFs [06:15:34] [PASSED] 63 VFs [06:15:34] ================== [PASSED] fair_contexts ================== [06:15:34] ===================== fair_doorbells ====================== [06:15:34] [PASSED] 1 VF [06:15:34] [PASSED] 2 VFs [06:15:34] [PASSED] 3 VFs [06:15:34] [PASSED] 4 VFs [06:15:34] [PASSED] 5 VFs [06:15:34] [PASSED] 6 VFs [06:15:34] [PASSED] 7 VFs [06:15:34] [PASSED] 8 VFs [06:15:34] [PASSED] 9 VFs [06:15:34] [PASSED] 10 VFs [06:15:34] [PASSED] 11 VFs [06:15:34] [PASSED] 12 VFs [06:15:34] [PASSED] 13 VFs [06:15:34] [PASSED] 14 VFs [06:15:34] [PASSED] 15 VFs [06:15:34] [PASSED] 16 VFs [06:15:34] [PASSED] 17 VFs [06:15:34] [PASSED] 18 VFs [06:15:34] [PASSED] 19 VFs [06:15:34] [PASSED] 20 VFs [06:15:34] [PASSED] 21 VFs [06:15:34] [PASSED] 22 VFs [06:15:34] [PASSED] 23 VFs [06:15:34] [PASSED] 24 VFs [06:15:34] [PASSED] 25 VFs [06:15:34] [PASSED] 26 VFs [06:15:34] [PASSED] 27 VFs [06:15:34] [PASSED] 28 VFs [06:15:34] [PASSED] 29 VFs [06:15:34] [PASSED] 30 VFs [06:15:34] [PASSED] 31 VFs [06:15:34] [PASSED] 32 VFs [06:15:34] [PASSED] 33 VFs [06:15:34] [PASSED] 34 VFs [06:15:34] [PASSED] 35 VFs [06:15:34] [PASSED] 36 VFs [06:15:34] [PASSED] 37 VFs [06:15:34] [PASSED] 38 VFs [06:15:34] [PASSED] 39 VFs [06:15:34] [PASSED] 40 VFs [06:15:34] [PASSED] 41 VFs [06:15:34] [PASSED] 42 VFs [06:15:34] [PASSED] 43 VFs [06:15:34] [PASSED] 44 VFs [06:15:34] [PASSED] 45 VFs [06:15:34] [PASSED] 46 VFs [06:15:34] [PASSED] 47 VFs [06:15:34] [PASSED] 48 VFs [06:15:34] [PASSED] 49 VFs [06:15:34] [PASSED] 50 VFs [06:15:34] [PASSED] 51 VFs [06:15:34] [PASSED] 52 VFs [06:15:34] [PASSED] 53 VFs [06:15:34] [PASSED] 54 VFs [06:15:34] [PASSED] 55 VFs [06:15:34] [PASSED] 56 VFs [06:15:34] [PASSED] 57 VFs [06:15:34] [PASSED] 58 VFs [06:15:34] [PASSED] 59 VFs [06:15:34] [PASSED] 60 VFs [06:15:34] [PASSED] 61 VFs [06:15:34] [PASSED] 62 VFs [06:15:34] [PASSED] 63 VFs [06:15:34] ================= [PASSED] fair_doorbells ================== [06:15:34] ======================== fair_ggtt ======================== [06:15:34] [PASSED] 1 VF [06:15:34] [PASSED] 2 VFs [06:15:34] [PASSED] 3 VFs [06:15:34] [PASSED] 4 VFs [06:15:34] [PASSED] 5 VFs [06:15:34] [PASSED] 6 VFs [06:15:34] [PASSED] 7 VFs [06:15:34] [PASSED] 8 VFs [06:15:34] [PASSED] 9 VFs [06:15:34] [PASSED] 10 VFs [06:15:34] [PASSED] 11 VFs [06:15:34] [PASSED] 12 VFs [06:15:34] [PASSED] 13 VFs [06:15:34] [PASSED] 14 VFs [06:15:34] [PASSED] 15 VFs [06:15:34] [PASSED] 16 VFs [06:15:34] [PASSED] 17 VFs [06:15:34] [PASSED] 18 VFs [06:15:34] [PASSED] 19 VFs [06:15:34] [PASSED] 20 VFs [06:15:34] [PASSED] 21 VFs [06:15:34] [PASSED] 22 VFs [06:15:34] [PASSED] 23 VFs [06:15:34] [PASSED] 24 VFs [06:15:34] [PASSED] 25 VFs [06:15:34] [PASSED] 26 VFs [06:15:34] [PASSED] 27 VFs [06:15:34] [PASSED] 28 VFs [06:15:34] [PASSED] 29 VFs [06:15:34] [PASSED] 30 VFs [06:15:34] [PASSED] 31 VFs [06:15:34] [PASSED] 32 VFs [06:15:34] [PASSED] 33 VFs [06:15:34] [PASSED] 34 VFs [06:15:34] [PASSED] 35 VFs [06:15:34] [PASSED] 36 VFs [06:15:34] [PASSED] 37 VFs [06:15:34] [PASSED] 38 VFs [06:15:34] [PASSED] 39 VFs [06:15:34] [PASSED] 40 VFs [06:15:34] [PASSED] 41 VFs [06:15:34] [PASSED] 42 VFs [06:15:34] [PASSED] 43 VFs [06:15:34] [PASSED] 44 VFs [06:15:34] [PASSED] 45 VFs [06:15:34] [PASSED] 46 VFs [06:15:34] [PASSED] 47 VFs [06:15:34] [PASSED] 48 VFs [06:15:34] [PASSED] 49 VFs [06:15:34] [PASSED] 50 VFs [06:15:34] [PASSED] 51 VFs [06:15:34] [PASSED] 52 VFs [06:15:34] [PASSED] 53 VFs [06:15:34] [PASSED] 54 VFs [06:15:34] [PASSED] 55 VFs [06:15:34] [PASSED] 56 VFs [06:15:34] [PASSED] 57 VFs [06:15:34] [PASSED] 58 VFs [06:15:34] [PASSED] 59 VFs [06:15:34] [PASSED] 60 VFs [06:15:34] [PASSED] 61 VFs [06:15:34] [PASSED] 62 VFs [06:15:34] [PASSED] 63 VFs [06:15:34] ==================== [PASSED] fair_ggtt ==================== [06:15:34] ================== [PASSED] pf_gt_config =================== [06:15:34] ===================== lmtt (1 subtest) ===================== [06:15:34] ======================== test_ops ========================= [06:15:34] [PASSED] 2-level [06:15:34] [PASSED] multi-level [06:15:34] ==================== [PASSED] test_ops ===================== [06:15:34] ====================== [PASSED] lmtt ======================= [06:15:34] ================= pf_service (11 subtests) ================= [06:15:34] [PASSED] pf_negotiate_any [06:15:34] [PASSED] pf_negotiate_base_match [06:15:34] [PASSED] pf_negotiate_base_newer [06:15:34] [PASSED] pf_negotiate_base_next [06:15:34] [SKIPPED] pf_negotiate_base_older [06:15:34] [PASSED] pf_negotiate_base_prev [06:15:34] [PASSED] pf_negotiate_latest_match [06:15:34] [PASSED] pf_negotiate_latest_newer [06:15:34] [PASSED] pf_negotiate_latest_next [06:15:34] [SKIPPED] pf_negotiate_latest_older [06:15:34] [SKIPPED] pf_negotiate_latest_prev [06:15:34] =================== [PASSED] pf_service ==================== [06:15:34] ================= xe_guc_g2g (2 subtests) ================== [06:15:34] ============== xe_live_guc_g2g_kunit_default ============== [06:15:34] ========= [SKIPPED] xe_live_guc_g2g_kunit_default ========== [06:15:34] ============== xe_live_guc_g2g_kunit_allmem =============== [06:15:34] ========== [SKIPPED] xe_live_guc_g2g_kunit_allmem ========== [06:15:34] =================== [SKIPPED] xe_guc_g2g =================== [06:15:34] =================== xe_mocs (2 subtests) =================== [06:15:34] ================ xe_live_mocs_kernel_kunit ================ [06:15:34] =========== [SKIPPED] xe_live_mocs_kernel_kunit ============ [06:15:34] ================ xe_live_mocs_reset_kunit ================= [06:15:34] ============ [SKIPPED] xe_live_mocs_reset_kunit ============ [06:15:34] ==================== [SKIPPED] xe_mocs ===================== [06:15:34] ================= xe_migrate (2 subtests) ================== [06:15:34] ================= xe_migrate_sanity_kunit ================= [06:15:34] ============ [SKIPPED] xe_migrate_sanity_kunit ============= [06:15:34] ================== xe_validate_ccs_kunit ================== [06:15:34] ============= [SKIPPED] xe_validate_ccs_kunit ============== [06:15:34] =================== [SKIPPED] xe_migrate =================== [06:15:34] ================== xe_dma_buf (1 subtest) ================== [06:15:34] ==================== xe_dma_buf_kunit ===================== [06:15:34] ================ [SKIPPED] xe_dma_buf_kunit ================ [06:15:34] =================== [SKIPPED] xe_dma_buf =================== [06:15:34] ================= xe_bo_shrink (1 subtest) ================= [06:15:34] =================== xe_bo_shrink_kunit ==================== [06:15:34] =============== [SKIPPED] xe_bo_shrink_kunit =============== [06:15:34] ================== [SKIPPED] xe_bo_shrink ================== [06:15:34] ==================== xe_bo (2 subtests) ==================== [06:15:34] ================== xe_ccs_migrate_kunit =================== [06:15:34] ============== [SKIPPED] xe_ccs_migrate_kunit ============== [06:15:34] ==================== xe_bo_evict_kunit ==================== [06:15:34] =============== [SKIPPED] xe_bo_evict_kunit ================ [06:15:34] ===================== [SKIPPED] xe_bo ====================== [06:15:34] ==================== args (13 subtests) ==================== [06:15:34] [PASSED] count_args_test [06:15:34] [PASSED] call_args_example [06:15:34] [PASSED] call_args_test [06:15:34] [PASSED] drop_first_arg_example [06:15:34] [PASSED] drop_first_arg_test [06:15:34] [PASSED] first_arg_example [06:15:34] [PASSED] first_arg_test [06:15:34] [PASSED] last_arg_example [06:15:34] [PASSED] last_arg_test [06:15:34] [PASSED] pick_arg_example [06:15:34] [PASSED] if_args_example [06:15:34] [PASSED] if_args_test [06:15:34] [PASSED] sep_comma_example [06:15:34] ====================== [PASSED] args ======================= [06:15:34] =================== xe_pci (3 subtests) ==================== [06:15:34] ==================== check_graphics_ip ==================== [06:15:34] [PASSED] 12.00 Xe_LP [06:15:34] [PASSED] 12.10 Xe_LP+ [06:15:34] [PASSED] 12.55 Xe_HPG [06:15:34] [PASSED] 12.60 Xe_HPC [06:15:34] [PASSED] 12.70 Xe_LPG [06:15:34] [PASSED] 12.71 Xe_LPG [06:15:34] [PASSED] 12.74 Xe_LPG+ [06:15:34] [PASSED] 20.01 Xe2_HPG [06:15:34] [PASSED] 20.02 Xe2_HPG [06:15:34] [PASSED] 20.04 Xe2_LPG [06:15:34] [PASSED] 30.00 Xe3_LPG [06:15:34] [PASSED] 30.01 Xe3_LPG [06:15:34] [PASSED] 30.03 Xe3_LPG [06:15:34] [PASSED] 30.04 Xe3_LPG [06:15:34] [PASSED] 30.05 Xe3_LPG [06:15:34] [PASSED] 35.11 Xe3p_XPC [06:15:34] ================ [PASSED] check_graphics_ip ================ [06:15:34] ===================== check_media_ip ====================== [06:15:34] [PASSED] 12.00 Xe_M [06:15:34] [PASSED] 12.55 Xe_HPM [06:15:34] [PASSED] 13.00 Xe_LPM+ [06:15:34] [PASSED] 13.01 Xe2_HPM [06:15:34] [PASSED] 20.00 Xe2_LPM [06:15:34] [PASSED] 30.00 Xe3_LPM [06:15:34] [PASSED] 30.02 Xe3_LPM [06:15:34] [PASSED] 35.00 Xe3p_LPM [06:15:34] [PASSED] 35.03 Xe3p_HPM [06:15:34] ================= [PASSED] check_media_ip ================== [06:15:34] =================== check_platform_desc =================== [06:15:34] [PASSED] 0x9A60 (TIGERLAKE) [06:15:34] [PASSED] 0x9A68 (TIGERLAKE) [06:15:34] [PASSED] 0x9A70 (TIGERLAKE) [06:15:34] [PASSED] 0x9A40 (TIGERLAKE) [06:15:34] [PASSED] 0x9A49 (TIGERLAKE) [06:15:34] [PASSED] 0x9A59 (TIGERLAKE) [06:15:34] [PASSED] 0x9A78 (TIGERLAKE) [06:15:34] [PASSED] 0x9AC0 (TIGERLAKE) [06:15:34] [PASSED] 0x9AC9 (TIGERLAKE) [06:15:34] [PASSED] 0x9AD9 (TIGERLAKE) [06:15:34] [PASSED] 0x9AF8 (TIGERLAKE) [06:15:34] [PASSED] 0x4C80 (ROCKETLAKE) [06:15:34] [PASSED] 0x4C8A (ROCKETLAKE) [06:15:34] [PASSED] 0x4C8B (ROCKETLAKE) [06:15:34] [PASSED] 0x4C8C (ROCKETLAKE) [06:15:34] [PASSED] 0x4C90 (ROCKETLAKE) [06:15:34] [PASSED] 0x4C9A (ROCKETLAKE) [06:15:34] [PASSED] 0x4680 (ALDERLAKE_S) [06:15:34] [PASSED] 0x4682 (ALDERLAKE_S) [06:15:34] [PASSED] 0x4688 (ALDERLAKE_S) [06:15:34] [PASSED] 0x468A (ALDERLAKE_S) [06:15:34] [PASSED] 0x468B (ALDERLAKE_S) [06:15:34] [PASSED] 0x4690 (ALDERLAKE_S) [06:15:34] [PASSED] 0x4692 (ALDERLAKE_S) [06:15:34] [PASSED] 0x4693 (ALDERLAKE_S) [06:15:34] [PASSED] 0x46A0 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46A1 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46A2 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46A3 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46A6 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46A8 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46AA (ALDERLAKE_P) [06:15:34] [PASSED] 0x462A (ALDERLAKE_P) [06:15:34] [PASSED] 0x4626 (ALDERLAKE_P) [06:15:34] [PASSED] 0x4628 (ALDERLAKE_P) stty: 'standard input': Inappropriate ioctl for device [06:15:34] [PASSED] 0x46B0 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46B1 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46B2 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46B3 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46C0 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46C1 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46C2 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46C3 (ALDERLAKE_P) [06:15:34] [PASSED] 0x46D0 (ALDERLAKE_N) [06:15:34] [PASSED] 0x46D1 (ALDERLAKE_N) [06:15:34] [PASSED] 0x46D2 (ALDERLAKE_N) [06:15:34] [PASSED] 0x46D3 (ALDERLAKE_N) [06:15:34] [PASSED] 0x46D4 (ALDERLAKE_N) [06:15:34] [PASSED] 0xA721 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7A1 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7A9 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7AC (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7AD (ALDERLAKE_P) [06:15:34] [PASSED] 0xA720 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7A0 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7A8 (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7AA (ALDERLAKE_P) [06:15:34] [PASSED] 0xA7AB (ALDERLAKE_P) [06:15:34] [PASSED] 0xA780 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA781 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA782 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA783 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA788 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA789 (ALDERLAKE_S) [06:15:34] [PASSED] 0xA78A (ALDERLAKE_S) [06:15:34] [PASSED] 0xA78B (ALDERLAKE_S) [06:15:34] [PASSED] 0x4905 (DG1) [06:15:34] [PASSED] 0x4906 (DG1) [06:15:34] [PASSED] 0x4907 (DG1) [06:15:34] [PASSED] 0x4908 (DG1) [06:15:34] [PASSED] 0x4909 (DG1) [06:15:34] [PASSED] 0x56C0 (DG2) [06:15:34] [PASSED] 0x56C2 (DG2) [06:15:34] [PASSED] 0x56C1 (DG2) [06:15:34] [PASSED] 0x7D51 (METEORLAKE) [06:15:34] [PASSED] 0x7DD1 (METEORLAKE) [06:15:34] [PASSED] 0x7D41 (METEORLAKE) [06:15:34] [PASSED] 0x7D67 (METEORLAKE) [06:15:34] [PASSED] 0xB640 (METEORLAKE) [06:15:34] [PASSED] 0x56A0 (DG2) [06:15:34] [PASSED] 0x56A1 (DG2) [06:15:34] [PASSED] 0x56A2 (DG2) [06:15:34] [PASSED] 0x56BE (DG2) [06:15:34] [PASSED] 0x56BF (DG2) [06:15:34] [PASSED] 0x5690 (DG2) [06:15:34] [PASSED] 0x5691 (DG2) [06:15:34] [PASSED] 0x5692 (DG2) [06:15:34] [PASSED] 0x56A5 (DG2) [06:15:34] [PASSED] 0x56A6 (DG2) [06:15:34] [PASSED] 0x56B0 (DG2) [06:15:34] [PASSED] 0x56B1 (DG2) [06:15:34] [PASSED] 0x56BA (DG2) [06:15:34] [PASSED] 0x56BB (DG2) [06:15:34] [PASSED] 0x56BC (DG2) [06:15:34] [PASSED] 0x56BD (DG2) [06:15:34] [PASSED] 0x5693 (DG2) [06:15:34] [PASSED] 0x5694 (DG2) [06:15:34] [PASSED] 0x5695 (DG2) [06:15:34] [PASSED] 0x56A3 (DG2) [06:15:34] [PASSED] 0x56A4 (DG2) [06:15:34] [PASSED] 0x56B2 (DG2) [06:15:34] [PASSED] 0x56B3 (DG2) [06:15:34] [PASSED] 0x5696 (DG2) [06:15:34] [PASSED] 0x5697 (DG2) [06:15:34] [PASSED] 0xB69 (PVC) [06:15:34] [PASSED] 0xB6E (PVC) [06:15:34] [PASSED] 0xBD4 (PVC) [06:15:34] [PASSED] 0xBD5 (PVC) [06:15:34] [PASSED] 0xBD6 (PVC) [06:15:34] [PASSED] 0xBD7 (PVC) [06:15:34] [PASSED] 0xBD8 (PVC) [06:15:34] [PASSED] 0xBD9 (PVC) [06:15:34] [PASSED] 0xBDA (PVC) [06:15:34] [PASSED] 0xBDB (PVC) [06:15:34] [PASSED] 0xBE0 (PVC) [06:15:34] [PASSED] 0xBE1 (PVC) [06:15:34] [PASSED] 0xBE5 (PVC) [06:15:34] [PASSED] 0x7D40 (METEORLAKE) [06:15:34] [PASSED] 0x7D45 (METEORLAKE) [06:15:34] [PASSED] 0x7D55 (METEORLAKE) [06:15:34] [PASSED] 0x7D60 (METEORLAKE) [06:15:34] [PASSED] 0x7DD5 (METEORLAKE) [06:15:34] [PASSED] 0x6420 (LUNARLAKE) [06:15:34] [PASSED] 0x64A0 (LUNARLAKE) [06:15:34] [PASSED] 0x64B0 (LUNARLAKE) [06:15:34] [PASSED] 0xE202 (BATTLEMAGE) [06:15:34] [PASSED] 0xE209 (BATTLEMAGE) [06:15:34] [PASSED] 0xE20B (BATTLEMAGE) [06:15:34] [PASSED] 0xE20C (BATTLEMAGE) [06:15:34] [PASSED] 0xE20D (BATTLEMAGE) [06:15:34] [PASSED] 0xE210 (BATTLEMAGE) [06:15:34] [PASSED] 0xE211 (BATTLEMAGE) [06:15:34] [PASSED] 0xE212 (BATTLEMAGE) [06:15:34] [PASSED] 0xE216 (BATTLEMAGE) [06:15:34] [PASSED] 0xE220 (BATTLEMAGE) [06:15:34] [PASSED] 0xE221 (BATTLEMAGE) [06:15:34] [PASSED] 0xE222 (BATTLEMAGE) [06:15:34] [PASSED] 0xE223 (BATTLEMAGE) [06:15:34] [PASSED] 0xB080 (PANTHERLAKE) [06:15:34] [PASSED] 0xB081 (PANTHERLAKE) [06:15:34] [PASSED] 0xB082 (PANTHERLAKE) [06:15:34] [PASSED] 0xB083 (PANTHERLAKE) [06:15:34] [PASSED] 0xB084 (PANTHERLAKE) [06:15:34] [PASSED] 0xB085 (PANTHERLAKE) [06:15:34] [PASSED] 0xB086 (PANTHERLAKE) [06:15:34] [PASSED] 0xB087 (PANTHERLAKE) [06:15:34] [PASSED] 0xB08F (PANTHERLAKE) [06:15:34] [PASSED] 0xB090 (PANTHERLAKE) [06:15:34] [PASSED] 0xB0A0 (PANTHERLAKE) [06:15:34] [PASSED] 0xB0B0 (PANTHERLAKE) [06:15:34] [PASSED] 0xFD80 (PANTHERLAKE) [06:15:34] [PASSED] 0xFD81 (PANTHERLAKE) [06:15:34] [PASSED] 0xD740 (NOVALAKE_S) [06:15:34] [PASSED] 0xD741 (NOVALAKE_S) [06:15:34] [PASSED] 0xD742 (NOVALAKE_S) [06:15:34] [PASSED] 0xD743 (NOVALAKE_S) [06:15:34] [PASSED] 0xD744 (NOVALAKE_S) [06:15:34] [PASSED] 0xD745 (NOVALAKE_S) [06:15:34] [PASSED] 0x674C (CRESCENTISLAND) [06:15:34] =============== [PASSED] check_platform_desc =============== [06:15:34] ===================== [PASSED] xe_pci ====================== [06:15:34] =================== xe_rtp (2 subtests) ==================== [06:15:34] =============== xe_rtp_process_to_sr_tests ================ [06:15:34] [PASSED] coalesce-same-reg [06:15:34] [PASSED] no-match-no-add [06:15:34] [PASSED] match-or [06:15:34] [PASSED] match-or-xfail [06:15:34] [PASSED] no-match-no-add-multiple-rules [06:15:34] [PASSED] two-regs-two-entries [06:15:34] [PASSED] clr-one-set-other [06:15:34] [PASSED] set-field [06:15:34] [PASSED] conflict-duplicate [06:15:34] [PASSED] conflict-not-disjoint [06:15:34] [PASSED] conflict-reg-type [06:15:34] =========== [PASSED] xe_rtp_process_to_sr_tests ============ [06:15:34] ================== xe_rtp_process_tests =================== [06:15:34] [PASSED] active1 [06:15:34] [PASSED] active2 [06:15:34] [PASSED] active-inactive [06:15:34] [PASSED] inactive-active [06:15:34] [PASSED] inactive-1st_or_active-inactive [06:15:34] [PASSED] inactive-2nd_or_active-inactive [06:15:34] [PASSED] inactive-last_or_active-inactive [06:15:34] [PASSED] inactive-no_or_active-inactive [06:15:34] ============== [PASSED] xe_rtp_process_tests =============== [06:15:34] ===================== [PASSED] xe_rtp ====================== [06:15:34] ==================== xe_wa (1 subtest) ===================== [06:15:34] ======================== xe_wa_gt ========================= [06:15:34] [PASSED] TIGERLAKE B0 [06:15:34] [PASSED] DG1 A0 [06:15:34] [PASSED] DG1 B0 [06:15:34] [PASSED] ALDERLAKE_S A0 [06:15:34] [PASSED] ALDERLAKE_S B0 [06:15:34] [PASSED] ALDERLAKE_S C0 [06:15:34] [PASSED] ALDERLAKE_S D0 [06:15:34] [PASSED] ALDERLAKE_P A0 [06:15:34] [PASSED] ALDERLAKE_P B0 [06:15:34] [PASSED] ALDERLAKE_P C0 [06:15:34] [PASSED] ALDERLAKE_S RPLS D0 [06:15:34] [PASSED] ALDERLAKE_P RPLU E0 [06:15:34] [PASSED] DG2 G10 C0 [06:15:34] [PASSED] DG2 G11 B1 [06:15:34] [PASSED] DG2 G12 A1 [06:15:34] [PASSED] METEORLAKE 12.70(Xe_LPG) A0 13.00(Xe_LPM+) A0 [06:15:34] [PASSED] METEORLAKE 12.71(Xe_LPG) A0 13.00(Xe_LPM+) A0 [06:15:34] [PASSED] METEORLAKE 12.74(Xe_LPG+) A0 13.00(Xe_LPM+) A0 [06:15:34] [PASSED] LUNARLAKE 20.04(Xe2_LPG) A0 20.00(Xe2_LPM) A0 [06:15:34] [PASSED] LUNARLAKE 20.04(Xe2_LPG) B0 20.00(Xe2_LPM) A0 [06:15:34] [PASSED] BATTLEMAGE 20.01(Xe2_HPG) A0 13.01(Xe2_HPM) A1 [06:15:34] [PASSED] PANTHERLAKE 30.00(Xe3_LPG) A0 30.00(Xe3_LPM) A0 [06:15:34] ==================== [PASSED] xe_wa_gt ===================== [06:15:34] ====================== [PASSED] xe_wa ====================== [06:15:34] ============================================================ [06:15:34] Testing complete. Ran 512 tests: passed: 494, skipped: 18 [06:15:34] Elapsed time: 36.276s total, 4.183s configuring, 31.575s building, 0.466s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig [06:15:34] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[06:15:36] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48
ERROR:root:../drivers/gpu/drm/tests/drm_hdmi_state_helper_test.c: In function ‘drm_test_check_reject_hdr_infoframe_bpc_10’:
../drivers/gpu/drm/tests/drm_hdmi_state_helper_test.c:2926:49: warning: passing argument 6 of ‘drm_property_replace_blob_from_id’ makes integer from pointer without a cast [-Wint-conversion]
2926 | &replaced);
| ^~~~~~~~~
| |
| bool * {aka _Bool *}
In file included from ../include/drm/drm_connector.h:33,
from ../include/drm/drm_modes.h:33,
from ../include/drm/drm_crtc.h:32,
from ../include/drm/drm_atomic.h:31,
from ../drivers/gpu/drm/tests/drm_hdmi_state_helper_test.c:7:
../include/drm/drm_property.h:287:47: note: expected ‘ssize_t’ {aka ‘long int’} but argument is of type ‘bool *’ {aka ‘_Bool *’}
287 | ssize_t max_size,
| ~~~~~~~~^~~~~~~~
../drivers/gpu/drm/tests/drm_hdmi_state_helper_test.c:2922:15: error: too few arguments to function ‘drm_property_replace_blob_from_id’
2922 | ret = drm_property_replace_blob_from_id(drm,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../include/drm/drm_property.h:282:5: note: declared here
282 | int drm_property_replace_blob_from_id(struct drm_device *dev,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[7]: *** [../scripts/Makefile.build:287: drivers/gpu/drm/tests/drm_hdmi_state_helper_test.o] Error 1
make[7]: *** Waiting for unfinished jobs....
make[6]: *** [../scripts/Makefile.build:544: drivers/gpu/drm/tests] Error 2
make[6]: *** Waiting for unfinished jobs....
make[5]: *** [../scripts/Makefile.build:544: drivers/gpu/drm] Error 2
make[4]: *** [../scripts/Makefile.build:544: drivers/gpu] Error 2
make[3]: *** [../scripts/Makefile.build:544: drivers] Error 2
make[2]: *** [/kernel/Makefile:2054: .] Error 2
make[1]: *** [/kernel/Makefile:248: __sub-make] Error 2
make: *** [Makefile:248: __sub-make] Error 2+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel