drm/xe: Driver-Managed exhaustive eviction
Subsystems (3)
Thomas Hellström
maintainerPATCH v3 1/1 drm/xe: Driver-Managed exhaustive eviction
For CI only. Please don't review at this point.
Squashed commit of the following:
commit 628fa5759d25e125e4d08e4752174956f6da5b33
Author: Thomas Hellström <[email protected]> Date: Mon Aug 4 18:15:57 2025 +0200
drm/xe: Handle xe_bo_create_pin_map()
commit 7ce2385d6c169d0c1c077967241a2bcf9093c7a4
Author: Thomas Hellström <[email protected]> Date: Mon Aug 4 17:55:36 2025 +0200
drm/xe: Handle xe_bo_create_pin_map_at()
commit 2de49a5e034d40c4b8492e6285ce9ca2954f3e23
Author: Thomas Hellström <[email protected]> Date: Fri Aug 8 11:51:53 2025 +0200
drm/xe: Rename ___xe_bo_create_locked()
Don't start extern function names with underscores.
commit 32d3d107e4fe7e5985ed2a9867e7cc9d8270e9bf
Author: Thomas Hellström <[email protected]> Date: Sun Jun 30 16:33:28 2024 +0200
drm/xe/validation: Convert xe_dma_buf.c
commit 19a28e02a36ae5185aa40c463c574eaf12086609
Author: Thomas Hellström <[email protected]> Date: Sun Jun 30 16:05:42 2024 +0200
drm/exec/validation: Convert __xe_pin_fb_vma()
commit f118cc1cd8696b637125fe01e14c2c7206dde195
Author: Thomas Hellström <[email protected]> Date: Wed Jun 12 17:13:13 2024 +0200
drm/xe: Conversion of the fault handler to support drm_exec locking
commit cfdd6f1fafbc23dfbc75dd1f1e4f5f088c8e0cfb
Author: Thomas Hellström <[email protected]> Date: Mon Aug 4 17:52:35 2025 +0200
drm/xe: Wrap all instances of drm_exec_init / drm_exec_fini.
commit b45da2d7024235990f0563af4daf2ed0312ff4f3
Author: Thomas Hellström <[email protected]> Date: Wed Jun 25 10:48:44 2025 +0200
drm/xe: Fix up svm validation.
commit da2f75328737250bcce51186e5700a8548cca463
Author: Thomas Hellström <[email protected]> Date: Mon Aug 4 17:50:53 2025 +0200
drm/xe: Fix up xe_bo_create_user()
commit f62d65d20844659e96eaa922f9247f5fe6b777bc
Author: Thomas Hellström <[email protected]> Date: Wed Jun 12 09:30:31 2024 +0200
drm/xe: Introduce an xe_validation wrapper around drm_exec
commit 8cf893065acad3f702ab4c888c694798a5e1077a
Author: Thomas Hellström <[email protected]> Date: Wed Jun 12 14:41:01 2024 +0200
drm/xe: Pass down drm_exec context to validation
We want all validation (potential backing store allocation) to be part of a drm_exec transaction. Therefore add a drm_exec pointer argument to xe_bo_validate() and ___xe_bo_create_locked(). Upcoming patches will deal with making all (or nearly all) calls to these functions part of a drm_exec transaction. In the meantime, define special values of the drm_exec pointer:
XE_VALIDATION_UNIMPLEMENTED: Implementation of the drm_exec transaction has not been done yet.
XE_VALIDATION_UNSUPPORTED: Some Middle-layers (dma-buf) doesn't allow the drm_exec context to be passed down to map_attachment where validation takes place.
XE_VALIDATION_OPT_OUT: May be used only for kunit tests where exhaustive eviction isn't crucial and the ROI of converting those is very small.
For XE_VALIDATION_UNIMPLEMENTED and XE_VALIDATION_OPT_OUT there is also a lockdep check that a drm_exec transaction can indeed start at the location where the macro is expanded. This is to encourage developers to take this into consideration early in the code development process.
commit 8886dcd8882ed82b19920bf4712213fdc6570aa6
Author: Thomas Hellström <[email protected]> Date: Wed Jul 3 11:50:32 2024 +0200
drm/xe/vm: Clear the scratch_pt pointer on error
Avoid triggering a dereference of an error pointer on cleanup in xe_vm_free_scratch() by clearing any scratch_pt error pointer.
commit 56ca81a99e6b524fd6fd5d8e621638725f645ebd
Author: Thomas Hellström <[email protected]> Date: Sun Jun 30 22:20:06 2024 +0200
drm/xe/tests/xe_dma_buf: Set the drm_object::dma_buf member
This member is set when exporting using prime. However the xe_gem_prime_export() alone doesn't set it, since it's done later in the prime export flow.
For the test, set it manually and remove the hack that set it temporarily when it was really needed.
--- drivers/gpu/drm/xe/Makefile | 1 + .../compat-i915-headers/gem/i915_gem_stolen.h | 24 +- drivers/gpu/drm/xe/display/intel_fbdev_fb.c | 18 +- drivers/gpu/drm/xe/display/xe_dsb_buffer.c | 10 +- drivers/gpu/drm/xe/display/xe_fb_pin.c | 62 +-- drivers/gpu/drm/xe/display/xe_hdcp_gsc.c | 8 +- drivers/gpu/drm/xe/display/xe_plane_initial.c | 4 +- drivers/gpu/drm/xe/tests/xe_bo.c | 36 +- drivers/gpu/drm/xe/tests/xe_dma_buf.c | 24 +- drivers/gpu/drm/xe/tests/xe_migrate.c | 66 +-- drivers/gpu/drm/xe/xe_bo.c | 398 +++++++++++++----- drivers/gpu/drm/xe/xe_bo.h | 56 ++- drivers/gpu/drm/xe/xe_device.c | 2 + drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_dma_buf.c | 70 ++- drivers/gpu/drm/xe/xe_eu_stall.c | 6 +- drivers/gpu/drm/xe/xe_exec.c | 26 +- drivers/gpu/drm/xe/xe_ggtt.c | 15 +- drivers/gpu/drm/xe/xe_ggtt.h | 5 +- drivers/gpu/drm/xe/xe_gsc.c | 8 +- drivers/gpu/drm/xe/xe_gt_pagefault.c | 24 +- drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c | 22 +- drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c | 24 +- drivers/gpu/drm/xe/xe_guc_engine_activity.c | 13 +- drivers/gpu/drm/xe/xe_lmtt.c | 12 +- drivers/gpu/drm/xe/xe_lrc.c | 7 +- drivers/gpu/drm/xe/xe_migrate.c | 18 +- drivers/gpu/drm/xe/xe_oa.c | 6 +- drivers/gpu/drm/xe/xe_pt.c | 10 +- drivers/gpu/drm/xe/xe_pt.h | 3 +- drivers/gpu/drm/xe/xe_pxp_submit.c | 34 +- drivers/gpu/drm/xe/xe_svm.c | 65 ++- drivers/gpu/drm/xe/xe_validation.c | 248 +++++++++++ drivers/gpu/drm/xe/xe_validation.h | 176 ++++++++ drivers/gpu/drm/xe/xe_vm.c | 285 +++++++------ drivers/gpu/drm/xe/xe_vm.h | 37 +- drivers/gpu/drm/xe/xe_vm_types.h | 32 +- 37 files changed, 1308 insertions(+), 550 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_validation.c create mode 100644 drivers/gpu/drm/xe/xe_validation.h
drivers/gpu/drm/xe/Makefile +1 -0
@@ -127,6 +127,7 @@ xe-y += xe_bb.o \ xe_tuning.o \ xe_uc.o \ xe_uc_fw.o \ + xe_validation.o \ xe_vm.o \ xe_vram.o \ xe_vram_freq.o \
drivers/gpu/drm/xe/compat-i915-headers/gem/i915_gem_stolen.h +7 -17
@@ -8,6 +8,7 @@#include "xe_ttm_stolen_mgr.h" #include "xe_res_cursor.h" +#include "xe_validation.h" struct xe_bo ;
@@ -21,7 +22,6 @@ static inline int i915_gem_stolen_insert_node_in_range(struct xe_device *xe, u32 start, u32 end) {struct xe_bo *bo; -int err;u32 flags =XE_BO_FLAG_PINNED |XE_BO_FLAG_STOLEN ;if (start <SZ_4K )
@@ -32,25 +32,15 @@ static inline int i915_gem_stolen_insert_node_in_range(struct xe_device *xe, start =ALIGN (start, align); } - bo =xe_bo_create_locked_range (xe,xe_device_get_root_tile (xe), -NULL , size, start, end, - ttm_bo_type_kernel, flags,0 ); -if (IS_ERR (bo)) { - err =PTR_ERR (bo); - bo =NULL ; -return err; - } - err =xe_bo_pin (bo); -xe_bo_unlock_vm_held (bo); - -if (err) { -xe_bo_put (fb->bo); - bo =NULL ; - } + bo =xe_bo_create_pin_map_at_novm (xe,xe_device_get_root_tile (xe), + size, start, ttm_bo_type_kernel, flags, + false,0 , true); +if (IS_ERR (bo)) +return PTR_ERR (bo); fb->bo = bo; -return err; +return 0 ; } static inline int i915_gem_stolen_insert_node(struct xe_device *xe,
drivers/gpu/drm/xe/display/intel_fbdev_fb.c +9 -9
@@ -42,11 +42,11 @@ struct intel_framebuffer *intel_fbdev_fb_alloc(struct drm_fb_helper *helper, obj =ERR_PTR (-ENODEV ); if (!IS_DGFX(xe) && !XE_WA(xe_root_mmio_gt(xe), 22019338487_display)) { - obj =xe_bo_create_pin_map (xe,xe_device_get_root_tile (xe), -NULL , size, - ttm_bo_type_kernel,XE_BO_FLAG_SCANOUT | -XE_BO_FLAG_STOLEN | -XE_BO_FLAG_GGTT ); + obj =xe_bo_create_pin_map_novm (xe,xe_device_get_root_tile (xe), + size, + ttm_bo_type_kernel,XE_BO_FLAG_SCANOUT | +XE_BO_FLAG_STOLEN | +XE_BO_FLAG_GGTT , false);if (!IS_ERR (obj))drm_info (&xe->drm,"Allocated fbdev into stolen\n" );else
@@ -54,10 +54,10 @@ struct intel_framebuffer *intel_fbdev_fb_alloc(struct drm_fb_helper *helper, }if (IS_ERR (obj)) { - obj =xe_bo_create_pin_map (xe,xe_device_get_root_tile (xe),NULL , size, - ttm_bo_type_kernel,XE_BO_FLAG_SCANOUT | -XE_BO_FLAG_VRAM_IF_DGFX (xe_device_get_root_tile (xe)) | -XE_BO_FLAG_GGTT ); + obj =xe_bo_create_pin_map_novm (xe,xe_device_get_root_tile (xe), size, + ttm_bo_type_kernel,XE_BO_FLAG_SCANOUT | +XE_BO_FLAG_VRAM_IF_DGFX (xe_device_get_root_tile (xe)) | +XE_BO_FLAG_GGTT , false); }if (IS_ERR (obj)) {
drivers/gpu/drm/xe/display/xe_dsb_buffer.c +5 -5
@@ -43,11 +43,11 @@ bool intel_dsb_buffer_create(struct intel_crtc *crtc, struct intel_dsb_buffer *dreturn false;/* Set scanout flag for WC mapping */ - obj =xe_bo_create_pin_map (xe,xe_device_get_root_tile (xe), -NULL ,PAGE_ALIGN (size), - ttm_bo_type_kernel, -XE_BO_FLAG_VRAM_IF_DGFX (xe_device_get_root_tile (xe)) | -XE_BO_FLAG_SCANOUT |XE_BO_FLAG_GGTT ); + obj =xe_bo_create_pin_map_novm (xe,xe_device_get_root_tile (xe), +PAGE_ALIGN (size), + ttm_bo_type_kernel, +XE_BO_FLAG_VRAM_IF_DGFX (xe_device_get_root_tile (xe)) | +XE_BO_FLAG_SCANOUT |XE_BO_FLAG_GGTT , false);if (IS_ERR (obj)) {kfree (vma);return false;
drivers/gpu/drm/xe/display/xe_fb_pin.c +31 -31
83 lines not on this page · raw message
drivers/gpu/drm/xe/display/xe_hdcp_gsc.c +4 -4
@@ -72,10 +72,10 @@ static int intel_hdcp_gsc_initialize_message(struct xe_device *xe,int ret =0 ;/* allocate object of two page for HDCP command memory and store it */ - bo =xe_bo_create_pin_map (xe,xe_device_get_root_tile (xe),NULL ,PAGE_SIZE *2 , - ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM | -XE_BO_FLAG_GGTT ); + bo =xe_bo_create_pin_map_novm (xe,xe_device_get_root_tile (xe),PAGE_SIZE *2 , + ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM | +XE_BO_FLAG_GGTT , false);if (IS_ERR (bo)) {drm_err (&xe->drm,"Failed to allocate bo for HDCP streaming command!\n" );
drivers/gpu/drm/xe/display/xe_plane_initial.c +2 -2
@@ -140,8 +140,8 @@ initial_plane_bo(struct xe_device *xe, page_size); size -= base; - bo =xe_bo_create_pin_map_at (xe, tile0,NULL , size, phys_base, - ttm_bo_type_kernel, flags); + bo =xe_bo_create_pin_map_at_novm (xe, tile0, size, phys_base, + ttm_bo_type_kernel, flags, true,0 , false);if (IS_ERR (bo)) { drm_dbg(&xe->drm, "Failed to create bo phys_base=%pa size %u with flags %x: %li\n",
drivers/gpu/drm/xe/tests/xe_bo.c +19 -17
@@ -23,7 +23,7 @@static int ccs_test_migrate (struct xe_tile *tile,struct xe_bo *bo,bool clear,u64 get_val,u64 assign_val, -struct kunit *test) +struct kunit *test,struct drm_exec *exec) {struct dma_fence *fence;struct ttm_tt *ttm;
@@ -35,7 +35,7 @@ static int ccs_test_migrate(struct xe_tile *tile, struct xe_bo *bo,u32 offset;/* Move bo to VRAM if not already there. */ - ret =xe_bo_validate (bo,NULL , false); + ret =xe_bo_validate (bo,NULL , false, exec);if (ret) {KUNIT_FAIL (test,"Failed to validate bo.\n" );return ret;
@@ -60,7 +60,7 @@ static int ccs_test_migrate(struct xe_tile *tile, struct xe_bo *bo, }/* Evict to system. CCS data should be copied. */ - ret =xe_bo_evict (bo); + ret =xe_bo_evict (bo, exec);if (ret) {KUNIT_FAIL (test,"Failed to evict bo.\n" );return ret;
@@ -132,14 +132,15 @@ static void ccs_test_run_tile(struct xe_device *xe, struct xe_tile *tile,/* TODO: Sanity check */ unsigned int bo_flags =XE_BO_FLAG_VRAM_IF_DGFX (tile); +struct drm_exec *exec =XE_VALIDATION_OPT_OUT ;if (IS_DGFX (xe))kunit_info (test,"Testing vram id %u\n" , tile->id);else kunit_info (test,"Testing system memory\n" ); - bo =xe_bo_create_user (xe,NULL ,NULL ,SZ_1M ,DRM_XE_GEM_CPU_CACHING_WC , - bo_flags); + bo =xe_bo_create_user (xe,NULL ,SZ_1M ,DRM_XE_GEM_CPU_CACHING_WC , + bo_flags, exec);if (IS_ERR (bo)) {KUNIT_FAIL (test,"Failed to create bo.\n" );return ;
@@ -149,18 +150,18 @@ static void ccs_test_run_tile(struct xe_device *xe, struct xe_tile *tile,kunit_info (test,"Verifying that CCS data is cleared on creation.\n" ); ret =ccs_test_migrate (tile, bo, false,0ULL ,0xdeadbeefdeadbeefULL , - test); + test, exec);if (ret) goto out_unlock;kunit_info (test,"Verifying that CCS data survives migration.\n" ); ret =ccs_test_migrate (tile, bo, false,0xdeadbeefdeadbeefULL , -0xdeadbeefdeadbeefULL , test); +0xdeadbeefdeadbeefULL , test, exec);if (ret) goto out_unlock;kunit_info (test,"Verifying that CCS data can be properly cleared.\n" ); - ret =ccs_test_migrate (tile, bo, true,0ULL ,0ULL , test); + ret =ccs_test_migrate (tile, bo, true,0ULL ,0ULL , test, exec); out_unlock:xe_bo_unlock (bo);
@@ -210,6 +211,7 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, strucstruct xe_bo *bo, *external;unsigned int bo_flags =XE_BO_FLAG_VRAM_IF_DGFX (tile);struct xe_vm *vm =xe_migrate_get_vm (xe_device_get_root_tile (xe)->migrate); +struct drm_exec *exec =XE_VALIDATION_OPT_OUT ;struct xe_gt *__gt;int err, i, id;
@@ -218,25 +220,25 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, strucfor (i =0 ; i <2 ; ++i) {xe_vm_lock (vm, false); - bo =xe_bo_create_user (xe,NULL , vm,0x10000 , + bo =xe_bo_create_user (xe, vm,0x10000 ,DRM_XE_GEM_CPU_CACHING_WC , - bo_flags); + bo_flags, exec);xe_vm_unlock (vm);if (IS_ERR (bo)) {KUNIT_FAIL (test,"bo create err=%pe\n" , bo);break ; } - external =xe_bo_create_user (xe,NULL ,NULL ,0x10000 , + external =xe_bo_create_user (xe,NULL ,0x10000 ,DRM_XE_GEM_CPU_CACHING_WC , - bo_flags); + bo_flags,NULL );if (IS_ERR (external)) {KUNIT_FAIL (test,"external bo create err=%pe\n" , external); goto cleanup_bo; }xe_bo_lock (external, false); - err =xe_bo_pin_external (external); + err =xe_bo_pin_external (external, exec);xe_bo_unlock (external);if (err) { KUNIT_FAIL(test, "external bo pin err=%pe\n",
@@ -294,7 +296,7 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, strucif (i) {down_read (&vm->lock);xe_vm_lock (vm, false); - err =xe_bo_validate (bo, bo->vm, false); + err =xe_bo_validate (bo, bo->vm, false, exec);xe_vm_unlock (vm);up_read (&vm->lock);if (err) {
@@ -303,7 +305,7 @@ static int evict_test_run_tile(struct xe_device *xe, struct xe_tile *tile, struc goto cleanup_all; }xe_bo_lock (external, false); - err =xe_bo_validate (external,NULL , false); + err =xe_bo_validate (external,NULL , false, exec);xe_bo_unlock (external);if (err) { KUNIT_FAIL(test, "external bo valid err=%pe\n",
@@ -495,9 +497,9 @@ static int shrink_test_run_device(struct xe_device *xe)INIT_LIST_HEAD (&link->link);/* We can create bos using WC caching here. But it is slower. */ - bo =xe_bo_create_user (xe,NULL ,NULL ,XE_BO_SHRINK_SIZE , + bo =xe_bo_create_user (xe,NULL ,XE_BO_SHRINK_SIZE ,DRM_XE_GEM_CPU_CACHING_WB , -XE_BO_FLAG_SYSTEM ); +XE_BO_FLAG_SYSTEM ,NULL );if (IS_ERR (bo)) {if (bo !=ERR_PTR (-ENOMEM ) && bo !=ERR_PTR (-ENOSPC ) && bo !=ERR_PTR (-EINTR ) && bo !=ERR_PTR (-ERESTARTSYS ))
drivers/gpu/drm/xe/tests/xe_dma_buf.c +12 -12
@@ -27,7 +27,8 @@ static bool is_dynamic(struct dma_buf_test_params *params) }static void check_residency (struct kunit *test,struct xe_bo *exported, -struct xe_bo *imported,struct dma_buf *dmabuf) +struct xe_bo *imported,struct dma_buf *dmabuf, +struct drm_exec *exec) {struct dma_buf_test_params *params =to_dma_buf_test_params (test->priv);u32 mem_type;
@@ -57,16 +58,12 @@ static void check_residency(struct kunit *test, struct xe_bo *exported,return ;/* -* Evict exporter. Note that the gem object dma_buf member isn't -* set from xe_gem_prime_export(), and it's needed for the move_notify() -* functionality, so hack that up here. Evicting the exported bo will +* Evict exporter. Evicting the exported bo will * evict also the imported bo through the move_notify() functionality if * importer is on a different device. If they're on the same device, * the exporter and the importer should be the same bo. */ -swap (exported->ttm.base.dma_buf, dmabuf); - ret =xe_bo_evict (exported); -swap (exported->ttm.base.dma_buf, dmabuf); + ret =xe_bo_evict (exported, exec);if (ret) {if (ret != -EINTR && ret != -ERESTARTSYS ) KUNIT_FAIL(test, "Evicting exporter failed with err=%d.\n",
@@ -81,7 +78,7 @@ static void check_residency(struct kunit *test, struct xe_bo *exported, }/* Re-validate the importer. This should move also exporter in. */ - ret =xe_bo_validate (imported,NULL , false); + ret =xe_bo_validate (imported,NULL , false, exec);if (ret) {if (ret != -EINTR && ret != -ERESTARTSYS ) KUNIT_FAIL(test, "Validating importer failed with err=%d.\n",
@@ -125,8 +122,8 @@ static void xe_test_dmabuf_import_same_driver(struct xe_device *xe) size =SZ_64K ;kunit_info (test,"running %s\n" , __func__); - bo =xe_bo_create_user (xe,NULL ,NULL , size,DRM_XE_GEM_CPU_CACHING_WC , - params->mem_mask); + bo =xe_bo_create_user (xe,NULL , size,DRM_XE_GEM_CPU_CACHING_WC , + params->mem_mask,NULL );if (IS_ERR (bo)) {KUNIT_FAIL (test,"xe_bo_create() failed with err=%ld\n" ,PTR_ERR (bo));
@@ -139,6 +136,7 @@ static void xe_test_dmabuf_import_same_driver(struct xe_device *xe) PTR_ERR(dmabuf)); goto out; } + bo->ttm.base.dma_buf = dmabuf; import =xe_gem_prime_import (&xe->drm, dmabuf);if (!IS_ERR (import)) {
@@ -153,11 +151,12 @@ static void xe_test_dmabuf_import_same_driver(struct xe_device *xe)KUNIT_FAIL (test,"xe_gem_prime_import() succeeded when it shouldn't have\n" ); }else { +struct drm_exec *exec =XE_VALIDATION_OPT_OUT ;int err;/* Is everything where we expect it to be? */ xe_bo_lock (import_bo, false); - err =xe_bo_validate (import_bo,NULL , false); + err =xe_bo_validate (import_bo,NULL , false, exec);/* Pinning in VRAM is not allowed. */ if (!is_dynamic(params) &&
@@ -170,7 +169,7 @@ static void xe_test_dmabuf_import_same_driver(struct xe_device *xe) err == -ERESTARTSYS);if (!err) -check_residency (test, bo, import_bo, dmabuf); +check_residency (test, bo, import_bo, dmabuf, exec);xe_bo_unlock (import_bo); }drm_gem_object_put (import);
@@ -186,6 +185,7 @@ static void xe_test_dmabuf_import_same_driver(struct xe_device *xe)KUNIT_FAIL (test,"dynamic p2p attachment failed with err=%ld\n" ,PTR_ERR (import)); } + bo->ttm.base.dma_buf =NULL ;dma_buf_put (dmabuf); out:drm_gem_object_put (&bo->ttm.base);
drivers/gpu/drm/xe/tests/xe_migrate.c +37 -29
215 lines not on this page · raw message
drivers/gpu/drm/xe/xe_bo.c +304 -94
641 lines not on this page · raw message
drivers/gpu/drm/xe/xe_bo.h +27 -29
90 lines not on this page · raw message
drivers/gpu/drm/xe/xe_device.c +2 -0
@@ -452,6 +452,8 @@ struct xe_device *xe_device_create(struct pci_dev *pdev,if (err) goto err; +xe_validation_device_init (&xe->val); +init_waitqueue_head (&xe->ufence_wq);init_rwsem (&xe->usm.lock);
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -26,6 +26,7 @@#include "xe_sriov_vf_ccs_types.h" #include "xe_step_types.h" #include "xe_survivability_mode_types.h" +#include "xe_validation.h" #if IS_ENABLED (CONFIG_DRM_XE_DEBUG )#define TEST_VM_OPS_ERROR
@@ -575,6 +576,8 @@ struct xe_device {*/ atomic64_t global_total_pages;#endif +/** @val: The domain for exhaustive eviction, which is currently per device. */ +struct xe_validation_device val;/* private: */
drivers/gpu/drm/xe/xe_dma_buf.c +50 -20
131 lines not on this page · raw message
drivers/gpu/drm/xe/xe_eu_stall.c +3 -3
@@ -617,9 +617,9 @@ static int xe_eu_stall_data_buf_alloc(struct xe_eu_stall_data_stream *stream, size = stream->per_xecore_buf_size * last_xecore; - bo =xe_bo_create_pin_map_at_aligned (tile->xe, tile,NULL , - size, ~0ull , ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM |XE_BO_FLAG_GGTT ,SZ_64 ); + bo =xe_bo_create_pin_map_at_novm (tile->xe, tile, size, ~0ull , ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM |XE_BO_FLAG_GGTT , true, +SZ_64 , false);if (IS_ERR (bo)) {kfree (stream->xecore_buf);return PTR_ERR (bo);
drivers/gpu/drm/xe/xe_exec.c +13 -13
@@ -97,9 +97,13 @@static int xe_exec_fn (struct drm_gpuvm_exec *vm_exec) { struct xe_vm *vm = container_of(vm_exec->vm, struct xe_vm, gpuvm); +int ret;/* The fence slot added here is intended for the exec sched job. */ -return xe_vm_validate_rebind (vm, &vm_exec->exec,1 ); +xe_vm_set_validation_exec (vm, &vm_exec->exec); + ret =xe_vm_validate_rebind (vm, &vm_exec->exec,1 ); +xe_vm_set_validation_exec (vm,NULL ); +return ret; }int xe_exec_ioctl (struct drm_device *dev,void *data,struct drm_file *file)
@@ -115,10 +119,10 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)struct drm_gpuvm_exec vm_exec = {.extra.fn = xe_exec_fn};struct drm_exec *exec = &vm_exec.exec;u32 i, num_syncs, num_ufence =0 ; +struct xe_validation_ctx ctx;struct xe_sched_job *job;struct xe_vm *vm;bool write_locked, skip_retry = false; -ktime_t end =0 ;int err =0 ;struct xe_hw_engine_group *group;enum xe_hw_engine_group_execution_mode mode, previous_mode;
@@ -237,17 +241,12 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_unlock_list; } - vm_exec.vm = &vm->gpuvm; - vm_exec.flags =DRM_EXEC_INTERRUPTIBLE_WAIT ; -if (xe_vm_in_lr_mode (vm)) { -drm_exec_init (exec, vm_exec.flags,0 ); - }else { - err =drm_gpuvm_exec_lock (&vm_exec); -if (err) { -if (xe_vm_validate_should_retry (exec, err, &end)) - err = -EAGAIN ; +if (!xe_vm_in_lr_mode (vm)) { + vm_exec.vm = &vm->gpuvm; + vm_exec.flags =DRM_EXEC_INTERRUPTIBLE_WAIT ; + err =xe_validation_exec_lock (&ctx, &vm_exec, &xe->val); +if (err) goto err_unlock_list; - } }if (xe_vm_is_closed_or_banned (q->vm)) {
@@ -341,7 +340,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (err)xe_sched_job_put (job); err_exec: -drm_exec_fini (exec); +if (!xe_vm_in_lr_mode (vm)) +xe_validation_ctx_fini (&ctx); err_unlock_list:up_read (&vm->lock);if (err == -EAGAIN && !skip_retry)
drivers/gpu/drm/xe/xe_ggtt.c +9 -6
@@ -731,7 +731,7 @@ void xe_ggtt_map_bo_unlocked(struct xe_ggtt *ggtt, struct xe_bo *bo) }static int __xe_ggtt_insert_bo_at (struct xe_ggtt *ggtt,struct xe_bo *bo, -u64 start,u64 end) +u64 start,u64 end,struct drm_exec *exec) {u64 alignment = bo->min_align >0 ? bo->min_align :XE_PAGE_SIZE ;u8 tile_id = ggtt->tile->id;
@@ -746,7 +746,7 @@ static int __xe_ggtt_insert_bo_at(struct xe_ggtt *ggtt, struct xe_bo *bo,return 0 ; } - err =xe_bo_validate (bo,NULL , false); + err =xe_bo_validate (bo,NULL , false, exec);if (err)return err;
@@ -788,25 +788,28 @@ static int __xe_ggtt_insert_bo_at(struct xe_ggtt *ggtt, struct xe_bo *bo,* @bo: the &xe_bo to be inserted * @start: address where it will be inserted * @end: end of the range where it will be inserted +* @exec: The drm_exec transaction to use for exhaustive eviction. * * Return: 0 on success or a negative error code on failure. */ int xe_ggtt_insert_bo_at (struct xe_ggtt *ggtt,struct xe_bo *bo, -u64 start,u64 end) +u64 start,u64 end,struct drm_exec *exec) { -return __xe_ggtt_insert_bo_at (ggtt, bo, start, end); +return __xe_ggtt_insert_bo_at (ggtt, bo, start, end, exec); }/** * xe_ggtt_insert_bo - Insert BO into GGTT * @ggtt: the &xe_ggtt where bo will be inserted * @bo: the &xe_bo to be inserted +* @exec: The drm_exec transaction to use for exhaustive eviction. * * Return: 0 on success or a negative error code on failure. */ -int xe_ggtt_insert_bo (struct xe_ggtt *ggtt,struct xe_bo *bo) +int xe_ggtt_insert_bo (struct xe_ggtt *ggtt,struct xe_bo *bo, +struct drm_exec *exec) { -return __xe_ggtt_insert_bo_at (ggtt, bo,0 ,U64_MAX ); +return __xe_ggtt_insert_bo_at (ggtt, bo,0 ,U64_MAX , exec); }/**
drivers/gpu/drm/xe/xe_ggtt.h +3 -2
@@ -10,6 +10,7 @@struct drm_printer ;struct xe_tile ; +struct drm_exec ;struct xe_ggtt *xe_ggtt_alloc (struct xe_tile *tile);int xe_ggtt_init_early (struct xe_ggtt *ggtt);
@@ -31,9 +32,9 @@ bool xe_ggtt_node_allocated(const struct xe_ggtt_node *node);void xe_ggtt_map_bo (struct xe_ggtt *ggtt,struct xe_ggtt_node *node,struct xe_bo *bo,u16 pat_index);void xe_ggtt_map_bo_unlocked (struct xe_ggtt *ggtt,struct xe_bo *bo); -int xe_ggtt_insert_bo (struct xe_ggtt *ggtt,struct xe_bo *bo); +int xe_ggtt_insert_bo (struct xe_ggtt *ggtt,struct xe_bo *bo,struct drm_exec *exec);int xe_ggtt_insert_bo_at (struct xe_ggtt *ggtt,struct xe_bo *bo, -u64 start,u64 end); +u64 start,u64 end,struct drm_exec *exec);void xe_ggtt_remove_bo (struct xe_ggtt *ggtt,struct xe_bo *bo);u64 xe_ggtt_largest_hole (struct xe_ggtt *ggtt,u64 alignment,u64 *spare);
drivers/gpu/drm/xe/xe_gsc.c +4 -4
@@ -136,10 +136,10 @@ static int query_compatibility_version(struct xe_gsc *gsc)u64 ggtt_offset;int err; - bo =xe_bo_create_pin_map (xe, tile,NULL ,GSC_VER_PKT_SZ *2 , - ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM | -XE_BO_FLAG_GGTT ); + bo =xe_bo_create_pin_map_novm (xe, tile,GSC_VER_PKT_SZ *2 , + ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM | +XE_BO_FLAG_GGTT , false);if (IS_ERR (bo)) {xe_gt_err (gt,"failed to allocate bo for GSC version query\n" );return PTR_ERR (bo);
drivers/gpu/drm/xe/xe_gt_pagefault.c +11 -13
@@ -94,12 +94,12 @@ static int xe_pf_begin(struct drm_exec *exec, struct xe_vma *vma, }/* Migrate to VRAM, move should invalidate the VMA first */ - err =xe_bo_migrate (bo, vram->placement); + err =xe_bo_migrate (bo, vram->placement, exec);if (err)return err; }else if (bo) {/* Create backing store if needed */ - err =xe_bo_validate (bo, vm, true); + err =xe_bo_validate (bo, vm, true, exec);if (err)return err; }
@@ -112,9 +112,9 @@ static int handle_vma_pagefault(struct xe_gt *gt, struct xe_vma *vma, {struct xe_vm *vm =xe_vma_vm (vma);struct xe_tile *tile =gt_to_tile (gt); +struct xe_validation_ctx ctx;struct drm_exec exec;struct dma_fence *fence; -ktime_t end =0 ;int err;lockdep_assert_held_write (&vm->lock);
@@ -139,12 +139,11 @@ static int handle_vma_pagefault(struct xe_gt *gt, struct xe_vma *vma, }/* Lock VM and BOs dma-resv */ -drm_exec_init (&exec,0 ,0 ); +xe_validation_ctx_init (&ctx, &vm->xe->val, &exec,0 ,0 , false); drm_exec_until_all_locked(&exec) { err =xe_pf_begin (&exec, vma, atomic, tile->mem.vram);drm_exec_retry_on_contention (&exec); -if (xe_vm_validate_should_retry (&exec, err, &end)) - err = -EAGAIN ; +xe_validation_retry_on_oom (&ctx, &err);if (err) goto unlock_dma_resv;
@@ -153,8 +152,7 @@ static int handle_vma_pagefault(struct xe_gt *gt, struct xe_vma *vma, fence =xe_vma_rebind (vm, vma,BIT (tile->id));if (IS_ERR (fence)) { err =PTR_ERR (fence); -if (xe_vm_validate_should_retry (&exec, err, &end)) - err = -EAGAIN ; +xe_validation_retry_on_oom (&ctx, &err); goto unlock_dma_resv; } }
@@ -163,7 +161,7 @@ static int handle_vma_pagefault(struct xe_gt *gt, struct xe_vma *vma,dma_fence_put (fence); unlock_dma_resv: -drm_exec_fini (&exec); +xe_validation_ctx_fini (&ctx);if (err == -EAGAIN ) goto retry_userptr;
@@ -545,6 +543,7 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc) {struct xe_device *xe =gt_to_xe (gt);struct xe_tile *tile =gt_to_tile (gt); +struct xe_validation_ctx ctx;struct drm_exec exec;struct xe_vm *vm;struct xe_vma *vma;
@@ -574,15 +573,14 @@ static int handle_acc(struct xe_gt *gt, struct acc *acc) goto unlock_vm;/* Lock VM and BOs dma-resv */ -drm_exec_init (&exec,0 ,0 ); +xe_validation_ctx_init (&ctx, &vm->xe->val, &exec,0 ,0 , false); drm_exec_until_all_locked(&exec) { ret =xe_pf_begin (&exec, vma, true, tile->mem.vram);drm_exec_retry_on_contention (&exec); -if (ret) -break ; +xe_validation_retry_on_oom (&ctx, &ret); } -drm_exec_fini (&exec); +xe_validation_ctx_fini (&ctx); unlock_vm:up_read (&vm->lock);xe_vm_put (vm);
drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c +8 -14
@@ -1475,23 +1475,17 @@ static int pf_provision_vf_lmem(struct xe_gt *gt, unsigned int vfid, u64 size)return 0 ;xe_gt_assert (gt,pf_get_lmem_alignment (gt) ==SZ_2M ); - bo =xe_bo_create_locked (xe, tile,NULL , -ALIGN (size,PAGE_SIZE ), - ttm_bo_type_kernel, -XE_BO_FLAG_VRAM_IF_DGFX (tile) | -XE_BO_FLAG_NEEDS_2M | -XE_BO_FLAG_PINNED | -XE_BO_FLAG_PINNED_LATE_RESTORE ); + bo =xe_bo_create_pin_map_at_novm (xe, tile, +ALIGN (size,PAGE_SIZE ), + ~0ull , + ttm_bo_type_kernel, +XE_BO_FLAG_VRAM_IF_DGFX (tile) | +XE_BO_FLAG_NEEDS_2M | +XE_BO_FLAG_PINNED_LATE_RESTORE , + false,0 , false);if (IS_ERR (bo))return PTR_ERR (bo); - err =xe_bo_pin (bo); -xe_bo_unlock (bo); -if (unlikely (err)) { -xe_bo_put (bo); -return err; - } - config->lmem_obj = bo;if (xe_device_has_lmtt (xe)) {
drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c +12 -12
@@ -55,12 +55,12 @@ static int pf_send_guc_save_vf_state(struct xe_gt *gt, unsigned int vfid,xe_gt_assert (gt, size %sizeof (u32) ==0 );xe_gt_assert (gt, size == ndwords *sizeof (u32)); - bo =xe_bo_create_pin_map (xe, tile,NULL , -ALIGN (size,PAGE_SIZE ), - ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM | -XE_BO_FLAG_GGTT | -XE_BO_FLAG_GGTT_INVALIDATE ); + bo =xe_bo_create_pin_map_novm (xe, tile, +ALIGN (size,PAGE_SIZE ), + ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM | +XE_BO_FLAG_GGTT | +XE_BO_FLAG_GGTT_INVALIDATE , false);if (IS_ERR (bo))return PTR_ERR (bo);
@@ -91,12 +91,12 @@ static int pf_send_guc_restore_vf_state(struct xe_gt *gt, unsigned int vfid,xe_gt_assert (gt, size %sizeof (u32) ==0 );xe_gt_assert (gt, size == ndwords *sizeof (u32)); - bo =xe_bo_create_pin_map (xe, tile,NULL , -ALIGN (size,PAGE_SIZE ), - ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM | -XE_BO_FLAG_GGTT | -XE_BO_FLAG_GGTT_INVALIDATE ); + bo =xe_bo_create_pin_map_novm (xe, tile, +ALIGN (size,PAGE_SIZE ), + ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM | +XE_BO_FLAG_GGTT | +XE_BO_FLAG_GGTT_INVALIDATE , false);if (IS_ERR (bo))return PTR_ERR (bo);
drivers/gpu/drm/xe/xe_guc_engine_activity.c +7 -6
@@ -94,16 +94,17 @@ static int allocate_engine_activity_buffers(struct xe_guc *guc,struct xe_tile *tile =gt_to_tile (gt);struct xe_bo *bo, *metadata_bo; - metadata_bo =xe_bo_create_pin_map (gt_to_xe (gt), tile,NULL ,PAGE_ALIGN (metadata_size), - ttm_bo_type_kernel,XE_BO_FLAG_SYSTEM | -XE_BO_FLAG_GGTT |XE_BO_FLAG_GGTT_INVALIDATE ); + metadata_bo =xe_bo_create_pin_map_novm (gt_to_xe (gt), tile,PAGE_ALIGN (metadata_size), + ttm_bo_type_kernel,XE_BO_FLAG_SYSTEM | +XE_BO_FLAG_GGTT |XE_BO_FLAG_GGTT_INVALIDATE , + false);if (IS_ERR (metadata_bo))return PTR_ERR (metadata_bo); - bo =xe_bo_create_pin_map (gt_to_xe (gt), tile,NULL ,PAGE_ALIGN (size), - ttm_bo_type_kernel,XE_BO_FLAG_VRAM_IF_DGFX (tile) | -XE_BO_FLAG_GGTT |XE_BO_FLAG_GGTT_INVALIDATE ); + bo =xe_bo_create_pin_map_novm (gt_to_xe (gt), tile,PAGE_ALIGN (size), + ttm_bo_type_kernel,XE_BO_FLAG_VRAM_IF_DGFX (tile) | +XE_BO_FLAG_GGTT |XE_BO_FLAG_GGTT_INVALIDATE , false);if (IS_ERR (bo)) {xe_bo_unpin_map_no_vm (metadata_bo);
drivers/gpu/drm/xe/xe_lmtt.c +6 -6
@@ -67,12 +67,12 @@ static struct xe_lmtt_pt *lmtt_pt_alloc(struct xe_lmtt *lmtt, unsigned int level goto out; } - bo =xe_bo_create_pin_map (lmtt_to_xe (lmtt),lmtt_to_tile (lmtt),NULL , -PAGE_ALIGN (lmtt->ops->lmtt_pte_size (level) * - lmtt->ops->lmtt_pte_num (level)), - ttm_bo_type_kernel, -XE_BO_FLAG_VRAM_IF_DGFX (lmtt_to_tile (lmtt)) | -XE_BO_FLAG_NEEDS_64K ); + bo =xe_bo_create_pin_map_novm (lmtt_to_xe (lmtt),lmtt_to_tile (lmtt), +PAGE_ALIGN (lmtt->ops->lmtt_pte_size (level) * + lmtt->ops->lmtt_pte_num (level)), + ttm_bo_type_kernel, +XE_BO_FLAG_VRAM_IF_DGFX (lmtt_to_tile (lmtt)) | +XE_BO_FLAG_NEEDS_64K , false);if (IS_ERR (bo)) { err =PTR_ERR (bo); goto out_free_pt;
drivers/gpu/drm/xe/xe_lrc.c +4 -3
@@ -1268,9 +1268,10 @@ static int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,if (vm && vm->xef)/* userspace */ bo_flags |=XE_BO_FLAG_PINNED_LATE_RESTORE ; - lrc->bo =xe_bo_create_pin_map (xe, tile,NULL , bo_size, - ttm_bo_type_kernel, - bo_flags); + lrc->bo =xe_bo_create_pin_map_novm (xe, tile, + bo_size, + ttm_bo_type_kernel, + bo_flags, false);if (IS_ERR (lrc->bo))return PTR_ERR (lrc->bo);
drivers/gpu/drm/xe/xe_migrate.c +13 -5
@@ -34,6 +34,7 @@#include "xe_sched_job.h" #include "xe_sync.h" #include "xe_trace_bo.h" +#include "xe_validation.h" #include "xe_vm.h" #include "xe_vram.h"
@@ -173,7 +174,7 @@ static void xe_migrate_program_identity(struct xe_device *xe, struct xe_vm *vm, }static int xe_migrate_prepare_vm (struct xe_tile *tile,struct xe_migrate *m, -struct xe_vm *vm) +struct xe_vm *vm,struct drm_exec *exec) {struct xe_device *xe =tile_to_xe (tile);u16 pat_index = xe->pat.idx[XE_CACHE_WB ];
@@ -200,7 +201,7 @@ static int xe_migrate_prepare_vm(struct xe_tile *tile, struct xe_migrate *m, num_entries *XE_PAGE_SIZE , ttm_bo_type_kernel,XE_BO_FLAG_VRAM_IF_DGFX (tile) | - XE_BO_FLAG_PAGETABLE); + XE_BO_FLAG_PAGETABLE, exec);if (IS_ERR (bo))return PTR_ERR (bo);
@@ -405,6 +406,8 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)struct xe_device *xe =tile_to_xe (tile);struct xe_gt *primary_gt = tile->primary_gt;struct xe_migrate *m = tile->migrate; +struct xe_validation_ctx ctx; +struct drm_exec exec;struct xe_vm *vm;int err;
@@ -414,9 +417,14 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)if (IS_ERR (vm))return ERR_CAST (vm); -xe_vm_lock (vm, false); - err =xe_migrate_prepare_vm (tile, m, vm); -xe_vm_unlock (vm); + err =0 ; + xe_validation_guard(&ctx, &xe->val, &exec, 0, err, false) { + err =xe_vm_drm_exec_lock (vm, &exec); +drm_exec_retry_on_contention (&exec); + err =xe_migrate_prepare_vm (tile, m, vm, &exec); +drm_exec_retry_on_contention (&exec); +xe_validation_retry_on_oom (&ctx, &err); + }if (err) {xe_vm_close_and_put (vm);return ERR_PTR (err);
drivers/gpu/drm/xe/xe_oa.c +3 -3
@@ -883,9 +883,9 @@ static int xe_oa_alloc_oa_buffer(struct xe_oa_stream *stream, size_t size) {struct xe_bo *bo; - bo =xe_bo_create_pin_map (stream->oa->xe, stream->gt->tile,NULL , - size, ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM |XE_BO_FLAG_GGTT ); + bo =xe_bo_create_pin_map_novm (stream->oa->xe, stream->gt->tile, + size, ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM |XE_BO_FLAG_GGTT , false);if (IS_ERR (bo))return PTR_ERR (bo);
drivers/gpu/drm/xe/xe_pt.c +7 -3
@@ -88,6 +88,7 @@ static void xe_pt_free(struct xe_pt *pt)* @vm: The vm to create for. * @tile: The tile to create for. * @level: The page-table level. +* @exec: The drm_exec object used to lock the vm. * * Allocate and initialize a single struct xe_pt metadata structure. Also * create the corresponding page-table bo, but don't initialize it. If the
@@ -99,7 +100,7 @@ static void xe_pt_free(struct xe_pt *pt)* error. */ struct xe_pt *xe_pt_create (struct xe_vm *vm,struct xe_tile *tile, -unsigned int level) +unsigned int level,struct drm_exec *exec) {struct xe_pt *pt;struct xe_bo *bo;
@@ -123,9 +124,11 @@ struct xe_pt *xe_pt_create(struct xe_vm *vm, struct xe_tile *tile, bo_flags |=XE_BO_FLAG_PINNED_LATE_RESTORE ; pt->level = level; + +drm_WARN_ON (&vm->xe->drm,IS_ERR_OR_NULL (exec)); bo =xe_bo_create_pin_map (vm->xe, tile, vm,SZ_4K , ttm_bo_type_kernel, - bo_flags); + bo_flags, exec);if (IS_ERR (bo)) { err =PTR_ERR (bo); goto err_kfree;
@@ -589,7 +592,8 @@ xe_pt_stage_bind_entry(struct xe_ptw *parent, pgoff_t offset,if (covers || !*child) {u64 flags =0 ; - xe_child =xe_pt_create (xe_walk->vm, xe_walk->tile, level -1 ); + xe_child =xe_pt_create (xe_walk->vm, xe_walk->tile, level -1 , +xe_vm_validation_exec (vm));if (IS_ERR (xe_child))return PTR_ERR (xe_child);
drivers/gpu/drm/xe/xe_pt.h +2 -1
@@ -10,6 +10,7 @@#include "xe_pt_types.h" struct dma_fence ; +struct drm_exec ;struct xe_bo ;struct xe_device ;struct xe_exec_queue ;
@@ -29,7 +30,7 @@ struct xe_vma_ops;unsigned int xe_pt_shift (unsigned int level);struct xe_pt *xe_pt_create (struct xe_vm *vm,struct xe_tile *tile, -unsigned int level); +unsigned int level,struct drm_exec *exec);void xe_pt_populate_empty (struct xe_tile *tile,struct xe_vm *vm,struct xe_pt *pt);
drivers/gpu/drm/xe/xe_pxp_submit.c +24 -10
@@ -54,8 +54,9 @@ static int allocate_vcs_execution_resources(struct xe_pxp *pxp)* Each termination is 16 DWORDS, so 4K is enough to contain a * termination for each sessions. */ - bo =xe_bo_create_pin_map (xe, tile,NULL ,SZ_4K , ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM |XE_BO_FLAG_PINNED |XE_BO_FLAG_GGTT ); + bo =xe_bo_create_pin_map_novm (xe, tile,SZ_4K , ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM |XE_BO_FLAG_PINNED |XE_BO_FLAG_GGTT , + false);if (IS_ERR (bo)) { err =PTR_ERR (bo); goto out_queue;
@@ -87,7 +88,9 @@ static int allocate_gsc_client_resources(struct xe_gt *gt, {struct xe_tile *tile =gt_to_tile (gt);struct xe_device *xe =tile_to_xe (tile); +struct xe_validation_ctx ctx;struct xe_hw_engine *hwe; +struct drm_exec exec;struct xe_vm *vm;struct xe_bo *bo;struct xe_exec_queue *q;
@@ -106,15 +109,26 @@ static int allocate_gsc_client_resources(struct xe_gt *gt,return PTR_ERR (vm);/* We allocate a single object for the batch and the in/out memory */ -xe_vm_lock (vm, false); - bo =xe_bo_create_pin_map (xe, tile, vm,PXP_BB_SIZE + inout_size *2 , - ttm_bo_type_kernel, -XE_BO_FLAG_SYSTEM |XE_BO_FLAG_PINNED |XE_BO_FLAG_NEEDS_UC ); -xe_vm_unlock (vm); -if (IS_ERR (bo)) { - err =PTR_ERR (bo); - goto vm_out; + + xe_validation_guard(&ctx, &xe->val, &exec, 0, err, false) { + err =xe_vm_drm_exec_lock (vm, &exec); +drm_exec_retry_on_contention (&exec); +if (err) +break ; + + bo =xe_bo_create_pin_map (xe, tile, vm,PXP_BB_SIZE + inout_size *2 , + ttm_bo_type_kernel, +XE_BO_FLAG_SYSTEM |XE_BO_FLAG_PINNED | +XE_BO_FLAG_NEEDS_UC , &exec); +drm_exec_retry_on_contention (&exec); +if (IS_ERR (bo)) { + err =PTR_ERR (bo); +xe_validation_retry_on_oom (&ctx, &err); +break ; + } } +if (err) + goto vm_out; fence =xe_vm_bind_kernel_bo (vm, bo,NULL ,0 ,XE_CACHE_WB );if (IS_ERR (fence)) {
drivers/gpu/drm/xe/xe_svm.c +30 -35
101 lines not on this page · raw message
drivers/gpu/drm/xe/xe_validation.c new file +248 -0
248 lines not on this page · raw message
drivers/gpu/drm/xe/xe_validation.h new file +176 -0
176 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm.c +156 -129
450 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.50.1
Part of the diff is not shown here. View the raw message.