Enable SVM atomics in Xe / GPU SVM
Subsystems (4)
Matthew Brost
maintainercover Enable SVM atomics in Xe / GPU SVM
Minimal set of patches to enable compute UMD SVM atomics.
Collaboration with Himal.
Sending as RFC to see if we should pursue merging this series ASAP, the solution of timeslicing may not be the final solution but it is quite simple as a stopgate / software enabling.
Matt
Himal Prasad Ghimiray (1):
drm/gpusvm: Introduce vram_only flag for VRAM allocation
Matthew Brost (3):
drm/xe: Strict migration policy for atomic SVM faults drm/xe: Timeslice GPU on atomic SVM fault drm/xe: Add atomic_svm_timeslice_ms debugfs entry
drivers/gpu/drm/drm_gpusvm.c | 14 +++++++ drivers/gpu/drm/xe/xe_debugfs.c | 38 ++++++++++++++++++ drivers/gpu/drm/xe/xe_device.c | 1 + drivers/gpu/drm/xe/xe_device_types.h | 3 ++ drivers/gpu/drm/xe/xe_svm.c | 58 +++++++++++++++++++++------- drivers/gpu/drm/xe/xe_svm.h | 5 --- include/drm/drm_gpusvm.h | 7 ++++ 7 files changed, 108 insertions(+), 18 deletions(-)
Signature · 1 line
2.34.1
Matthew Brost
maintainerRFC PATCH 1/4 drm/gpusvm: Introduce vram_only flag for VRAM allocation
From: Himal Prasad Ghimiray <[email protected]>
This commit adds a new flag, vram_only, to the drm_gpusvm structure. The purpose of this flag is to ensure that the get_pages function allocates memory exclusively from the device's VRAM. If the allocation from VRAM fails, the function will return an -EFAULT error.
--- drivers/gpu/drm/drm_gpusvm.c | 5 +++++ include/drm/drm_gpusvm.h | 2 ++ 2 files changed, 7 insertions(+)
drivers/gpu/drm/drm_gpusvm.c +5 -0
@@ -1454,6 +1454,11 @@ int drm_gpusvm_range_get_pages(struct drm_gpusvm *gpusvm, goto err_unmap; } +if (ctx->vram_only) { + err = -EFAULT ; + goto err_unmap; + } + addr = dma_map_page(gpusvm->drm->dev, page, 0, PAGE_SIZE << order,
include/drm/drm_gpusvm.h +2 -0
@@ -286,6 +286,7 @@ struct drm_gpusvm {* @in_notifier: entering from a MMU notifier * @read_only: operating on read-only memory * @devmem_possible: possible to use device memory +* @vram_only: Use only device memory * * Context that is DRM GPUSVM is operating in (i.e. user arguments). */
@@ -294,6 +295,7 @@ struct drm_gpusvm_ctx { unsigned int in_notifier :1; unsigned int read_only :1; unsigned int devmem_possible :1; + unsigned int vram_only :1; }; int drm_gpusvm_init(struct drm_gpusvm *gpusvm,
Signature · 1 line
2.34.1
Matthew Brost
maintainerRFC PATCH 2/4 drm/xe: Strict migration policy for atomic SVM faults
In reply to Matthew BrostMixing GPU and CPU atomics does not work unless a strict migration policy of GPU atomics must be device memory. Enforce a policy of must be in VRAM with a retry loop of 2 attempts, if retry loop fails abort fault.
--- drivers/gpu/drm/xe/xe_svm.c | 57 ++++++++++++++++++++++++++++--------- drivers/gpu/drm/xe/xe_svm.h | 5 ---- 2 files changed, 44 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_svm.c +44 -13
@@ -726,6 +726,35 @@ static int xe_svm_alloc_vram(struct xe_vm *vm, struct xe_tile *tile, }#endif +static bool supports_4K_migration (struct xe_device *xe) +{ +if (xe->info.vram_flags &XE_VRAM_FLAGS_NEED64K ) +return false; + +return true; +} + +static bool xe_svm_range_needs_migrate_to_vram (struct xe_svm_range *range, +struct xe_vma *vma) +{ +struct xe_vm *vm =range_to_vm (&range->base); +u64 range_size =xe_svm_range_size (range); + +if (!range->base.flags.migrate_devmem) +return false; + +if (xe_svm_range_in_vram (range)) { +drm_dbg (&vm->xe->drm,"Range is already in VRAM\n" ); +return false; + } + +if (range_size <=SZ_64K && !supports_4K_migration (vm->xe)) { +drm_dbg (&vm->xe->drm,"Platform doesn't support SZ_4K range migration\n" ); +return false; + } + +return true; +}/** * xe_svm_handle_pagefault() - SVM handle page fault
@@ -750,12 +779,14 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma,IS_ENABLED (CONFIG_DRM_XE_DEVMEM_MIRROR ), .check_pages_threshold = IS_DGFX(vm->xe) &&IS_ENABLED (CONFIG_DRM_XE_DEVMEM_MIRROR ) ?SZ_64K :0 , + .vram_only = atomic, };struct xe_svm_range *range;struct drm_gpusvm_range *r;struct drm_exec exec;struct dma_fence *fence;struct xe_tile *tile =gt_to_tile (gt); +int migrate_try_count =2 ;ktime_t end =0 ;int err;
@@ -782,18 +813,21 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma,range_debug (range,"PAGE FAULT" ); -/* XXX: Add migration policy, for now migrate range once */ -if (!range->skip_migrate && range->base.flags.migrate_devmem && -xe_svm_range_size (range) >=SZ_64K ) { - range->skip_migrate = true; - +if (--migrate_try_count >=0 && +xe_svm_range_needs_migrate_to_vram (range, vma)) { err =xe_svm_alloc_vram (vm, tile, range, &ctx);if (err) { -drm_dbg (&vm->xe->drm, -"VRAM allocation failed, falling back to " -"retrying fault, asid=%u, errno=%pe\n" , - vm->usm.asid,ERR_PTR (err)); - goto retry; +if (migrate_try_count || !ctx.vram_only) { +drm_dbg (&vm->xe->drm, +"VRAM allocation failed, falling back to retrying fault, asid=%u, errno=%pe\n" , + vm->usm.asid,ERR_PTR (err)); + goto retry; + }else { +drm_err (&vm->xe->drm, +"VRAM allocation failed, retry count exceeded, asid=%u, errno=%pe\n" , + vm->usm.asid,ERR_PTR (err)); +return err; + } } }
@@ -843,9 +877,6 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma, }drm_exec_fini (&exec); -if (xe_modparam.always_migrate_to_vram) - range->skip_migrate = false; -dma_fence_wait (fence, false);dma_fence_put (fence);
drivers/gpu/drm/xe/xe_svm.h +0 -5
@@ -39,11 +39,6 @@ struct xe_svm_range {* range. Protected by GPU SVM notifier lock. */ u8 tile_invalidated; -/** -* @skip_migrate: Skip migration to VRAM, protected by GPU fault handler -* locking. -*/ - u8 skip_migrate :1; };/**
Signature · 1 line
2.34.1
Matthew Brost
maintainerRFC PATCH 3/4 drm/xe: Timeslice GPU on atomic SVM fault
In reply to Matthew BrostEnsure GPU can make forward progress on an atomic SVM GPU fault by giving the GPU a timeslice of 10 MS.
--- drivers/gpu/drm/drm_gpusvm.c | 9 +++++++++ drivers/gpu/drm/xe/xe_svm.c | 1 + include/drm/drm_gpusvm.h | 5 +++++ 3 files changed, 15 insertions(+)
drivers/gpu/drm/drm_gpusvm.c +9 -0
@@ -1770,6 +1770,8 @@ int drm_gpusvm_migrate_to_devmem(struct drm_gpusvm *gpusvm, goto err_finalize;/* Upon success bind devmem allocation to range and zdd */ + devmem_allocation->timeslice_expiration =get_jiffies_64 () + +msecs_to_jiffies (ctx->timeslice_ms); zdd->devmem_allocation = devmem_allocation;/* Owns ref */ err_finalize:
@@ -1990,6 +1992,13 @@ static int __drm_gpusvm_migrate_to_ram(struct vm_area_struct *vas,void *buf;int i, err =0 ; +if (page) { + zdd = page->zone_device_data; +if (time_before64 (get_jiffies_64 (), + zdd->devmem_allocation->timeslice_expiration)) +return 0 ; + } + start =ALIGN_DOWN (fault_addr, size); end =ALIGN (fault_addr +1 , size);
drivers/gpu/drm/xe/xe_svm.c +1 -0
@@ -780,6 +780,7 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma, .check_pages_threshold = IS_DGFX(vm->xe) && IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR) ? SZ_64K : 0, .vram_only = atomic, + .timeslice_ms = atomic ? 10 : 0, };struct xe_svm_range *range;struct drm_gpusvm_range *r;
include/drm/drm_gpusvm.h +5 -0
@@ -89,6 +89,7 @@ struct drm_gpusvm_devmem_ops {* @ops: Pointer to the operations structure for GPU SVM device memory * @dpagemap: The struct drm_pagemap of the pages this allocation belongs to. * @size: Size of device memory allocation +* @timeslice_expiration: Timeslice expiration in jiffies */ struct drm_gpusvm_devmem {struct device *dev;
@@ -97,6 +98,7 @@ struct drm_gpusvm_devmem {const struct drm_gpusvm_devmem_ops *ops;struct drm_pagemap *dpagemap;size_t size; +u64 timeslice_expiration; };/**
@@ -283,6 +285,8 @@ struct drm_gpusvm {* @check_pages_threshold: Check CPU pages for present if chunk is less than or * equal to threshold. If not present, reduce chunk * size. +* @timeslice_ms: The timeslice MS which in minimum time a piece of memory +* remains with either exclusive GPU or CPU access. * @in_notifier: entering from a MMU notifier * @read_only: operating on read-only memory * @devmem_possible: possible to use device memory
@@ -292,6 +296,7 @@ struct drm_gpusvm {*/ struct drm_gpusvm_ctx {unsigned long check_pages_threshold; +unsigned long timeslice_ms;unsigned int in_notifier :1 ;unsigned int read_only :1 ;unsigned int devmem_possible :1 ;
Signature · 1 line
2.34.1
Matthew Brost
maintainerRFC PATCH 4/4 drm/xe: Add atomic_svm_timeslice_ms debugfs entry
In reply to Matthew BrostAdd some informal control for atomic SVM fault GPU timeslice to be able to play around with values and tweak performance.
--- drivers/gpu/drm/xe/xe_debugfs.c | 38 ++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_device.c | 1 + drivers/gpu/drm/xe/xe_device_types.h | 3 +++ drivers/gpu/drm/xe/xe_svm.c | 2 +- 4 files changed, 43 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_debugfs.c +38 -0
@@ -191,6 +191,41 @@ static const struct file_operations wedged_mode_fops = { .write = wedged_mode_set, }; +static ssize_t atomic_svm_timeslice_ms_show (struct file *f,char __user *ubuf, +size_t size,loff_t *pos) +{ +struct xe_device *xe =file_inode (f)->i_private; +char buf[32 ]; +int len =0 ; + + len =scnprintf (buf,sizeof (buf),"%d\n" , xe->atomic_svm_timeslice_ms); + +return simple_read_from_buffer (ubuf, size, pos, buf, len); +} + +static ssize_t atomic_svm_timeslice_ms_set (struct file *f, +const char __user *ubuf, +size_t size,loff_t *pos) +{ +struct xe_device *xe =file_inode (f)->i_private; +u32 atomic_svm_timeslice_ms; +ssize_t ret; + + ret =kstrtouint_from_user (ubuf, size,0 , &atomic_svm_timeslice_ms); +if (ret) +return ret; + + xe->atomic_svm_timeslice_ms = atomic_svm_timeslice_ms; + +return size; +} + +static const struct file_operations atomic_svm_timeslice_ms_fops = { + .owner =THIS_MODULE , + .read = atomic_svm_timeslice_ms_show, + .write = atomic_svm_timeslice_ms_set, +}; +void xe_debugfs_register (struct xe_device *xe) {struct ttm_device *bdev = &xe->ttm;
@@ -211,6 +246,9 @@ void xe_debugfs_register(struct xe_device *xe)debugfs_create_file ("wedged_mode" ,0600 , root, xe, &wedged_mode_fops); +debugfs_create_file ("atomic_svm_timeslice_ms" ,0600 , root, xe, + &atomic_svm_timeslice_ms_fops); +for (mem_type =XE_PL_VRAM0 ; mem_type <=XE_PL_VRAM1 ; ++mem_type) { man =ttm_manager_type (bdev, mem_type);
drivers/gpu/drm/xe/xe_device.c +1 -0
@@ -444,6 +444,7 @@ struct xe_device *xe_device_create(struct pci_dev *pdev, xe->info.devid = pdev->device; xe->info.revid = pdev->revision; xe->info.force_execlist = xe_modparam.force_execlist; + xe->atomic_svm_timeslice_ms =10 ; err =xe_irq_init (xe);if (err)
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -567,6 +567,9 @@ struct xe_device {/** @pmu: performance monitoring unit */ struct xe_pmu pmu; +/** @atomic_svm_timeslice_ms: Atomic SVM fault timeslice MS */ +u32 atomic_svm_timeslice_ms; +#ifdef TEST_VM_OPS_ERROR /** * @vm_inject_error_position: inject errors at different places in VM
drivers/gpu/drm/xe/xe_svm.c +1 -1
@@ -780,7 +780,7 @@ int xe_svm_handle_pagefault(struct xe_vm *vm, struct xe_vma *vma, .check_pages_threshold = IS_DGFX(vm->xe) && IS_ENABLED(CONFIG_DRM_XE_DEVMEM_MIRROR) ? SZ_64K : 0, .vram_only = atomic, - .timeslice_ms = atomic ? 10 : 0, + .timeslice_ms = atomic ? vm->xe->atomic_svm_timeslice_ms : 0, };struct xe_svm_range *range;struct drm_gpusvm_range *r;
Signature · 1 line
2.34.1
Patchwork
CI✓ Xe.CI.BAT: success for Enable SVM atomics in Xe / GPU SVM
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: Enable SVM atomics in Xe / GPU SVM
URL : https://patchwork.freedesktop.org/series/147846/
State : success
== Summary ==
CI Bug Log - changes from xe-2962-34472c052bf7c482187a422340b2b228d4d0fe69_BAT -> xe-pw-147846v1_BAT
====================================================
Summary
-------
**SUCCESS**
No regressions found.
Participating hosts (9 -> 8)
------------------------------
Missing (1): bat-adlp-vm
Changes
-------
No changes found
Build changes
-------------
* Linux: xe-2962-34472c052bf7c482187a422340b2b228d4d0fe69 -> xe-pw-147846v1
IGT_8323: 09df30b83439380ed9b7f76d31c38d08e31a450c @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
xe-2962-34472c052bf7c482187a422340b2b228d4d0fe69: 34472c052bf7c482187a422340b2b228d4d0fe69
xe-pw-147846v1: 147846v1
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-147846v1/index.html
Matthew Brost
maintainerQuoted 10 lines · Matthew Brost
Minimal set of patches to enable compute UMD SVM atomics.
Collaboration with Himal.
Sending as RFC to see if we should pursue merging this series ASAP, the solution of timeslicing may not be the final solution but it is quite simple as a stopgate / software enabling.
Matt
Something in series broke processes-evict-malloc-mix-bo, will need to fix that.
Matt
Quoted 20 lines
Himal Prasad Ghimiray (1):
drm/gpusvm: Introducevram_onlyflag for VRAM allocationMatthew Brost (3):
drm/xe: Strict migration policy for atomic SVM faults drm/xe: Timeslice GPU on atomic SVM fault drm/xe: Addatomic_svm_timeslice_msdebugfs entrydrivers/gpu/drm/drm_gpusvm.c | 14 +++++++ drivers/gpu/drm/xe/xe_debugfs.c | 38 ++++++++++++++++++ drivers/gpu/drm/xe/xe_device.c | 1 + drivers/gpu/drm/xe/xe_device_types.h | 3 ++ drivers/gpu/drm/xe/xe_svm.c | 58 +++++++++++++++++++++------- drivers/gpu/drm/xe/xe_svm.h | 5 --- include/drm/drm_gpusvm.h | 7 ++++ 7 files changed, 108 insertions(+), 18 deletions(-)Signature · 1 line
2.34.1
Matthew Brost
maintainerQuoted 9 lines · Matthew Brost
Quoted 8 lines · Matthew Brost
Minimal set of patches to enable compute UMD SVM atomics.
Collaboration with Himal.
Sending as RFC to see if we should pursue merging this series ASAP, the solution of timeslicing may not be the final solution but it is quite simple as a stopgate / software enabling.
Quoted 8 lines
Quoted 2 lines
Matt
Something in series broke processes-evict-malloc-mix-bo, will need to fix that.
Matt
Nevermind, this series [1] is actually causing problems.
Matt
[1] https://patchwork.freedesktop.org/series/147849/
Quoted 20 lines
Quoted 20 lines
Himal Prasad Ghimiray (1):
drm/gpusvm: Introducevram_onlyflag for VRAM allocationMatthew Brost (3):
drm/xe: Strict migration policy for atomic SVM faults drm/xe: Timeslice GPU on atomic SVM fault drm/xe: Addatomic_svm_timeslice_msdebugfs entrydrivers/gpu/drm/drm_gpusvm.c | 14 +++++++ drivers/gpu/drm/xe/xe_debugfs.c | 38 ++++++++++++++++++ drivers/gpu/drm/xe/xe_device.c | 1 + drivers/gpu/drm/xe/xe_device_types.h | 3 ++ drivers/gpu/drm/xe/xe_svm.c | 58 +++++++++++++++++++++------- drivers/gpu/drm/xe/xe_svm.h | 5 --- include/drm/drm_gpusvm.h | 7 ++++ 7 files changed, 108 insertions(+), 18 deletions(-)Signature · 1 line
2.34.1