drm/amdkfd: Unmap SVM range from GPU set to no-access
Subsystems (3)
v3 cover drm/amdkfd: Unmap SVM range from GPU set to no-access
Setting KFD_IOCTL_SVM_ATTR_NO_ACCESS on a GPU used to do nothing: the range stayed mapped, so the MMU notifier kept evicting GPU queues on CPU page faults even though the app wanted no access from that GPU.
Patch 1 adds a helper to update the fault checkpoint timestamp and makes the timestamp atomic so the page fault handler can read it locklessly.
Patch 2 unmaps the range from each GPU set to no-access. Once no GPU maps the range, the MMU notifier skips queue eviction. It bumps the checkpoint timestamp so retry faults queued before the unmap are dropped.
v3:
- Patch 1: fix checkpoint_ts atomic double-read race
- Patch 2: move checkpoint timestamp update into svm_range_needs_unmap
Philip Yang (2):
drm/amdkfd: Add helper svm_range_update_checkpoint_timestamp drm/amdkfd: Unmap svm range from GPU set to no-access
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 5 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 191 +++++++++++++++---------- drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 7 +- 4 files changed, 126 insertions(+), 79 deletions(-)
Signature · 1 line
2.50.1
PATCH v3 1/2 drm/amdkfd: Add helper svm_range_update_checkpoint_timestamp
Extract svm_range_update_checkpoint_timestamp() from svm_range_unmap_from_cpu(). The next patch calls it when the app sets the no-access attribute.
Change checkpoint_ts in svm_range_list from uint64_t to atomic64_t so svm_range_restore_pages() can read it from the page fault handler without holding the svms lock.
No functional change, preparation for the next patch.
--- drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 85 +++++++++++++++------------ 2 files changed, 50 insertions(+), 37 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1 -1
@@ -895,7 +895,7 @@ struct svm_range_list {DECLARE_BITMAP (bitmap_supported,MAX_GPU_INSTANCE );struct task_struct *faulting_task;/* check point ts decides if page fault recovery need be dropped */ -uint64_t checkpoint_ts[MAX_GPU_INSTANCE ]; +atomic64_t checkpoint_ts[MAX_GPU_INSTANCE ];/* Default granularity to use in buffer migration * and restoration of backing memory while handling
drivers/gpu/drm/amd/amdkfd/kfd_svm.c +49 -36
122 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH v3 2/2 drm/amdkfd: Unmap svm range from GPU set to no-access
In reply to Philip YangWhen KFD_IOCTL_SVM_ATTR_NO_ACCESS is applied to a GPU that has an SVM range mapped, unmap the range from that GPU. Once no GPU maps the range, the MMU notifier can skip queue eviction on CPU page faults.
Replace the mapped_to_gpu boolean with bitmap_mapped to track which GPUs currently have the range mapped. Set bits in svm_range_map_to_gpus() and clear them in svm_range_unmap_from_gpus(). This is separate from bitmap_access/bitmap_aip which track user-requested attributes and must not be used to determine mapping state.
Add bitmap_needs_unmap to svm_range, set for each GPU given no-access. Add svm_range_needs_unmap() to unmap the range from those GPUs when the app sets the no-access attribute.
Bump the checkpoint timestamp on unmap so retry faults queued before the no-access unmap are dropped instead of restoring the mapping.
v4:
- Rename and set prange->mapping_done to false if validate and map not complete successfully (Felix)
v3:
- Correct error handling, support app retry update mapping (Felix)
v2:
- Add bitmap_mapped to not break get_attr (Felix)
--- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 5 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 106 ++++++++++++++++--------- drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 7 +- 3 files changed, 76 insertions(+), 42 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_queue.c +2 -3
@@ -112,12 +112,11 @@ static int kfd_queue_buffer_svm_get(struct kfd_process_device *pdd, u64 addr, u6if (!prange)break ; -if (!prange->mapped_to_gpu) -break ; - r =kfd_process_gpuid_from_node (p, pdd->dev, &gpuid, &gpuidx);if (r <0 )break ; +if (!test_bit (gpuidx, prange->bitmap_mapped)) +break ;if (!test_bit (gpuidx, prange->bitmap_access) && !test_bit (gpuidx, prange->bitmap_aip))break ;
drivers/gpu/drm/amd/amdkfd/kfd_svm.c +68 -38
252 lines not on this page · raw message
drivers/gpu/drm/amd/amdkfd/kfd_svm.h +6 -1
@@ -99,6 +99,9 @@ struct svm_work_list_item {* @child_list: list header for split ranges which are not added to svms yet * @bitmap_access: index bitmap of GPUs which can access the range * @bitmap_aip: index bitmap of GPUs which can access the range in place +* @bitmap_needs_unmap: index bitmap of GPUs which currently set NO_ACCESS +* @bitmap_mapped: index bitmap of GPUs which currently have the range mapped +* @mapping_done: true if range_validate_and_map complete successfully * * Data structure for virtual memory range shared by CPU and GPUs, it can be * allocated from system memory ram or device vram, and migrate from ram to vram
@@ -134,7 +137,9 @@ struct svm_range {struct list_head child_list;DECLARE_BITMAP (bitmap_access,MAX_GPU_INSTANCE );DECLARE_BITMAP (bitmap_aip,MAX_GPU_INSTANCE ); -bool mapped_to_gpu; +DECLARE_BITMAP (bitmap_needs_unmap,MAX_GPU_INSTANCE ); +DECLARE_BITMAP (bitmap_mapped,MAX_GPU_INSTANCE ); +bool mapping_done;atomic_t queue_refcount; };
Signature · 1 line
2.50.1
Quoted 7 lines · Philip Yang
Setting
KFD_IOCTL_SVM_ATTR_NO_ACCESSon a GPU used to do nothing: the range stayed mapped, so the MMU notifier kept evicting GPU queues on CPU page faults even though the app wanted no access from that GPU.Patch 1 adds a helper to update the fault checkpoint timestamp and makes the timestamp atomic so the page fault handler can read it locklessly.
Quoted 7 lines
Patch 2 unmaps the range from each GPU set to no-access. Once no GPU maps the range, the MMU notifier skips queue eviction. It bumps the checkpoint timestamp so retry faults queued before the unmap are dropped.
v3:
- Patch 1: fixcheckpoint_tsatomic double-read race
- Patch 2: move checkpoint timestamp update intosvm_range_needs_unmap
The series is
Quoted 11 lines
Philip Yang (2):
drm/amdkfd: Add helpersvm_range_update_checkpoint_timestampdrm/amdkfd: Unmap svm range from GPU set to no-accessdrivers/gpu/drm/amd/amdkfd/kfd_priv.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_queue.c | 5 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 191 +++++++++++++++---------- drivers/gpu/drm/amd/amdkfd/kfd_svm.h | 7 +- 4 files changed, 126 insertions(+), 79 deletions(-)