Skip to contentSkip to conversation

drm/amdkfd: ignore userptr NUMA auto balancing event

PATCH In review Philip Yang 2 participants 2 messages Aug 19, 2020 – Aug 21, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdkfd: ignore userptr NUMA auto balancing event

NUMA auto balancing task_numa_work periodically change memory range protection proto, in order to trigger CPU page hinting fault when CPU access the memory later to check if that memory range need to migrate to different NUMA node.

GPU can still access the memory after NUMA change the proto because the memory is not moved yet, so it is safe to ignore the MMU notifier event MMU_NOTIFY_PROTECTION_VMA to avoid unnecessary queue eviction and restore.

Before NUMA auto balancing actually move the memory to different NUMA node, driver will receive MMU notifier event MMU_NOTIFY_CLEAR to evict the queues.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +2 -0
index 828b5167ff12..dc4e2317dc31 100644
@@ -106,6 +106,8 @@ static bool amdgpu_mn_invalidate_hsa(struct mmu_interval_notifier *mni,
 
 	if (!mmu_notifier_range_blockable(range))
 		return false;
+	if (range->event == MMU_NOTIFY_PROTECTION_VMA)
+		return true;
 
 	mutex_lock(&adev->notifier_lock);
 
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#1 lore raw

MMU_NOTIFY_PROTECTION_VMA is not specific to NUMA auto-balancing. It can also be the result of an mprotect system call which actually makes the VMA read-only. I don't think it's OK to ignore that notifier in the general case.

Regards,
  Felix

Am 2020-08-19 um 2:00 p.m. schrieb Philip Yang:

Quoted 24 lines

NUMA auto balancing task_numa_work periodically change memory range protection proto, in order to trigger CPU page hinting fault when CPU access the memory later to check if that memory range need to migrate to different NUMA node.

GPU can still access the memory after NUMA change the proto because the memory is not moved yet, so it is safe to ignore the MMU notifier event MMU_NOTIFY_PROTECTION_VMA to avoid unnecessary queue eviction and restore.

Before NUMA auto balancing actually move the memory to different NUMA node, driver will receive MMU notifier event MMU_NOTIFY_CLEAR to evict the queues.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +0 -0
index 828b5167ff12..dc4e2317dc31 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c static bool amdgpu_mn_invalidate_hsa(struct mmu_interval_notifier *mni, · line 113

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c +2 -0
@@ -106,6 +106,8 @@ static bool amdgpu_mn_invalidate_hsa(struct mmu_interval_notifier *mni,
 
 	if (!mmu_notifier_range_blockable(range))
 		return false;
+	if (range->event == MMU_NOTIFY_PROTECTION_VMA)
+		return true;
 
 	mutex_lock(&adev->notifier_lock);
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx