Skip to contentSkip to conversation

drm/amdgpu: fix potential VM faults

PATCH In review Christian König 4 participants 6 messages Sep 19, 2019 – Sep 25, 2019 3 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdgpu: fix potential VM faults

When we allocate new page tables under memory
pressure we should not evict old ones.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
index 70d45d48907a..8e44ecaada35 100644
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 		.interruptible = (bp->type != ttm_bo_type_kernel),
 		.no_wait_gpu = bp->no_wait_gpu,
 		.resv = bp->resv,
-		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+		.flags = bp->type != ttm_bo_type_kernel ?
+			TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
 	};
 	struct amdgpu_bo *bo;
 	unsigned long page_align, size = bp->size;
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20190919084136.82658-1-christian.koenig-5C7GfCeVMHo@public.gmane.org

#2 lore raw

________________________________
From: amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org> on behalf of Christian König <[email protected]>
Sent: Thursday, September 19, 2019 4:41 AM
To: [email protected] <[email protected]>
Subject: [PATCH] drm/amdgpu: fix potential VM faults

When we allocate new page tables under memory
pressure we should not evict old ones.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
index 70d45d48907a..8e44ecaada35 100644
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
                 .interruptible = (bp->type != ttm_bo_type_kernel),
                 .no_wait_gpu = bp->no_wait_gpu,
                 .resv = bp->resv,
-               .flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+               .flags = bp->type != ttm_bo_type_kernel ?
+                       TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
         };
         struct amdgpu_bo *bo;
         unsigned long page_align, size = bp->size;

--
2.17.1

_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#3 lore raw
In reply to a message not in the archive

I'm not disagreeing with the change. Just trying to understand how this could have caused a VM fault. If the page tables are reserved or fenced while you allocate a new one, they would not be evicted. If they are not reserved or fenced, there should be no expectation that they stay resident.

Is this related to recoverable page fault handling? Do we need some more generic way to handle eviction of page tables and update the parent page directory (invalidate the corresponding PDE)?

Regards,
  Felix

Quoted 13 lines · Christian König

When we allocate new page tables under memory
pressure we should not evict old ones.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
index 70d45d48907a..8e44ecaada35 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c static int amdgpu_bo_do_create(struct amdgpu_device *adev, · line 521

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
  		.interruptible = (bp->type != ttm_bo_type_kernel),
  		.no_wait_gpu = bp->no_wait_gpu,
  		.resv = bp->resv,
-		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+		.flags = bp->type != ttm_bo_type_kernel ?
+			TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
  	};
  	struct amdgpu_bo *bo;
  	unsigned long page_align, size = bp->size;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): fd47f3c6-a336-5a02-6ff2-3c24ad81dcb9-5C7GfCeVMHo@public.gmane.org

#5 lore raw
Quoted 1 line

If the page tables are reserved or fenced while you allocate a new one, they would not be evicted.

And exactly that's not correct. The TTM_OPT_FLAG_ALLOW_RES_EVICT flag allows evicting of reserved objects.

This is useful for allocating per VM BOs, but is of course completely fatal in all other cases.

Regards,
Christian.

Am 19.09.19 um 16:29 schrieb Kuehling, Felix:

Quoted 21 lines

I'm not disagreeing with the change. Just trying to understand how this could have caused a VM fault. If the page tables are reserved or fenced while you allocate a new one, they would not be evicted. If they are not reserved or fenced, there should be no expectation that they stay resident.

Is this related to recoverable page fault handling? Do we need some more generic way to handle eviction of page tables and update the parent page directory (invalidate the corresponding PDE)?

Regards,
  Felix

Quoted 8 lines · Christian König

When we allocate new page tables under memory
pressure we should not evict old ones.

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
   1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 14 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
index 70d45d48907a..8e44ecaada35 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c static int amdgpu_bo_do_create(struct amdgpu_device *adev, · line 521

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
   		.interruptible = (bp->type != ttm_bo_type_kernel),
   		.no_wait_gpu = bp->no_wait_gpu,
   		.resv = bp->resv,
-		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+		.flags = bp->type != ttm_bo_type_kernel ?
+			TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
   	};
   	struct amdgpu_bo *bo;
   	unsigned long page_align, size = bp->size;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#6 lore raw
In reply to a message not in the archive

Hi Christian

Theoretically the vm pt/pd should be allowed to be evicted like other BOs ..

If you encountered page fault and could be avoided by this patch, that means there is bug in the VM/ttm system , and your patch simply

w/a the root cause.

_____________________________________
Monk Liu|GPU Virtualization Team |AMD

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Christian K?nig
Sent: Thursday, September 19, 2019 4:42 PM
To: [email protected]
Subject: [PATCH] drm/amdgpu: fix potential VM faults

When we allocate new page tables under memory pressure we should not evict old ones.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -1
index 70d45d48907a..8e44ecaada35 100644
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 		.interruptible = (bp->type != ttm_bo_type_kernel),
 		.no_wait_gpu = bp->no_wait_gpu,
 		.resv = bp->resv,
-		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+		.flags = bp->type != ttm_bo_type_kernel ?
+			TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
 	};
 	struct amdgpu_bo *bo;
 	unsigned long page_align, size = bp->size;

--
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): MN2PR12MB393312A21B75620C726F18DF84870-rweVpJHSKTq/[email protected]

#8 lore raw

Hi Monk,

this patch doesn't prevents PD/PT eviction.

The intention of the code here was that per VM BOs can evict other per VM BOs during allocation.

The problem my patch fixes is that this unfortunately also meant that allocation PDs/PTs could evict other PDs/PTs from the same process.

Regards,
Christian.

Am 25.09.19 um 15:51 schrieb Liu, Monk:

Quoted 42 lines

Hi Christian

Theoretically the vm pt/pd should be allowed to be evicted like other BOs ..

If you encountered page fault and could be avoided by this patch, that means there is bug in the VM/ttm system , and your patch simply

w/a the root cause.

_____________________________________
Monk Liu|GPU Virtualization Team |AMD

-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Christian K?nig
Sent: Thursday, September 19, 2019 4:42 PM
To: [email protected]
Subject: [PATCH] drm/amdgpu: fix potential VM faults

When we allocate new page tables under memory pressure we should not evict old ones.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -2
index 70d45d48907a..8e44ecaada35 100644
@@ -514,7 +514,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
  		.interruptible = (bp->type != ttm_bo_type_kernel),
  		.no_wait_gpu = bp->no_wait_gpu,
  		.resv = bp->resv,
-		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+		.flags = bp->type != ttm_bo_type_kernel ?
+			TTM_OPT_FLAG_ALLOW_RES_EVICT : 0
  	};
  	struct amdgpu_bo *bo;
  	unsigned long page_align, size = bp->size;
--

2.17.1

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx