Skip to contentSkip to conversation

drm/xe: Allow atomic operations in pages for Xe2

PATCH In review José Roberto de Souza 3 participants 4 messages Oct 4, 2023 – Oct 5, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/xe: Allow atomic operations in pages for Xe2

Without this if a atomic operations is executed it causes engine memory catastrophic error.

This bit was already being added in the pte_encode_vma() path but not in the pde_encode_bo() path.

This fixes at least the 3 failures in piglit sanity and 2 failures in crucible.

---
 drivers/gpu/drm/xe/xe_vm.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/gpu/drm/xe/xe_vm.c +3 -0
index ea17d026546cd..6be5d92706932 100644
@@ -1261,6 +1261,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,
 	pte |= pte_encode_cache(xe, cache);
 	pte |= pte_encode_ps(pt_level);
 
+	if (GRAPHICS_VERx100(xe) >= 20)
+		pte |= XE_USM_PPGTT_PTE_AE;
+
 	if (xe_bo_is_vram(bo) || xe_bo_is_stolen_devmem(bo))
 		pte |= XE_PPGTT_PTE_DM;
 
Signature · 1 line
2.42.0
#1 lore raw
Quoted 20 lines · Jose Souza

Without this if a atomic operations is executed it causes engine memory catastrophic error.

This bit was already being added in the pte_encode_vma() path but not in the pde_encode_bo() path.

This fixes at least the 3 failures in piglit sanity and 2 failures in crucible.

---
drivers/gpu/drm/xe/xe_vm.c | 3 +++

1 file changed, 3 insertions(+)
drivers/gpu/drm/xe/xe_vm.c +0 -0
index ea17d026546cd..6be5d92706932 100644

drivers/gpu/drm/xe/xe_vm.c static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset, · line 1265

Quoted 5 lines
drivers/gpu/drm/xe/xe_vm.c +2 -0
@@ -1261,6 +1261,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,
 	pte |= pte_encode_cache(xe, cache);
 	pte |= pte_encode_ps(pt_level);
 
+	if (GRAPHICS_VERx100(xe) >= 20)
+		pte |= XE_USM_PPGTT_PTE_AE;

I think there's a clash with doing this here and doing it where it was done in commit 6193a2ba4704 ("drm/xe: set PTE_AE for all platforms supporting it")

There it's adding the flag in the gpuva.flags which is later added to the pte. However, in xe_pt_stage_bind() that checks for XE_VMA_ATOMIC_PTE_BIT is only inside doing that for devmem. I was thinking that I might had screwed it with my recent refactors, but looking at the log it was always like that and the fix in the commit above didn't fix it for *all* the platforms.

Does it fix it if you simply change that function to consider system memory? Otherwise we should probably change it only in the pte encode instead of doing it in both places.

+Fei, +Matt Brost, +Matt Roper

Lucas De Marchi

Quoted 7 lines
drivers/gpu/drm/xe/xe_vm.c +1 -0
+
 	if (xe_bo_is_vram(bo) || xe_bo_is_stolen_devmem(bo))
 		pte |= XE_PPGTT_PTE_DM;
 
Signature · 1 line
2.42.0
#2 lore raw
Quoted 37 lines · Lucas De Marchi
Quoted 20 lines · Jose Souza

Without this if a atomic operations is executed it causes engine memory catastrophic error.

This bit was already being added in the pte_encode_vma() path but not in the pde_encode_bo() path.

This fixes at least the 3 failures in piglit sanity and 2 failures in crucible.

---
drivers/gpu/drm/xe/xe_vm.c | 3 +++

1 file changed, 3 insertions(+)
drivers/gpu/drm/xe/xe_vm.c +0 -0
index ea17d026546cd..6be5d92706932 100644

drivers/gpu/drm/xe/xe_vm.c static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset, · line 1265

Quoted 5 lines
drivers/gpu/drm/xe/xe_vm.c +2 -0
@@ -1261,6 +1261,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,
 	pte |= pte_encode_cache(xe, cache);
 	pte |= pte_encode_ps(pt_level);
 
+	if (GRAPHICS_VERx100(xe) >= 20)
+		pte |= XE_USM_PPGTT_PTE_AE;

I think there's a clash with doing this here and doing it where it was done in commit 6193a2ba4704 ("drm/xe: set PTE_AE for all platforms supporting it")

There it's adding the flag in the gpuva.flags which is later added to the pte. However, in xe_pt_stage_bind() that checks for XE_VMA_ATOMIC_PTE_BIT is only inside doing that for devmem. I was thinking that I might had screwed it with my recent refactors, but looking at the log it was always like that and the fix in the commit above didn't fix it for *all* the platforms.

Quoted 3 lines

Does it fix it if you simply change that function to consider system memory? Otherwise we should probably change it only in the pte encode instead of doing it in both places.

If we set this bit on system memory aren't we going to stop getting page faults? My understanding was that we need to migrate from smem->vram before performing GPU atomic operations, which is why we leave that bit clear on sram pages today.

Matt

Quoted 12 lines

+Fei, +Matt Brost, +Matt Roper

Lucas De Marchi

Quoted 7 lines
drivers/gpu/drm/xe/xe_vm.c +1 -0
+
 	if (xe_bo_is_vram(bo) || xe_bo_is_stolen_devmem(bo))
 		pte |= XE_PPGTT_PTE_DM;
 
Signature · 1 line
2.42.0
Signature · 4 lines
Matt Roper
Graphics Software Engineer
Linux GPU Platform Enablement
Intel Corporation
#3 lore raw
Quoted 38 lines · Matt Roper
Quoted 37 lines · Lucas De Marchi
Quoted 20 lines · Jose Souza

Without this if a atomic operations is executed it causes engine memory catastrophic error.

This bit was already being added in the pte_encode_vma() path but not in the pde_encode_bo() path.

This fixes at least the 3 failures in piglit sanity and 2 failures in crucible.

---
drivers/gpu/drm/xe/xe_vm.c | 3 +++

1 file changed, 3 insertions(+)
drivers/gpu/drm/xe/xe_vm.c +0 -0
index ea17d026546cd..6be5d92706932 100644

drivers/gpu/drm/xe/xe_vm.c static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset, · line 1265

Quoted 5 lines
drivers/gpu/drm/xe/xe_vm.c +2 -0
@@ -1261,6 +1261,9 @@ static u64 xelp_pte_encode_bo(struct xe_bo *bo, u64 bo_offset,
 	pte |= pte_encode_cache(xe, cache);
 	pte |= pte_encode_ps(pt_level);
 
+	if (GRAPHICS_VERx100(xe) >= 20)
+		pte |= XE_USM_PPGTT_PTE_AE;

I think there's a clash with doing this here and doing it where it was done in commit 6193a2ba4704 ("drm/xe: set PTE_AE for all platforms supporting it")

There it's adding the flag in the gpuva.flags which is later added to the pte. However, in xe_pt_stage_bind() that checks for XE_VMA_ATOMIC_PTE_BIT is only inside doing that for devmem. I was thinking that I might had screwed it with my recent refactors, but looking at the log it was always like that and the fix in the commit above didn't fix it for *all* the platforms.

Quoted 8 lines
Quoted 3 lines

Does it fix it if you simply change that function to consider system memory? Otherwise we should probably change it only in the pte encode instead of doing it in both places.

If we set this bit on system memory aren't we going to stop getting page faults? My understanding was that we need to migrate from smem->vram before performing GPU atomic operations, which is why we leave that bit clear on sram pages today.

I don't see a limitation on it being devmem only. See bspec 68148, "Atomic Destination Memory" - Destination memory can be Shared Local Memory (SLM), local device memory, a remote device memory or System memory.

Looking at previous implementations I'm not sure the handling for "this only works in devmem" was correct and I'd think it's because the feature started with PVC.

Lucas De Marchi