drm/amdgpu: Improve retry fault handling (v4)
Subsystems (2)
cover drm/amdgpu: Improve retry fault handling (v4)
With this series, the kernel is able to mitigate most page faults on Navi 3 and 4 without causing a hang and without a need to reset the GPU, when the amdgpu.noretry=0 module parameter is set.
Fix various issues with retry fault handling.
Enable filter CAM on Navi 3 and 4 which is a HW block that filters the repeated page fault interrupts that happen when retry faults are enabled, making the handling more efficient.
Changes in v2:
* Reordered patches in the series to put bug fixes first
* Enable retry fault interrupt in init_system_aperture_regs() instead of in set_fault_enable_default()
* Added a patch to respect the noretry flag on GFX12.1 too
Changes in v3:
* This series now includes fixes for Navi 4 too which were previously submitted in a separate series.
* Dropped patch that improved handling of timestamps on the page fault interrupts. I'll come up with a different solution for that problem later.
* Dropped patches that solve race condition between retry fault handling and filter CAM ACK because they conflict with Christian's ongoing work. I'll revisit that later.
* Reworked the series into smaller patches for easier reviews.
* Addressed comments from Tvrtko's review. Thank you Tvrtko for taking the time to review!
Changes in v4:
* Program ENABLE_RETRY_FAULT_INTERRUPT bit to 1 and not according to the gmc.noretry flag, as requested by Alex and Christian.
* Applied Mukul's suggestion to use the doorbell for the IH retry CAM instead of the MMIO ACK.
* Dropped patch that enabled the retry CAM MMIO ACK.
* Dropped NOALLOC PTE flag as requested by Christian.
Timur Kristóf (11):
drm/amdgpu: Respect noretry flag for retry faults on GFX12.1 drm/amdgpu/gfxhub: Enable retry fault interrupts in
L2_PROTECTION_FAULT_CNTL2
drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of
PAGE_SIZE
drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of
PAGE_SIZE
drm/amdgpu/gmc11: Pass cam_index to retry fault handler drm/amdgpu/gmc12: Pass cam_index to retry fault handler drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on
GFX12.0
drm/amdgpu/vm: Use init PTE flags in amdgpu_vm_handle_fault() drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 ++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 2 +- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 ++++-- drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 +++- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 7 ++++- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 32 +++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 9 +++++- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 15 ++++++---- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 +++ drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c | 2 +- drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c | 2 +- 18 files changed, 121 insertions(+), 28 deletions(-)
Signature · 1 line
2.55.0
PATCH 1/11 drm/amdgpu: Respect noretry flag for retry faults on GFX12.1
When retry faults are disabled (amdgpu.noretry=1), we should program the RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit to zero.
Note that retry faults are enabled by default on GFX12.1 so this just fixes the case when they are explicitly disabled.
--- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +1 -1
@@ -436,7 +436,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,/* Send no-retry XNACK on fault to suppress VM fault storm */ tmp =REG_SET_FIELD (tmp,GCVM_CONTEXT1_CNTL ,RETRY_PERMISSION_OR_INVALID_PAGE_FAULT , -1 ); + !adev->gmc.noretry);WREG32_SOC15_OFFSET (GC ,GET_INST (GC , j), regGCVM_CONTEXT1_CNTL, i * hub->ctx_distance, tmp); WREG32_SOC15_OFFSET(GC, GET_INST(GC, j),
Signature · 1 line
2.55.0
Alex Deucher
AppliedRe: [PATCH 01/11] drm/amdgpu: Respect noretry flag for retry faults on GFX12.1
Applied the series. Thanks!
Alex
Quoted 30 lines · Timur Kristóf
When retry faults are disabled (amdgpu.noretry=1), we should program the
RETRY_PERMISSION_OR_INVALID_PAGE_FAULTbit to zero.Note that retry faults are enabled by default on GFX12.1 so this just fixes the case when they are explicitly disabled.
Signed-off-by Timur Kristóf <[email protected]>Reviewed-by Christian König <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +1
-2@@ -436,7 +436,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,/* Send no-retry XNACK on fault to suppress VM fault storm */ tmp =REG_SET_FIELD (tmp,GCVM_CONTEXT1_CNTL ,RETRY_PERMISSION_OR_INVALID_PAGE_FAULT , -1 ); + !adev->gmc.noretry);WREG32_SOC15_OFFSET (GC ,GET_INST (GC , j), regGCVM_CONTEXT1_CNTL, i * hub->ctx_distance, tmp); WREG32_SOC15_OFFSET(GC, GET_INST(GC, j), --2.55.0
PATCH 2/11 drm/amdgpu/gfxhub: Enable retry fault interrupts in L2_PROTECTION_FAULT_CNTL2
In reply to Timur KristófEnable retry fault interrupts when initializing the GFXHUB system aperture registers.
Needs to be done for each GFXHUB version, because none of them actually enabled this interrupt.
This bit gates all retry fault functionality on Navi GPUs. According to a mailing list conversation with AMD engineers, this register field should have been initialized to 1 according to hardware documentation. But it has been confirmed that it is in fact initialized to 0 by default.
I have personally confirmed on both Navi 31 and Navi 48 that the GPU does NOT send any retry fault interrupts without enabling this bit.
Initially I wanted to program this field according to the gmc.noretry flag in the previous versions of this patch, but I was asked to just program it always to 1 instead.
--- drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++++-- 8 files changed, 51 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +7 -2
@@ -155,6 +155,7 @@ static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v11_5_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BOT, adev->gmc.agp_start >>24 );
@@ -180,8 +181,12 @@ static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }static void gfxhub_v11_5_0_init_tlb_regs (struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +7 -2
@@ -158,6 +158,7 @@ static void gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v12_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );
@@ -184,8 +185,12 @@ static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +7 -2
@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v1_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev) || adev->asic_type <=CHIP_VEGA10 ) {/* Program the AGP BAR */
@@ -134,8 +135,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,VM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2, tmp); }/* In the case squeezing vram into GART aperture, we don't use
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +2 -0
@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct amdgpu_device *adev, tmp =RREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2); tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 );WREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +7 -2
@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev)) {/* Program the AGP BAR */
@@ -178,8 +179,12 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +7 -2
@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_1_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;
@@ -182,8 +183,12 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +7 -2
@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );
@@ -176,8 +177,12 @@ static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +7 -2
@@ -153,6 +153,7 @@ static void gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_3_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;
@@ -181,8 +182,12 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT ,1 ); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Signature · 1 line
2.55.0
PATCH 3/11 drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring
In reply to Timur KristófThe soft IH ring is implemented entirely in software. We shouldn't read (or write) any HW registers when accessing it.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ++++ 4 files changed, 25 insertions(+)
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7 -0
@@ -439,6 +439,10 @@ static u32 ih_v6_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -514,6 +518,9 @@ static void ih_v6_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;
drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +7 -0
@@ -410,6 +410,10 @@ static u32 ih_v6_1_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -481,6 +485,9 @@ static void ih_v6_1_irq_rearm(struct amdgpu_device *adev,static void ih_v6_1_set_rptr (struct amdgpu_device *adev,struct amdgpu_ih_ring *ih) { +if (ih == &adev->irq.ih_soft) +return ; +struct amdgpu_ih_regs *ih_regs;if (ih->use_doorbell) {
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +7 -0
@@ -460,6 +460,10 @@ static u32 ih_v7_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -530,6 +534,9 @@ static void ih_v7_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;
drivers/gpu/drm/amd/amdgpu/navi10_ih.c +4 -0
@@ -417,6 +417,10 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev,*/ wptr =le32_to_cpu (*ih->wptr_cpu); +if (ih == &adev->irq.ih_soft) + goto out; + +if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW )) goto out; }
Signature · 1 line
2.55.0
PATCH 4/11 drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
In reply to Timur KristófWhen there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow. PAGE_SIZE was too small, use IH_SW_RING_SIZE to match what other GPU generations are doing.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +1 -1
@@ -590,7 +590,7 @@ static int ih_v6_1_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v6_1_init_register_offset (adev); - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,PAGE_SIZE , true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
Signature · 1 line
2.55.0
PATCH 5/11 drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
In reply to Timur KristófWhen there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow. PAGE_SIZE was too small, use IH_SW_RING_SIZE to match what other GPU generations are doing.
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1 -4
@@ -599,7 +599,6 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)int r;struct amdgpu_device *adev = ip_block->adev;bool use_bus_addr; -unsigned int sw_ring_size; r =amdgpu_irq_add_id (adev,SOC21_IH_CLIENTID_IH ,0 , &adev->irq.self_irq);
@@ -631,9 +630,7 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v7_0_init_register_offset (adev); - sw_ring_size = (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) ? -IH_SW_RING_SIZE :PAGE_SIZE ; - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft, sw_ring_size, true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
Signature · 1 line
2.55.0
PATCH 6/11 drm/amdgpu/gmc11: Pass cam_index to retry fault handler
In reply to Timur KristófThis is necessary if we want to make use of the filter CAM.
--- drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +4 -1
@@ -108,13 +108,16 @@ static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr; addr = (u64 )entry->src_data[0 ] <<12 ; addr |= ((u64 )entry->src_data[1 ] &0xf ) <<44 ;if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
Signature · 1 line
2.55.0
PATCH 7/11 drm/amdgpu/gmc12: Pass cam_index to retry fault handler
In reply to Timur KristófThis is necessary if we want to make use of the filter CAM.
--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +4 -1
@@ -99,6 +99,7 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr; addr = (u64 )entry->src_data[0 ] <<12 ;
@@ -110,7 +111,9 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev, hub = &adev->vmhub[AMDGPU_GFXHUB (0 )];if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
Signature · 1 line
2.55.0
PATCH 8/11 drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on GFX12.0
In reply to Timur KristófThis flag seems to work around a "fault priority problem" and is necessary for handling faults on GFX12. The kernel seems unable to mitigate retry faults on GFX12 without this flag.
For reference see:amdgpu_vm_pte_update_flags() that explains the problem svm_range_get_pte_flags() that uses the flag on GFX12
--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +2 -0
@@ -644,6 +644,8 @@ static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block) adev->gmc.xgmi.connected_to_cpu = adev->smuio.funcs->is_host_gpu_xgmi_supported (adev); + adev->gmc.init_pte_flags =AMDGPU_PTE_IS_PTE ; +switch (amdgpu_ip_version (adev,GC_HWIP ,0 )) {case IP_VERSION (12 ,1 ,0 ):gmc_v12_1_set_gmc_funcs (adev);
Signature · 1 line
2.55.0
PATCH 9/11 drm/amdgpu/vm: Use init PTE flags in amdgpu_vm_handle_fault()
In reply to Timur KristófPTE_IS_PTE seems necessary for handling retry faults on GFX12.
For reference see:amdgpu_vm_pte_update_flags() that explains the problem svm_range_get_pte_flags() that uses the flag on GFX12
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +3 -3
@@ -3050,7 +3050,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, } addr /=AMDGPU_GPU_PAGE_SIZE ; - flags =AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | +AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED |AMDGPU_PTE_SYSTEM ;if (is_compute_context) {
@@ -3063,8 +3064,7 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE | -AMDGPU_PTE_WRITEABLE ; - +AMDGPU_PTE_WRITEABLE ; }else {/* Let the hw retry silently on the PTE */ value =0 ;
Signature · 1 line
2.55.0
PATCH 10/11 drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs
In reply to Timur KristófThe retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Enable the doorbell for the retry CAM.
Also change the doorbell range to actually enable the doorbell for the retry CAM.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 25 +++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c | 2 +- 2 files changed, 26 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +25 -0
@@ -307,6 +307,16 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,return 0 ; } +static uint32_t ih_v6_0_setup_retry_doorbell (u32 doorbell_index) +{ +u32 val =0 ; + + val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); + val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); + +return val; +} +/** * ih_v6_0_irq_init - init and enable the interrupt ring *
@@ -392,6 +402,21 @@ static int ih_v6_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); +if (!(adev->flags &AMD_IS_APU )) { +/* Allocate the doorbell for IH Retry CAM */ + adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih +2 ) <<1 ; +WREG32_SOC15 (OSSSYS ,0 , regIH_DOORBELL_RETRY_CAM, +ih_v6_0_setup_retry_doorbell (adev->irq.retry_cam_doorbell_index)); + +/* Enable IH Retry CAM */ + tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 ); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,CAM_SIZE ,0xF ); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL, tmp); + + adev->irq.retry_cam_enabled = true; + } +/* enable interrupts */ ret =ih_v6_0_toggle_interrupts (adev, true);if (ret)
drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c +1 -1
@@ -199,7 +199,7 @@ static void nbio_v4_3_ih_doorbell_range(struct amdgpu_device *adev, ih_doorbell_range =REG_SET_FIELD (ih_doorbell_range,S2A_DOORBELL_ENTRY_1_CTRL ,S2A_DOORBELL_PORT1_RANGE_SIZE , -2 ); +8 ); ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range, S2A_DOORBELL_ENTRY_1_CTRL, S2A_DOORBELL_PORT1_AWADDR_31_28_VALUE,
Signature · 1 line
2.55.0
PATCH 11/11 drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
In reply to Timur KristófThe retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Also change the doorbell range to actually enable the doorbell for the retry CAM.
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +2 -1
@@ -395,7 +395,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); -if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) { +if (!(adev->flags &AMD_IS_APU ) || +amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) {/* Allocate the doorbell for IH Retry CAM */ adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih +2 ) <<1 ; WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c +1 -1
@@ -300,7 +300,7 @@ static void nbif_v6_3_1_ih_doorbell_range(struct amdgpu_device *adev, ih_doorbell_range =REG_SET_FIELD (ih_doorbell_range,GDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL ,S2A_DOORBELL_PORT1_RANGE_SIZE , -2 ); +8 ); ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range, GDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL, S2A_DOORBELL_PORT1_AWADDR_31_28_VALUE,
Signature · 1 line
2.55.0