drm/amdgpu: Improve retry fault handling (v3)
Subsystems (2)
cover drm/amdgpu: Improve retry fault handling (v3)
With this series, the kernel is able to mitigate most page faults on Navi 3 and 4 without causing a hang and without a need to reset the GPU, when the amdgpu.noretry=0 module parameter is set.
Fix various issues with retry fault handling.
Enable filter CAM on Navi 3 and 4 which is a HW block that filters the repeated page fault interrupts that happen when retry faults are enabled, making the handling more efficient.
Changes in v2:
* Reordered patches in the series to put bug fixes first
* Enable retry fault interrupt in init_system_aperture_regs() instead of in set_fault_enable_default()
* Added a patch to respect the noretry flag on GFX12.1 too
Changes in v3:
* This series now includes fixes for Navi 4 too which were previously submitted in a separate series.
* Dropped patch that improved handling of timestamps on the page fault interrupts. I'll come up with a different solution for that problem later.
* Dropped patches that solve race condition between retry fault handling and filter CAM ACK because they conflict with Christian's ongoing work. I'll revisit that later.
* Reworked the series into smaller patches for easier reviews.
* Addressed comments from Tvrtko's review. Thank you Tvrtko for taking the time to review!
Timur Kristóf (14):
drm/amdgpu: Respect noretry flag for retry faults on GFX12.1 drm/amdgpu/gfxhub: Enable retry fault interrupts when needed drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring drm/amdgpu/ih: Add retry_cam_ack IH function pointer drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of
PAGE_SIZE
drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of
PAGE_SIZE
drm/amdgpu/gmc11: Pass cam_index to retry fault handler drm/amdgpu/gmc12: Pass cam_index to retry fault handler drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on
GFX12.0
drm/amdgpu/vm: Use init PTE flags and NOALLOC in
amdgpu_vm_handle_fault()
drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH
7.0drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 7 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 4 +-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++-- drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 +++- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 8 ++++-- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 25 ++++++++++++++++- drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 9 ++++++- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 30 +++++++++------------ drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 +++ drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 8 +++++- 20 files changed, 133 insertions(+), 43 deletions(-)
Signature · 1 line
2.54.0
PATCH 1/14 drm/amdgpu: Respect noretry flag for retry faults on GFX12.1
When retry faults are disabled (amdgpu.noretry=1), the ENABLE_RETRY_FAULT_INTERRUPT bit should be programmed to 0.
Note that retry faults are enabled by default on GFX12.1 so this just fixes the case when they are explicitly disabled.
--- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 4 ++-- drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +2 -2
@@ -243,7 +243,7 @@ static void gfxhub_v12_1_xcc_init_system_aperture_regs(struct amdgpu_device *ade tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ENABLE_RETRY_FAULT_INTERRUPT ,0x1 ); +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (GC ,GET_INST (GC , i), regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
@@ -447,7 +447,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,/* Send no-retry XNACK on fault to suppress VM fault storm */ tmp =REG_SET_FIELD (tmp,GCVM_CONTEXT1_CNTL ,RETRY_PERMISSION_OR_INVALID_PAGE_FAULT , -1 ); + !adev->gmc.noretry);WREG32_SOC15_OFFSET (GC ,GET_INST (GC , j), regGCVM_CONTEXT1_CNTL, i * hub->ctx_distance, tmp); WREG32_SOC15_OFFSET(GC, GET_INST(GC, j),
drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c +1 -1
@@ -320,7 +320,7 @@ static void mmhub_v4_2_0_mid_init_system_aperture_regs(struct amdgpu_device *ade tmp =REG_SET_FIELD (tmp,MMVM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); tmp =REG_SET_FIELD (tmp,MMVM_L2_PROTECTION_FAULT_CNTL2 , -ENABLE_RETRY_FAULT_INTERRUPT ,0x1 ); +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (MMHUB ,GET_INST (MMHUB , i), regMMVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Signature · 1 line
2.54.0
Re: [PATCH 01/14] drm/amdgpu: Respect noretry flag for retry faults on GFX12.1
Quoted 7 lines · Timur Kristóf
When retry faults are disabled (amdgpu.noretry=1), the
ENABLE_RETRY_FAULT_INTERRUPTbit should be programmed to 0.Note that retry faults are enabled by default on GFX12.1 so this just fixes the case when they are explicitly disabled.
Signed-off-by Timur Kristóf <[email protected]>
Quoted 40 lines
--- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 4 ++-- drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c +2
-2@@ -243,7 +243,7 @@ static void gfxhub_v12_1_xcc_init_system_aperture_regs(struct amdgpu_device *ade tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ENABLE_RETRY_FAULT_INTERRUPT ,0x1 ); +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (GC ,GET_INST (GC , i), regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }@@ -447,7 +447,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,/* Send no-retry XNACK on fault to suppress VM fault storm */ tmp =REG_SET_FIELD (tmp,GCVM_CONTEXT1_CNTL ,RETRY_PERMISSION_OR_INVALID_PAGE_FAULT , -1 ); + !adev->gmc.noretry);WREG32_SOC15_OFFSET (GC ,GET_INST (GC , j), regGCVM_CONTEXT1_CNTL, i * hub->ctx_distance, tmp); WREG32_SOC15_OFFSET(GC, GET_INST(GC, j),drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c +1
-1@@ -320,7 +320,7 @@ static void mmhub_v4_2_0_mid_init_system_aperture_regs(struct amdgpu_device *ade tmp =REG_SET_FIELD (tmp,MMVM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); tmp =REG_SET_FIELD (tmp,MMVM_L2_PROTECTION_FAULT_CNTL2 , -ENABLE_RETRY_FAULT_INTERRUPT ,0x1 ); +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (MMHUB ,GET_INST (MMHUB , i), regMMVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
PATCH 2/14 drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
In reply to Timur KristófEnable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
--- drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++++-- 8 files changed, 51 insertions(+), 14 deletions(-)
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +7 -2
@@ -155,6 +155,7 @@ static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v11_5_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BOT, adev->gmc.agp_start >>24 );
@@ -180,8 +181,12 @@ static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }static void gfxhub_v11_5_0_init_tlb_regs (struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +7 -2
@@ -158,6 +158,7 @@ static void gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v12_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );
@@ -184,8 +185,12 @@ static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +7 -2
@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v1_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev) || adev->asic_type <=CHIP_VEGA10 ) {/* Program the AGP BAR */
@@ -134,8 +135,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,VM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2, tmp); }/* In the case squeezing vram into GART aperture, we don't use
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +2 -0
@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct amdgpu_device *adev, tmp =RREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2); tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +7 -2
@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev)) {/* Program the AGP BAR */
@@ -178,8 +179,12 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +7 -2
@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_1_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;
@@ -182,8 +183,12 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +7 -2
@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );
@@ -176,8 +177,12 @@ static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +7 -2
@@ -153,6 +153,7 @@ static void gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_3_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;
@@ -181,8 +182,12 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Signature · 1 line
2.54.0
Christian König
maintainerRe: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
Quoted 6 lines · Timur Kristóf
Enable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.
IIRC only the RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit in the VM_CONTEXT0_CNTL register should be set or cleared by the kernel driver or firmware to control if the HW retries the access or not.
Regards,
Christian.
Quoted 215 lines
Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++++-- 8 files changed, 51 insertions(+), 14 deletions(-)drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +7
-2@@ -155,6 +155,7 @@ static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v11_5_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BOT, adev->gmc.agp_start >>24 );@@ -180,8 +181,12 @@ static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }static void gfxhub_v11_5_0_init_tlb_regs (struct amdgpu_device *adev)drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +7
-2@@ -158,6 +158,7 @@ static void gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v12_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -184,8 +185,12 @@ static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +7
-2@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v1_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev) || adev->asic_type <=CHIP_VEGA10 ) {/* Program the AGP BAR */ @@ -134,8 +135,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,VM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmVM_L2_PROTECTION_FAULT_CNTL2, tmp); }/* In the case squeezing vram into GART aperture, we don't use drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +2
-0@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct amdgpu_device *adev, tmp =RREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2); tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 ,ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,VM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry);WREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2, tmp); }drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +7
-2@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev)) {/* Program the AGP BAR */ @@ -178,8 +179,12 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +7
-2@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v2_1_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -182,8 +183,12 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +7
-2@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -176,8 +177,12 @@ static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +7
-2@@ -153,6 +153,7 @@ static void gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)static void gfxhub_v3_0_3_init_system_aperture_regs (struct amdgpu_device *adev) {uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -181,8 +182,12 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32, (u32)((u64 )adev->dummy_page_addr >>44 )); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =REG_SET_FIELD (tmp,GCVM_L2_PROTECTION_FAULT_CNTL2 , +ENABLE_RETRY_FAULT_INTERRUPT , !adev->gmc.noretry); +WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Re: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
Quoted 10 lines · On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian König…
Quoted 6 lines · Timur Kristóf
Enable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.
Why would it be enabled all the time?
I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling the ENABLE_RETRY_FAULT_INTERRUPT bit.
Quoted 4 lines
IIRC only the
RETRY_PERMISSION_OR_INVALID_PAGE_FAULTbit in theVM_CONTEXT0_CNTLregister should be set or cleared by the kernel driver or firmware to control if the HW retries the access or not.
That is clearly not the case on GFX12.1 and I haven't seen any indication that it would be different on GFX11.x and 12.0 either.
Quoted 22 lines
Quoted 22 lines
---
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++++-- 8 files changed, 51 insertions(+), 14 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.cindex652eea6eae4a..ef20eafd59ae100644drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +0
-0@@ -155,6 +155,7 @@ static void
gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev){
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +1
-0uint64_t value; +u32 tmp;WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 ); WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >>
24);
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +0
-0@@ -180,8 +181,12 @@ static void
gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }static void gfxhub_v11_5_0_init_tlb_regs (struct amdgpu_device *adev)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.cindex6cbf837d50dd..ec3ff4dec674100644drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +0
-0@@ -158,6 +158,7 @@ static void
gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +1
-0uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -184,8 +185,12 @@ static void
gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 18 lines
Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.cindexbfe247b1a333..27d7f7cb903f100644drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0uint64_t value; +u32 tmp;
Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0if (!amdgpu_sriov_vf (adev) || adev->asic_type <=CHIP_VEGA10 ) {/* Program the AGP BAR */ @@ -134,8 +135,12 @@ static void
gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,
mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 2 lines
Quoted 2 lines
(u32)((u64)adev->dummy_page_addr >>
44));
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-2- WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2, - ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY,
1);
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0+ tmp = RREG32_SOC15(GC, 0,
mmVM_L2_PROTECTION_FAULT_CNTL2);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2, +
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0+ WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2,
tmp);
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0}/* In the case squeezing vram into GART aperture, we don't use
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 10 lines
(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.cindexfbdf46070b38..ed9a64bc5aaa100644drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct
amdgpu_device*adev,>drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0tmp =RREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2); tmp = REG_SET_FIELD(tmp,
VM_L2_PROTECTION_FAULT_CNTL2,
Quoted 2 lines
Quoted 2 lines
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +1
-0+ tmp = REG_SET_FIELD(tmp,
VM_L2_PROTECTION_FAULT_CNTL2,
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +1
-0+
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
Quoted 2 lines
Quoted 2 lines
WREG32_SOC15(GC,
GET_INST(GC, i),
regVM_L2_PROTECTION_FAULT_CNTL2,
Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.cindex9ea593e2c719..152b2735d360100644drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +0
-0@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +1
-0uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev)) {/* Program the AGP BAR */ @@ -178,8 +179,12 @@ static void
gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +1
-0+WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.cindex30b90d35abd0..83c2ddbbd292100644drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +0
-0@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +1
-0uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -182,8 +183,12 @@ static void
gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 25 lines
Quoted 25 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +1
-0+WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.cindex9e6a6e13dec0..90bbb2fe4884100644drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +0
-0@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +1
-0uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -176,8 +177,12 @@ static void
gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.cindexb3b1085c7cd3..1b3c067ab48c100644drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +0
-0@@ -153,6 +153,7 @@ static void
gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +1
-0uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -181,8 +182,12 @@ static void
gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Christian König
maintainerRe: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
Quoted 10 lines · Timur Kristóf
Quoted 8 lines · On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian König…
Quoted 6 lines · Timur Kristóf
Enable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
Quoted 6 lines
Quoted 2 lines
Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.
Why would it be enabled all the time?
I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling theENABLE_RETRY_FAULT_INTERRUPTbit.
As far as I understand it the GCVM_L2_PROTECTION_FAULT_CNTL2 is actually a debug register for the fault behavior of the GC block.
In other words you can for example turn on interrupts for PRT accesses as well to debug those.
According to the register spec for Navi 44 the ENABLE_RETRY_FAULT_INTERRUPT should be default 1 and always be set under normal cricumstances.
Quoted 7 lines
Quoted 4 lines
IIRC only the
RETRY_PERMISSION_OR_INVALID_PAGE_FAULTbit in theVM_CONTEXT0_CNTLregister should be set or cleared by the kernel driver or firmware to control if the HW retries the access or not.That is clearly not the case on GFX12.1 and I haven't seen any indication that it would be different on GFX11.x and 12.0 either.
Mhm, then either the FW or golden register settings for that register is not correct.
That we set ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY to one is rather strange as well since we don't use that feature in the Linux driver.
My suggestion is to always set the ENABLE_RETRY_FAULT_INTERRUPT in GCVM_L2_PROTECTION_FAULT_CNTL2 no matter what the noretry flag says, but I'm going to ask Alex and our ROCm people about that again when I have time.
Regards,
Christian.
Quoted 309 lines
Quoted 24 lines
Quoted 24 lines
---
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c | 2 ++ drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 9 +++++++-- drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c | 9 +++++++-- 8 files changed, 51 insertions(+), 14 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.cindex652eea6eae4a..ef20eafd59ae100644drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +0
-0@@ -155,6 +155,7 @@ static void
gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev){Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +1
-0uint64_t value; +u32 tmp;WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 ); WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >>24);
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +0
-0@@ -180,8 +181,12 @@ static void
gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 27 lines
Quoted 27 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }static void gfxhub_v11_5_0_init_tlb_regs (struct amdgpu_device *adev)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.cindex6cbf837d50dd..ec3ff4dec674100644drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +0
-0@@ -158,6 +158,7 @@ static void
gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +1
-0uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -184,8 +185,12 @@ static void
gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 18 lines
Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.cindexbfe247b1a333..27d7f7cb903f100644drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0uint64_t value; +u32 tmp;Quoted 7 lines
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0if (!amdgpu_sriov_vf (adev) || adev->asic_type <=CHIP_VEGA10 ) {/* Program the AGP BAR */ @@ -134,8 +135,12 @@ static void
gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,
mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 2 lines
Quoted 2 lines
(u32)((u64)adev->dummy_page_addr >>
44));
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-2- WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2, - ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY,1);
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0+ tmp = RREG32_SOC15(GC, 0,
mmVM_L2_PROTECTION_FAULT_CNTL2);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2, +
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +1
-0+ WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2,tmp);
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +0
-0}/* In the case squeezing vram into GART aperture, we don't use Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 10 lines
(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.cindexfbdf46070b38..ed9a64bc5aaa100644drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct
amdgpu_device*adev,>drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0tmp =RREG32_SOC15 (GC ,GET_INST (GC , i), regVM_L2_PROTECTION_FAULT_CNTL2); tmp = REG_SET_FIELD(tmp,
VM_L2_PROTECTION_FAULT_CNTL2,Quoted 2 lines
Quoted 2 lines
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +1
-0+ tmp = REG_SET_FIELD(tmp,
VM_L2_PROTECTION_FAULT_CNTL2,Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +1
-0+
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);Quoted 2 lines
Quoted 2 lines
WREG32_SOC15(GC,
GET_INST(GC, i),
regVM_L2_PROTECTION_FAULT_CNTL2,Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c +0
-0tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.cindex9ea593e2c719..152b2735d360100644drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +0
-0@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +1
-0uint64_t value; +u32 tmp;if (!amdgpu_sriov_vf (adev)) {/* Program the AGP BAR */ @@ -178,8 +179,12 @@ static void
gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c +1
-0+WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.cindex30b90d35abd0..83c2ddbbd292100644drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +0
-0@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +1
-0uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -182,8 +183,12 @@ static void
gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15 (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 25 lines
Quoted 25 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c +1
-0+WREG32_SOC15 (GC ,0 , mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.cindex9e6a6e13dec0..90bbb2fe4884100644drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +0
-0@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct
amdgpu_device*adev)>
static voidgfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +1
-0uint64_t value; +u32 tmp;/* Program the AGP BAR */ WREG32_SOC15 (GC ,0 , regGCMC_VM_AGP_BASE,0 );@@ -176,8 +177,12 @@ static void
gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.cindexb3b1085c7cd3..1b3c067ab48c100644drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +0
-0@@ -153,6 +153,7 @@ static void
gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)>
static voidgfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev){drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +1
-0uint64_t value; +u32 tmp;if (amdgpu_sriov_vf (adev))return ;@@ -181,8 +182,12 @@ static void
gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)>
WREG32_SOC15(GC, 0,regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +3
-2(u32)((u64)adev->dummy_page_addr >> 44)); -WREG32_FIELD15_PREREG (GC ,0 ,GCVM_L2_PROTECTION_FAULT_CNTL2 , -ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY ,1 ); + tmp =RREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2); + tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, +
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +2
-0+ tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2, + ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c +1
-0+WREG32_SOC15 (GC ,0 , regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp); }
Re: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
Quoted 15 lines · Central European Summer Time Christian König
Quoted 14 lines · Timur Kristóf
Quoted 10 lines · On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian…
Quoted 6 lines · Timur Kristóf
Enable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.
Quoted 6 lines
Quoted 3 lines
Why would it be enabled all the time?
I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling theENABLE_RETRY_FAULT_INTERRUPTbit.As far as I understand it the
GCVM_L2_PROTECTION_FAULT_CNTL2is actually a debug register for the fault behavior of the GC block.
After our last conversation I tested this again and I can confirm that I do NOT get any retry fault interrupts on Navi 3 if I remove the code that enables this bit. That makes me think this is not enabled by default.
Quoted 2 lines
According to the register spec for Navi 44 the
ENABLE_RETRY_FAULT_INTERRUPTshould be default 1 and always be set under normal cricumstances.
This is definitely not the case on Navi 31 and also on Strix Halo. I haven't checked on Navi 4x but judging by the fact that the gfxhub_v12_1 code has to enable this bit explicitly, I doubt it would be enabled on Navi 4x either.
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
IIRC only the
RETRY_PERMISSION_OR_INVALID_PAGE_FAULTbit in theVM_CONTEXT0_CNTLregister should be set or cleared by the kernel driver or
firmware to control if the HW retries the access or not.That is clearly not the case on GFX12.1 and I haven't seen any indication that it would be different on GFX11.x and 12.0 either.
Mhm, then either the FW or golden register settings for that register is not correct.
Quoted 7 lines
That we set
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRYto one is rather strange as well since we don't use that feature in the Linux driver.My suggestion is to always set the
ENABLE_RETRY_FAULT_INTERRUPTinGCVM_L2_PROTECTION_FAULT_CNTL2no matter what the noretry flag says, but I'm going to ask Alex and our ROCm people about that again when I have time.
I don't see why we should enable this bit when we don't want to enable retry faults.
Thanks & best regards,
Timur
Christian König
maintainerRe: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
Quoted 28 lines · Timur Kristóf
Quoted 15 lines · Central European Summer Time Christian König
Quoted 14 lines · Timur Kristóf
Quoted 10 lines · On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian…
Quoted 6 lines · Timur Kristóf
Enable retry fault interrupts when initializing the GFXHUB system aperture registers according to whether retrying page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
Needs to be done for each GFXHUB version at once, because none of them actually enabled this interrupt.
Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.
Quoted 6 lines
Quoted 3 lines
Why would it be enabled all the time?
I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling theENABLE_RETRY_FAULT_INTERRUPTbit.As far as I understand it the
GCVM_L2_PROTECTION_FAULT_CNTL2is actually a debug register for the fault behavior of the GC block.After our last conversation I tested this again and I can confirm that I do NOT get any retry fault interrupts on Navi 3 if I remove the code that enables this bit. That makes me think this is not enabled by default.
Quoted 7 lines
Quoted 2 lines
According to the register spec for Navi 44 the
ENABLE_RETRY_FAULT_INTERRUPTshould be default 1 and always be set under normal cricumstances.This is definitely not the case on Navi 31 and also on Strix Halo. I haven't checked on Navi 4x but judging by the fact that the
gfxhub_v12_1code has to enable this bit explicitly, I doubt it would be enabled on Navi 4x either.
My guess is that this is either some bug in the golden register settings or the VBIOS so that isn't correctly initialized by default, or that this bit is controlled by FW to enable/disable the functionality depending on which queue is currently processing.
I need to double check that when I'm back from vacation.
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
Quoted 4 lines
IIRC only the
RETRY_PERMISSION_OR_INVALID_PAGE_FAULTbit in theVM_CONTEXT0_CNTLregister should be set or cleared by the kernel driver or
firmware to control if the HW retries the access or not.That is clearly not the case on GFX12.1 and I haven't seen any indication that it would be different on GFX11.x and 12.0 either.
Mhm, then either the FW or golden register settings for that register is not correct.
That we set
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRYto one is rather strange as well since we don't use that feature in the Linux driver.
Quoted 7 lines
Quoted 4 lines
My suggestion is to always set the
ENABLE_RETRY_FAULT_INTERRUPTinGCVM_L2_PROTECTION_FAULT_CNTL2no matter what the noretry flag says, but I'm going to ask Alex and our ROCm people about that again when I have time.I don't see why we should enable this bit when we don't want to enable retry faults.
Because according to the HW documentation it should always be on and never disabled. I'm pretty sure we shouldn't touch it from the kernel driver.
For now just always set it to one during initialization, if the retry fault interrupt is enabled or not is gated through the bit in the VM_CONTEXT1_CNTL register for the kernel driver.
Regards,
Christian.
Quoted 7 lines
Thanks & best regards,
Timur
PATCH 3/14 drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring
In reply to Timur KristófThe soft IH ring is implemented entirely in software. We shouldn't read (or write) any HW registers when accessing it.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ++++ 4 files changed, 25 insertions(+)
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7 -0
@@ -439,6 +439,10 @@ static u32 ih_v6_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -514,6 +518,9 @@ static void ih_v6_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;
drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +7 -0
@@ -410,6 +410,10 @@ static u32 ih_v6_1_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -481,6 +485,9 @@ static void ih_v6_1_irq_rearm(struct amdgpu_device *adev,static void ih_v6_1_set_rptr (struct amdgpu_device *adev,struct amdgpu_ih_ring *ih) { +if (ih == &adev->irq.ih_soft) +return ; +struct amdgpu_ih_regs *ih_regs;if (ih->use_doorbell) {
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +7 -0
@@ -457,6 +457,10 @@ static u32 ih_v7_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))
@@ -527,6 +531,9 @@ static void ih_v7_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;
drivers/gpu/drm/amd/amdgpu/navi10_ih.c +4 -0
@@ -417,6 +417,10 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev,*/ wptr =le32_to_cpu (*ih->wptr_cpu); +if (ih == &adev->irq.ih_soft) + goto out; + +if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW )) goto out; }
Signature · 1 line
2.54.0
Re: [PATCH 03/14] drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring
Quoted 5 lines · Timur Kristóf
The soft IH ring is implemented entirely in software. We shouldn't read (or write) any HW registers when accessing it.
Signed-off-by Timur Kristóf <[email protected]>Reviewed-by Tvrtko Ursulin <[email protected]>
Quoted 97 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 7 +++++++ drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ++++ 4 files changed, 25 insertions(+)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7
-0@@ -439,6 +439,10 @@ static u32 ih_v6_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))@@ -514,6 +518,9 @@ static void ih_v6_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +7
-0@@ -410,6 +410,10 @@ static u32 ih_v6_1_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))@@ -481,6 +485,9 @@ static void ih_v6_1_irq_rearm(struct amdgpu_device *adev,static void ih_v6_1_set_rptr (struct amdgpu_device *adev,struct amdgpu_ih_ring *ih) { +if (ih == &adev->irq.ih_soft) +return ; +struct amdgpu_ih_regs *ih_regs;if (ih->use_doorbell) {drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +7
-0@@ -457,6 +457,10 @@ static u32 ih_v7_0_get_wptr(struct amdgpu_device *adev,struct amdgpu_ih_regs *ih_regs; wptr =le32_to_cpu (*ih->wptr_cpu); + +if (ih == &adev->irq.ih_soft) + goto out; + ih_regs = &ih->ih_regs;if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW ))@@ -527,6 +531,9 @@ static void ih_v7_0_set_rptr(struct amdgpu_device *adev, {struct amdgpu_ih_regs *ih_regs; +if (ih == &adev->irq.ih_soft) +return ; +if (ih->use_doorbell) {/* XXX check if swapping is necessary on BE */ *ih->rptr_cpu = ih->rptr;drivers/gpu/drm/amd/amdgpu/navi10_ih.c +4
-0@@ -417,6 +417,10 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev,*/ wptr =le32_to_cpu (*ih->wptr_cpu); +if (ih == &adev->irq.ih_soft) + goto out; + +if (!REG_GET_FIELD (wptr,IH_RB_WPTR ,RB_OVERFLOW )) goto out; }
PATCH 4/14 drm/amdgpu/ih: Add retry_cam_ack IH function pointer
In reply to Timur KristófNo functional changes.
This is a refactor to allow different filter CAM implementation in subsequent commits. The actual new implementations are going to be in subsequent commits.
Instead of writing the doorbell in amdgpu_gmc_handle_retry_fault() directly, add an IH function pointer which can be defined in a different way for different IH versions.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 7 ++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h | 1 + drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 6 ++++++ drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 8 +++++++- 4 files changed, 20 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +6 -1
@@ -552,6 +552,11 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,int ret;if (adev->irq.retry_cam_enabled) { +if (!adev->irq.ih_funcs->retry_cam_ack) { +dev_warn (adev->dev,"retry CAM is enabled, but retry_cam_ack is NULL\n" ); +return -EOPNOTSUPP ; + } +/* Delegate it to a different ring if the hardware hasn't * already done it. */
@@ -562,7 +567,7 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev, ret =amdgpu_vm_handle_fault (adev, entry->pasid, entry->vmid, node_id, addr, entry->timestamp, write_fault); -WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); + adev->irq.ih_funcs->retry_cam_ack (adev, cam_index);if (ret)return 1 ; }else {
drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h +1 -0
@@ -97,6 +97,7 @@ struct amdgpu_ih_funcs {const char *(*node_id_to_die_name)(struct amdgpu_device *adev,unsigned int node_id,char *buf,size_t size); +void (*retry_cam_ack)(struct amdgpu_device *adev,u32 cam_index); };#define amdgpu_ih_get_wptr (adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih))
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +6 -0
@@ -289,6 +289,11 @@ static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)return val; } +static void ih_v7_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +} +#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122#define regIH_RING1_CLIENT_CFG_INDEX_V7_1_BASE_IDX 0#define regIH_RING1_CLIENT_CFG_DATA_V7_1 0x123
@@ -865,6 +870,7 @@ static const struct amdgpu_ih_funcs ih_v7_0_funcs = { .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, .set_rptr = ih_v7_0_set_rptr, .node_id_to_die_name = ih_v7_0_node_id_to_die_name, + .retry_cam_ack = ih_v7_0_retry_cam_ack, };static void ih_v7_0_set_interrupt_funcs (struct amdgpu_device *adev)
drivers/gpu/drm/amd/amdgpu/vega20_ih.c +7 -1
@@ -293,6 +293,11 @@ static uint32_t vega20_setup_retry_doorbell(u32 doorbell_index)return val; } +static void vega20_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +} +/** * vega20_ih_irq_init - init and enable the interrupt ring *
@@ -738,7 +743,8 @@ static const struct amdgpu_ih_funcs vega20_ih_funcs = { .get_wptr = vega20_ih_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = vega20_ih_set_rptr + .set_rptr = vega20_ih_set_rptr, + .retry_cam_ack = vega20_retry_cam_ack, };static void vega20_ih_set_interrupt_funcs (struct amdgpu_device *adev)
Signature · 1 line
2.54.0
Re: [PATCH 04/14] drm/amdgpu/ih: Add retry_cam_ack IH function pointer
Quoted 11 lines · Timur Kristóf
No functional changes.
This is a refactor to allow different filter CAM implementation in subsequent commits. The actual new implementations are going to be in subsequent commits.
Instead of writing the doorbell in
amdgpu_gmc_handle_retry_fault()directly, add an IH function pointer which can be defined in a different way for different IH versions.Signed-off-by Timur Kristóf <[email protected]>
Quoted 94 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 7 ++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h | 1 + drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 6 ++++++ drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 8 +++++++- 4 files changed, 20 insertions(+), 2 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +6
-1@@ -552,6 +552,11 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,int ret;if (adev->irq.retry_cam_enabled) { +if (!adev->irq.ih_funcs->retry_cam_ack) { +dev_warn (adev->dev,"retry CAM is enabled, but retry_cam_ack is NULL\n" ); +return -EOPNOTSUPP ; + } +/* Delegate it to a different ring if the hardware hasn't * already done it. */ @@ -562,7 +567,7 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev, ret =amdgpu_vm_handle_fault (adev, entry->pasid, entry->vmid, node_id, addr, entry->timestamp, write_fault); -WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); + adev->irq.ih_funcs->retry_cam_ack (adev, cam_index);if (ret)return 1 ; }else {drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h +1
-0@@ -97,6 +97,7 @@ struct amdgpu_ih_funcs {const char *(*node_id_to_die_name)(struct amdgpu_device *adev,unsigned int node_id,char *buf,size_t size); +void (*retry_cam_ack)(struct amdgpu_device *adev,u32 cam_index); };#define amdgpu_ih_get_wptr (adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih))drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +6
-0@@ -289,6 +289,11 @@ static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)return val; } +static void ih_v7_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +} +#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122#define regIH_RING1_CLIENT_CFG_INDEX_V7_1_BASE_IDX 0#define regIH_RING1_CLIENT_CFG_DATA_V7_1 0x123@@ -865,6 +870,7 @@ static const struct amdgpu_ih_funcs ih_v7_0_funcs = { .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, .set_rptr = ih_v7_0_set_rptr, .node_id_to_die_name = ih_v7_0_node_id_to_die_name, + .retry_cam_ack = ih_v7_0_retry_cam_ack, };static void ih_v7_0_set_interrupt_funcs (struct amdgpu_device *adev)drivers/gpu/drm/amd/amdgpu/vega20_ih.c +7
-1@@ -293,6 +293,11 @@ static uint32_t vega20_setup_retry_doorbell(u32 doorbell_index)return val; } +static void vega20_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +} +/** * vega20_ih_irq_init - init and enable the interrupt ring * @@ -738,7 +743,8 @@ static const struct amdgpu_ih_funcs vega20_ih_funcs = { .get_wptr = vega20_ih_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = vega20_ih_set_rptr + .set_rptr = vega20_ih_set_rptr, + .retry_cam_ack = vega20_retry_cam_ack, };static void vega20_ih_set_interrupt_funcs (struct amdgpu_device *adev)
PATCH 5/14 drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
In reply to Timur KristófWhen there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow. PAGE_SIZE was too small, use IH_SW_RING_SIZE to match what other GPU generations are doing.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +1 -1
@@ -590,7 +590,7 @@ static int ih_v6_1_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v6_1_init_register_offset (adev); - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,PAGE_SIZE , true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
Signature · 1 line
2.54.0
Re: [PATCH 05/14] drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
Quoted 7 lines · Timur Kristóf
When there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow.
PAGE_SIZEwas too small, useIH_SW_RING_SIZEto match what other GPU generations are doing.Signed-off-by Timur Kristóf <[email protected]>Reviewed-by Tvrtko Ursulin <[email protected]>
Quoted 17 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_1.c +1
-1@@ -590,7 +590,7 @@ static int ih_v6_1_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v6_1_init_register_offset (adev); - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,PAGE_SIZE , true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
PATCH 6/14 drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
In reply to Timur KristófWhen there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow. PAGE_SIZE was too small, use IH_SW_RING_SIZE to match what other GPU generations are doing.
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1 -4
@@ -601,7 +601,6 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)int r;struct amdgpu_device *adev = ip_block->adev;bool use_bus_addr; -unsigned int sw_ring_size; r =amdgpu_irq_add_id (adev,SOC21_IH_CLIENTID_IH ,0 , &adev->irq.self_irq);
@@ -633,9 +632,7 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v7_0_init_register_offset (adev); - sw_ring_size = (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) ? -IH_SW_RING_SIZE :PAGE_SIZE ; - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft, sw_ring_size, true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
Signature · 1 line
2.54.0
Re: [PATCH 06/14] drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE
Quoted 7 lines · Timur Kristóf
When there are a lot of retry faults happening, the soft IH ring can fill up really quickly and possibly overflow.
PAGE_SIZEwas too small, useIH_SW_RING_SIZEto match what other GPU generations are doing.Signed-off-by Timur Kristóf <[email protected]>Reviewed-by Tvrtko Ursulin <[email protected]>
Quoted 27 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-4@@ -601,7 +601,6 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)int r;struct amdgpu_device *adev = ip_block->adev;bool use_bus_addr; -unsigned int sw_ring_size; r =amdgpu_irq_add_id (adev,SOC21_IH_CLIENTID_IH ,0 , &adev->irq.self_irq);@@ -633,9 +632,7 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)/* initialize ih control register offset */ ih_v7_0_init_register_offset (adev); - sw_ring_size = (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) ? -IH_SW_RING_SIZE :PAGE_SIZE ; - r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft, sw_ring_size, true); + r =amdgpu_ih_ring_init (adev, &adev->irq.ih_soft,IH_SW_RING_SIZE , true);if (r)return r;
PATCH 7/14 drm/amdgpu/gmc11: Pass cam_index to retry fault handler
In reply to Timur KristófThis is necessary if we want to make use of the filter CAM.
--- drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +4 -1
@@ -108,13 +108,16 @@ static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr; addr = (u64 )entry->src_data[0 ] <<12 ; addr |= ((u64 )entry->src_data[1 ] &0xf ) <<44 ;if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
Signature · 1 line
2.54.0
Christian König
maintainerRe: [PATCH 07/14] drm/amdgpu/gmc11: Pass cam_index to retry fault handler
Quoted 12 lines · Timur Kristóf
This is necessary if we want to make use of the filter CAM.
Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +0
-0
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev, · line 112
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +1
-0@@ -108,13 +108,16 @@ static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr;
Please declare "int ret;" here instead.
Apart from that looks good to me,
Christian.
Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c +3
-1addr = (u64 )entry->src_data[0 ] <<12 ; addr |= ((u64 )entry->src_data[1 ] &0xf ) <<44 ;if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
PATCH 8/14 drm/amdgpu/gmc12: Pass cam_index to retry fault handler
In reply to Timur KristófThis is necessary if we want to make use of the filter CAM.
--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +4 -1
@@ -99,6 +99,7 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr; addr = (u64 )entry->src_data[0 ] <<12 ;
@@ -110,7 +111,9 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev, hub = &adev->vmhub[AMDGPU_GFXHUB (0 )];if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
Signature · 1 line
2.54.0
Christian König
maintainerRe: [PATCH 08/14] drm/amdgpu/gmc12: Pass cam_index to retry fault handler
Quoted 13 lines · Timur Kristóf
This is necessary if we want to make use of the filter CAM.
Signed-off-by Timur Kristóf <[email protected]>Reviewed-by Tvrtko Ursulin <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +0
-0
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev, · line 103
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +1
-0@@ -99,6 +99,7 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,bool write_fault = !!(entry->src_data[1 ] &AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE );uint32_t status =0 ; +uint32_t cam_index;u64 addr;
Same as patch #7, declare "int ret;" here instead.
Christian.
Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +3
-1addr = (u64 )entry->src_data[0 ] <<12 ;@@ -110,7 +111,9 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev, hub = &adev->vmhub[AMDGPU_GFXHUB (0 )];if (retry_fault) { -int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr,0 ,0 , + cam_index = entry->src_data[2 ] &0x3ff ; + +int ret =amdgpu_gmc_handle_retry_fault (adev, entry, addr, cam_index,0 , write_fault);/* Returning 1 here also prevents sending the IV to the KFD */ if (ret ==1 )
PATCH 9/14 drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on GFX12.0
In reply to Timur KristófThis flag seems to work around a "fault priority problem" and is necessary for handling faults on GFX12. The kernel seems unable to mitigate retry faults on GFX12 without this flag.
For reference see:amdgpu_vm_pte_update_flags() that explains the problem svm_range_get_pte_flags() that uses the flag on GFX12
--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +2 -1
@@ -644,11 +644,12 @@ static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block) adev->gmc.xgmi.connected_to_cpu = adev->smuio.funcs->is_host_gpu_xgmi_supported (adev); + adev->gmc.init_pte_flags =AMDGPU_PTE_IS_PTE ; +switch (amdgpu_ip_version (adev,GC_HWIP ,0 )) {case IP_VERSION (12 ,1 ,0 ):gmc_v12_1_set_gmc_funcs (adev);gmc_v12_1_set_irq_funcs (adev); - adev->gmc.init_pte_flags =AMDGPU_PTE_IS_PTE ;break ;default :gmc_v12_0_set_gmc_funcs (adev);
Signature · 1 line
2.54.0
Re: [PATCH 09/14] drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on GFX12.0
Quoted 10 lines · Timur Kristóf
This flag seems to work around a "fault priority problem" and is necessary for handling faults on GFX12. The kernel seems unable to mitigate retry faults on GFX12 without this flag.
For reference see:
amdgpu_vm_pte_update_flags()that explains the problemsvm_range_get_pte_flags()that uses the flag on GFX12Signed-off-by Timur Kristóf <[email protected]>
Quoted 22 lines
--- drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c +2
-1@@ -644,11 +644,12 @@ static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block) adev->gmc.xgmi.connected_to_cpu = adev->smuio.funcs->is_host_gpu_xgmi_supported (adev); + adev->gmc.init_pte_flags =AMDGPU_PTE_IS_PTE ; +switch (amdgpu_ip_version (adev,GC_HWIP ,0 )) {case IP_VERSION (12 ,1 ,0 ):gmc_v12_1_set_gmc_funcs (adev);gmc_v12_1_set_irq_funcs (adev); - adev->gmc.init_pte_flags =AMDGPU_PTE_IS_PTE ;break ;default :gmc_v12_0_set_gmc_funcs (adev);
PATCH 10/14 drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()
In reply to Timur KristófPTE_IS_PTE seems necessary for handling retry faults on GFX12.
For reference see:amdgpu_vm_pte_update_flags() that explains the problem svm_range_get_pte_flags() that uses the flag on GFX12
Also add NOALLOC on GFX10.3+ as we don't need to allocate the fault handling PTE in the infinity cache (MALL).
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +6 -2
@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, } addr /=AMDGPU_GPU_PAGE_SIZE ; - flags =AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | +AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED |AMDGPU_PTE_SYSTEM ;if (is_compute_context) {
@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, flags =AMDGPU_VM_NORETRY_FLAGS ; value =0 ; }else if (amdgpu_vm_fault_stop ==AMDGPU_VM_FAULT_STOP_NEVER ) { +/* Don't allocate this PTE in the MALL */ +if (amdgpu_ip_version (adev,GC_HWIP ,0 ) >=IP_VERSION (10 ,3 ,0 )) + flags |=AMDGPU_PTE_NOALLOC ; +/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE |AMDGPU_PTE_WRITEABLE ; - }else {/* Let the hw retry silently on the PTE */ value =0 ;
Signature · 1 line
2.54.0
Christian König
maintainerRe: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()
Quoted 19 lines · Timur Kristóf
PTE_IS_PTEseems necessary for handling retry faults on GFX12.For reference see:
amdgpu_vm_pte_update_flags()that explains the problemsvm_range_get_pte_flags()that uses the flag on GFX12Also add NOALLOC on GFX10.3+ as we don't need to allocate the fault handling PTE in the infinity cache (MALL).
Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · line 3049
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-1@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, } addr /=AMDGPU_GPU_PAGE_SIZE ; - flags =AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | +AMDGPU_PTE_VALID |AMDGPU_PTE_SNOOPED |AMDGPU_PTE_SYSTEM ;
That doesn't make sense the PTE flags should be adjusted to the device specific flags by the callbacks.
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · line 3061
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4
-0if (is_compute_context) {@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, flags =AMDGPU_VM_NORETRY_FLAGS ; value =0 ; }else if (amdgpu_vm_fault_stop ==AMDGPU_VM_FAULT_STOP_NEVER ) { +/* Don't allocate this PTE in the MALL */ +if (amdgpu_ip_version (adev,GC_HWIP ,0 ) >=IP_VERSION (10 ,3 ,0 )) + flags |=AMDGPU_PTE_NOALLOC ; +
This doesn't make sense either, mall allocation for the dummy page should be perfectly fine.
Regards,
Christian.
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-1/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE |AMDGPU_PTE_WRITEABLE ; - }else {/* Let the hw retry silently on the PTE */ value =0 ;
Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()
Quoted 27 lines · On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian König…
Quoted 26 lines · Timur Kristóf
PTE_IS_PTEseems necessary for handling retry faults on GFX12.For reference see:
amdgpu_vm_pte_update_flags()that explains the problemsvm_range_get_pte_flags()that uses the flag on GFX12Also add NOALLOC on GFX10.3+ as we don't need to allocate the fault handling PTE in the infinity cache (MALL).
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.cindex32719f31b6c9..a915d061085f100644drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,> } addr /=AMDGPU_GPU_PAGE_SIZE ;
Quoted 8 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-1- flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | + AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
AMDGPU_PTE_SYSTEM;That doesn't make sense the PTE flags should be adjusted to the device specific flags by the callbacks.
Can you please elaborate on what is it that doesn't make sense here and why, and what you recommend to do instead?
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device · line 3062
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0if (is_compute_context) {@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,> flags =AMDGPU_VM_NORETRY_FLAGS ; value =0 ; }else if (amdgpu_vm_fault_stop ==AMDGPU_VM_FAULT_STOP_NEVER ) { +/* Don't allocate this PTE in the MALL */ + if (amdgpu_ip_version(adev, GC_HWIP, 0) >=
IP_VERSION(10, 3, 0))
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · line 3061
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0+ flags |=AMDGPU_PTE_NOALLOC ; +This doesn't make sense either, mall allocation for the dummy page should be perfectly fine.
I think we shouldn't waste space for this PTE in the MALL.
Quoted 13 lines
Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-1/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE |AMDGPU_PTE_WRITEABLE ; - }else {/* Let the hw retry silently on the PTE */ value =0 ;
Christian König
maintainerRe: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()
Quoted 33 lines · Timur Kristóf
Quoted 31 lines · On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian König…
Quoted 30 lines · Timur Kristóf
PTE_IS_PTEseems necessary for handling retry faults on GFX12.For reference see:
amdgpu_vm_pte_update_flags()that explains the problemsvm_range_get_pte_flags()that uses the flag on GFX12Also add NOALLOC on GFX10.3+ as we don't need to allocate the fault handling PTE in the infinity cache (MALL).
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.cindex32719f31b6c9..a915d061085f100644drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-1@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,> } addr /=AMDGPU_GPU_PAGE_SIZE ; - flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | + AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
AMDGPU_PTE_SYSTEM;That doesn't make sense the PTE flags should be adjusted to the device specific flags by the callbacks.
Can you please elaborate on what is it that doesn't make sense here and why, and what you recommend to do instead?
I had to dig up what the problem here is as well. We use the ASIC specific AMDGPU_PTE_* flags directly.
Instead we should use the AMDGPU_VM_PAGE_* flags and then call amdgpu_gmc_get_vm_pte(adev, vm, NULL, in_flags, &out_flags) to translate the AMDGPU_VM_PAGE_* flags into the ASIC specific ones.
This will automatically add flags like AMDGPU_PTE_TF and AMDGPU_PTE_IS_PTE etc...
Quoted 12 lines
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device · line 3062
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0if (is_compute_context) {@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,> flags =AMDGPU_VM_NORETRY_FLAGS ; value =0 ; }else if (amdgpu_vm_fault_stop ==AMDGPU_VM_FAULT_STOP_NEVER ) { +/* Don't allocate this PTE in the MALL */ + if (amdgpu_ip_version(adev, GC_HWIP, 0) >=
IP_VERSION(10, 3, 0))
Quoted 7 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · line 3061
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0+ flags |=AMDGPU_PTE_NOALLOC ; +This doesn't make sense either, mall allocation for the dummy page should be perfectly fine.
I think we shouldn't waste space for this PTE in the MALL.
Hui? The PTE doesn't enter the MALL, it is the dummy page which enters the MALL and since it is only 4k it basically doesn't waste much space.
Regards,
Christian.
Quoted 18 lines
Quoted 14 lines
Quoted 14 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-1/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE |AMDGPU_PTE_WRITEABLE ; - }else {/* Let the hw retry silently on the PTE */ value =0 ;
Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()
Quoted 48 lines · On 2026. július 2., csütörtök 14:18:45 közép-európai nyári idő Christian König…
Quoted 36 lines · Timur Kristóf
Quoted 33 lines · On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian…
Quoted 32 lines · Timur Kristóf
PTE_IS_PTEseems necessary for handling retry faults on GFX12.For reference see:
amdgpu_vm_pte_update_flags()that explains the problemsvm_range_get_pte_flags()that uses the flag on GFX12Also add NOALLOC on GFX10.3+ as we don't need to allocate the fault handling PTE in the infinity cache (MALL).
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.cindex32719f31b6c9..a915d061085f
100644drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-1@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,> } addr /=AMDGPU_GPU_PAGE_SIZE ; - flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED | + flags = adev->gmc.init_pte_flags | + AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |Quoted 7 lines
Quoted 4 lines
Quoted 1 line
AMDGPU_PTE_SYSTEM;That doesn't make sense the PTE flags should be adjusted to the device specific flags by the callbacks.
Can you please elaborate on what is it that doesn't make sense here and why, and what you recommend to do instead?
I had to dig up what the problem here is as well. We use the ASIC specific AMDGPU_PTE_* flags directly.
Quoted 6 lines
Instead we should use the AMDGPU_VM_PAGE_* flags and then call
amdgpu_gmc_get_vm_pte(adev, vm, NULL, in_flags, &out_flags)to translate the AMDGPU_VM_PAGE_* flags into the ASIC specific ones.This will automatically add flags like
AMDGPU_PTE_TFandAMDGPU_PTE_IS_PTEetc...
As far as I understand, the init_pte_flags was added specifically for this purpose, and that's what SVM uses too.
Are you suggesting to call to amdgpu_gmc_get_vm_pte() here?
As far as I see the implementation of that function for Navi 4 is gmc_v12_0_get_vm_pte() which doesn't set the PTE_IS_PTE flag so it wouldn't resolve the issue with the fault that this commit is fixing.
Quoted 21 lines
Quoted 21 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-0if (is_compute_context) {@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,>Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0flags =AMDGPU_VM_NORETRY_FLAGS ; value =0 ; }else if (amdgpu_vm_fault_stop ==AMDGPU_VM_FAULT_STOP_NEVER ) { +/* Don't allocate this PTE in the MALL */ + if (amdgpu_ip_version(adev, GC_HWIP, 0) >=
IP_VERSION(10, 3, 0))Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid, · line 3061
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2
-0+ flags |=AMDGPU_PTE_NOALLOC ; +This doesn't make sense either, mall allocation for the dummy page should be perfectly fine.
Quoted 5 lines
Quoted 1 line
I think we shouldn't waste space for this PTE in the MALL.
Hui? The PTE doesn't enter the MALL, it is the dummy page which enters the MALL and since it is only 4k it basically doesn't waste much space.
Sure I can remove this line if you feel that strongly about it. Do we gain any benefit from allowing this page in the MALL?
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0
-1/* Redirect the access to the dummy page */ value = adev->dummy_page_addr; flags |=AMDGPU_PTE_EXECUTABLE |AMDGPU_PTE_READABLE |AMDGPU_PTE_WRITEABLE ; - }else {/* Let the hw retry silently on the PTE */ value =0 ;
PATCH 11/14 drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
In reply to Timur KristófThe doorbell is not working on Navi 31.
Use the IH_RETRY_CAM_ACK register.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7 -1
@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,return 0 ; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); +} +/** * ih_v6_0_irq_init - init and enable the interrupt ring *
@@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack, };static void ih_v6_0_set_interrupt_funcs (struct amdgpu_device *adev)
Signature · 1 line
2.54.0
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 4 lines · Timur Kristóf
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Signed-off-by Timur Kristóf <[email protected]>
I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?
Thanks,
Christian.
Quoted 30 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,return 0 ; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); +} +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack, };static void ih_v6_0_set_interrupt_funcs (struct amdgpu_device *adev)
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 10 lines · On 2026. július 2., csütörtök 12:23:42 közép-európai nyári idő Christian König…
Quoted 4 lines · Timur Kristóf
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Signed-off-by Timur Kristóf <[email protected]>Acked-by Christian König <[email protected]>I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?
The first mention of the retry CAM in the kernel is:
commit 318e431b306e - drm/amdgpu: Enable IH retry CAM on GFX9 which was written by Mukul (also on CC for this series). Mukul - can you please give us some information on that?
It seems that the IH 7.1 code was just copied from there.
As far as I see, Vega 20 already has the mmIH_RETRY_CAM_ACK register, though I don't know why that wasn't used. I guess you guys would have heard from your datacenter customers if the current code didn't work on Vega 20 and CDNA. I can say with certainty that I couldn't get the doorbell working on Navi 31 or Navi 48.
Unfortunately I don't have a Vega 20 card (or any datacenter GPU) so I can't test what actually works there. I have a Vega 10, but it's unclear if that has a retry CAM or not.
Thanks & best regards,
Timur
Quoted 41 lines
Quoted 41 lines
---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.cindex65e5d21753f9..a97c85b0cb99100644drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,>return 0 ; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); +} +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};
static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
In reply to Christian KönigAMD General
Quoted 18 lines
-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Thursday, July 2, 2026 6:24 AM
To: Timur Kristóf <[email protected]>; [email protected]; Deucher, Alexander <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei <[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul <[email protected]>
Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0Quoted 3 lines · Timur Kristóf
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.
Quoted 7 lines
Quoted 1 line
Signed-off-by Timur Kristóf <[email protected]>Acked-by Christian König <[email protected]>I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?
If I remember correctly, we weren't sure if the IH_RETRY_CAM_ACK register would be available under virtualization. And we were sure doorbells would work for both bare-metal and virtualization.
Regards,
Mukul
Quoted 34 lines
Thanks,
Christian.drivers/gpu/drm/amd/amdgpu/ih_v6_0.c static int ih_v6_0_enable_ring(struct amdgpu_device
Quoted 10 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +0
-0@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device*adev,
Quoted 20 lines
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1return 0 ; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack, };static void ih_v6_0_set_interrupt_funcs (struct amdgpu_device *adev)
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 9 lines · Mukul
Quoted 2 lines
I still haven't found any explanation why we use a doorbell for the CAM in the first
place? Lijo, Alex, Felix does anybody know that?
Quoted 1 line
If I remember correctly, we weren't sure if the
IH_RETRY_CAM_ACKregister would be available under
virtualization. And we were sure doorbells would
work for both bare-metal and virtualization.
Hi Mukul,
Can you clarify that please?
How are retry faults supposed to work under virtualization? Is the interrupt processed by the guest or the host? Why is the IH_RETRY_CAM_ACK register not available under virtualization?
Thanks & best regards,
Timur
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 12 lines · Timur Kristóf
Quoted 9 lines · Mukul
Quoted 2 lines
I still haven't found any explanation why we use a doorbell for the CAM in the first
place? Lijo, Alex, Felix does anybody know that?
Quoted 1 line
If I remember correctly, we weren't sure if the
IH_RETRY_CAM_ACKregister would be available under
virtualization. And we were sure doorbells would
work for both bare-metal and virtualization.
Quoted 8 lines
Hi Mukul,
Can you clarify that please?
How are retry faults supposed to work under virtualization? Is the interrupt processed by the guest or the host? Why is the
IH_RETRY_CAM_ACKregister not available under virtualization?
For clarification - CAM entries are separate for PF/VF. ACK register also has PF/VF copies. However, in general host driver imposes MMIO register access restrictions to VF, possibly that's the reason.
Please note that IH_RETRY_INT_CAM_CNTL is PF only. With new versions, please make sure to add a VF check for programming.
Thanks,
Lijo
Quoted 5 lines
Thanks & best regards,
Timur
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 24 lines · Lijo
Quoted 15 lines · Timur Kristóf
On 2026. július 3., péntek 20:11:59 közép-európai nyári idő Joshi, Mukul
Quoted 11 lines · wrote:
Quoted 3 lines
I still haven't found any explanation why we use a doorbell for the CAM in
the firstplace? Lijo, Alex, Felix does anybody know that?
If I remember correctly, we weren't sure if the
IH_RETRY_CAM_ACKregister would be available under
virtualization. And we were sure doorbells would
work for both bare-metal and virtualization.Quoted 7 lines
Hi Mukul,
Can you clarify that please?
How are retry faults supposed to work under virtualization? Is the interrupt processed by the guest or the host? Why is the
IH_RETRY_CAM_ACKregister not available under virtualization?
Quoted 6 lines
For clarification - CAM entries are separate for PF/VF. ACK register also has PF/VF copies. However, in general host driver imposes MMIO register access restrictions to VF, possibly that's the reason.
Please note that
IH_RETRY_INT_CAM_CNTLis PF only. With new versions, please make sure to add a VF check for programming.
Hi Lijo,
Can you please clarify what PF and VF means here?
I'm not familiar with those abbreviations.
Thanks,
Timur
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 30 lines · Timur Kristóf
Quoted 28 lines · Lijo
Quoted 15 lines · Timur Kristóf
On 2026. július 3., péntek 20:11:59 közép-európai nyári idő Joshi, Mukul
Quoted 11 lines · wrote:
Quoted 3 lines
I still haven't found any explanation why we use a doorbell for the CAM in
the firstplace? Lijo, Alex, Felix does anybody know that?
If I remember correctly, we weren't sure if the
IH_RETRY_CAM_ACKregister would be available under
virtualization. And we were sure doorbells would
work for both bare-metal and virtualization.Quoted 7 lines
Hi Mukul,
Can you clarify that please?
How are retry faults supposed to work under virtualization? Is the interrupt processed by the guest or the host? Why is the
IH_RETRY_CAM_ACKregister not available under virtualization?For clarification - CAM entries are separate for PF/VF. ACK register also has PF/VF copies. However, in general host driver imposes MMIO register access restrictions to VF, possibly that's the reason.
Quoted 6 lines
Quoted 2 lines
Please note that
IH_RETRY_INT_CAM_CNTLis PF only. With new versions, please make sure to add a VF check for programming.Hi Lijo,
Can you please clarify what PF and VF means here?
PF = physical function. VF = virtual function. On virtualization environments amdgpu is a guest driver.
Translates to below check for cam_cntl register programming.
if (!amdgpu_sriov_vf(adev))
Thanks,
Lijo
Quoted 6 lines
I'm not familiar with those abbreviations.
Thanks,
Timur
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
In reply to Christian KönigQuoted 8 lines · Christian König
Quoted 4 lines · Timur Kristóf
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Signed-off-by Timur Kristóf <[email protected]>Acked-by Christian König <[email protected]>I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?
Why are we using doorbells to update WPTRs instead of writing to the registers directly? I was assuming that using doorbells for the IH CAM had similar reasons. I think register writes can be problematic with certain power saving modes and virtualization. I'm not sure if there is a difference in terms of CPU overhead between writing a doorbell or writing an MMIO register.
If doorbells aren't working, it's probably some configuration issue. I'd rather fix those issues than throw away use of doorbells altogether.
Regards,
Felix
Quoted 34 lines
Thanks,
Christian.Quoted 30 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +7
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,return 0 ; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) +{ +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); +} +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack, };static void ih_v6_0_set_interrupt_funcs (struct amdgpu_device *adev)
RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
In reply to Timur KristófAMD General
Quoted 15 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei <[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 2 lines
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working. I think we need an update to the NBIO ih_doorbell_range() function to enable the doorbell path.
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
Regards,
Mukul
Quoted 35 lines
Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-2@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack, };static void ih_v6_0_set_interrupt_funcs (struct amdgpu_device *adev) --2.54.0
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 13 lines · Mukul
AMD General
Quoted 10 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo
Quoted 11 lines
<[email protected]>; He, Siwei <[email protected]>; Yang, Philip
Quoted 9 lines
<[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.
Hi Mukul,
Quoted 4 lines
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIOih_doorbell_range()function to enable the doorbell path.
Can you please be more specific and elaborate on what we need to do on the NBIO ih_doorbell_range() function? Also, can you say which NBIO version is used for for Navi 3 and Navi 4 GPUs?
Quoted 2 lines
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
I'm OK with that, if the doorbell really works on these GPUs.
Thanks & best regards,
Timur
Quoted 56 lines
Quoted 54 lines
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};
static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)--
2.54.0
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 30 lines · Timur Kristóf
Quoted 13 lines · Mukul
AMD General
Quoted 10 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, LijoQuoted 11 lines
<[email protected]>; He, Siwei <[email protected]>; Yang, Philip
Quoted 9 lines
<[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Hi Mukul,
Quoted 8 lines
Quoted 4 lines
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIOih_doorbell_range()function to enable the doorbell path.Can you please be more specific and elaborate on what we need to do on the NBIO
ih_doorbell_range()function? Also, can you say which NBIO version is used for for Navi 3 and Navi 4 GPUs?
navi3 dGPUs use nbio_v4_3.c. navi3 APUs use nbio_v7_11.c (strix) and nbio_v7_7.c (phoenix). navi4 uses nbif_v6_3_1.c.
Alex
Quoted 69 lines
Quoted 3 lines
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
I'm OK with that, if the doorbell really works on these GPUs.
Thanks & best regards,
TimurQuoted 56 lines
Quoted 54 lines
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};
static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)--
2.54.0
RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
In reply to Timur KristófAMD General
Quoted 38 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Friday, July 3, 2026 2:47 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei <[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul <[email protected]>
Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0Quoted 22 lines · Mukul
AMD General
Quoted 10 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo<[email protected]>; He, Siwei <[email protected]>; Yang, Philip
Quoted 8 lines
<[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
6.0
Quoted 11 lines
Quoted 3 lines
Quoted 2 lines
The doorbell is not working on Navi 31. Use the
IH_RETRY_CAM_ACKregister.Hi Mukul,
Quoted 4 lines
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIOih_doorbell_range()function to enable the doorbell path.
Hi Timur,
Quoted 3 lines
Can you please be more specific and elaborate on what we need to do on the NBIO
ih_doorbell_range()function?
Through NBIO, we route the doorbell to the correct HW block. That is currently not enabled/done for the IH CAM doorbell. So, you basically need to enable it inside ih_doorbell_range() function.
Quoted 3 lines
Also, can you say which NBIO version is used
for for Navi 3 and Navi 4 GPUs?
Navi3 --> NBIO v4.3 (nbio_v4_3.c)
Navi4 --> NBIO v6.3.1 (nbif_v6_3_1.c)
I am working on the change that you need to enable the doorbell path for Navi4x first. Once that is working, I can send you a test patch for enabling it on Navi3x.
Regards,
Mukul
Quoted 68 lines
Quoted 2 lines
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
I'm OK with that, if the doorbell really works on these GPUs.
Thanks & best regards,
TimurQuoted 57 lines
Quoted 55 lines
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(structamdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};
static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)--
2.54.0
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Hi Timur,
For Navi3x, please make the similar change as I suggested for Navi4x to enable the doorbell path along with your existing changes and use doorbell to clear the CAM entries.
drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c +0 -0
@@ -199,7 +199,7 @@ static void nbio_v4_3_ih_doorbell_range(struct amdgpu_device *adev,
ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range, S2A_DOORBELL_ENTRY_1_CTRL,S2A_DOORBELL_PORT1_RANGE_SIZE,
- 2);
+ 8); ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range,
Please let us know how it goes.
Thanks and regards,
Mukul
Quoted 133 lines · Mukul
AMD General
Quoted 38 lines
-----Original Message-----
From: Timur Kristóf<[email protected]>
Sent: Friday, July 3, 2026 2:47 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák<[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix<[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei<[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul<[email protected]>
Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0Quoted 22 lines · Mukul
AMD General
Quoted 10 lines
-----Original Message-----
From: Timur Kristóf<[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin<[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo<[email protected]>; He, Siwei<[email protected]>; Yang, Philip
Quoted 8 lines
<[email protected]>; Joshi, Mukul<[email protected]>
Cc: Timur Kristóf<[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
6.0Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Hi Mukul,
Quoted 4 lines
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIOih_doorbell_range()function to enable the doorbell path.Hi Timur,
Quoted 3 lines
Can you please be more specific and elaborate on what we need to do on the NBIO
ih_doorbell_range()function?Through NBIO, we route the doorbell to the correct HW block. That is currently not enabled/done for the IH CAM doorbell. So, you basically need to enable it inside
ih_doorbell_range()function.Quoted 3 lines
Also, can you say which NBIO version is used
for for Navi 3 and Navi 4 GPUs?Navi3 --> NBIO v4.3 (
nbio_v4_3.c)
Navi4 --> NBIO v6.3.1 (nbif_v6_3_1.c)I am working on the change that you need to enable the doorbell path for Navi4x first. Once that is working, I can send you a test patch for enabling it on Navi3x.
Regards,
MukulQuoted 63 lines
Quoted 2 lines
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
I'm OK with that, if the doorbell really works on these GPUs.
Thanks & best regards,
TimurQuoted 54 lines
Quoted 53 lines
Signed-off-by Timur Kristóf<[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(structamdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)--
2.54.0
Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
Quoted 22 lines · Central European Summer Time Mukul Joshi
Hi Timur,
For Navi3x, please make the similar change as I suggested for Navi4x to enable the doorbell path along with your
existing changes and use doorbell to clear the CAM entries.drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c +1
-1@@ -199,7 +199,7 @@ static void nbio_v4_3_ih_doorbell_range(structamdgpu_device *adev, ih_doorbell_range =REG_SET_FIELD (ih_doorbell_range,S2A_DOORBELL_ENTRY_1_CTRL ,S2A_DOORBELL_PORT1_RANGE_SIZE , -2 ); +8 ); ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range,Please let us know how it goes.
Quoted 3 lines
Thanks and regards,
Mukul
Hi Mukul,
Sorry for the late reply.
I've now tested this on Navi 31 and with this change the doorbell works correctly for the retry CAM. I will test the same on Navi 48 next week. If it works well then I will drop the patches that use the MMIO register and I'm fine using the doorbell.
Thanks & best regards,
Timur
Quoted 133 lines · Mukul
Quoted 131 lines
AMD General
Quoted 40 lines
-----Original Message-----
From: Timur Kristóf<[email protected]>
Sent: Friday, July 3, 2026 2:47 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák<[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix<[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei<[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul<[email protected]>
Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0On 2026. július 3., péntek 19:46:03 közép-európai nyári idő Joshi, Mukul
Quoted 23 lines · wrote:
AMD General
Quoted 10 lines
-----Original Message-----
From: Timur Kristóf<[email protected]>
Sent: Wednesday, July 1, 2026 12:17 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin<[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo<[email protected]>; He, Siwei<[email protected]>; Yang, Philip
Quoted 8 lines
<[email protected]>; Joshi, Mukul<[email protected]>
Cc: Timur Kristóf<[email protected]>
Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
6.0Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
The doorbell is not working on Navi 31.
Use theIH_RETRY_CAM_ACKregister.Hi Mukul,
Quoted 4 lines
I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIOih_doorbell_range()function to enable the doorbell path.Hi Timur,
Quoted 4 lines
Can you please be more specific and elaborate on what we need to do on the
NBIOih_doorbell_range()function?Through NBIO, we route the doorbell to the correct HW block. That is currently not enabled/done for the IH CAM doorbell. So, you basically need to enable it inside
ih_doorbell_range()function.Quoted 2 lines
Also, can you say which NBIO version is used
for for Navi 3 and Navi 4 GPUs?Navi3 --> NBIO v4.3 (
nbio_v4_3.c)
Navi4 --> NBIO v6.3.1 (nbif_v6_3_1.c)I am working on the change that you need to enable the doorbell path for Navi4x first. Once that is working, I can send you a test patch for enabling it on Navi3x.
Regards,
MukulQuoted 55 lines
Quoted 2 lines
Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.
I'm OK with that, if the doorbell really works on these GPUs.
Thanks & best regards,
TimurQuoted 47 lines
Quoted 47 lines
Signed-off-by Timur Kristóf<[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +6
-1@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(structamdgpu_device *adev, return 0; } +static void ih_v6_0_retry_cam_ack (struct amdgpu_device *adev,u32 +cam_index) { +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); } +/** * ih_v6_0_irq_init - init and enable the interrupt ring * @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = { .get_wptr = ih_v6_0_get_wptr, .decode_iv = amdgpu_ih_decode_iv_helper, .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper, - .set_rptr = ih_v6_0_set_rptr + .set_rptr = ih_v6_0_set_rptr, + .retry_cam_ack = ih_v6_0_retry_cam_ack,};static void
ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)--
2.54.0
PATCH 12/14 drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
In reply to Timur KristófThe doorbell is not working on Navi 48.
Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1 -16
@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev,return 0 ; } -static uint32_t ih_v7_0_setup_retry_doorbell (u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} -static void ih_v7_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) { -WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122
@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev);if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) { -/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih +2 ) <<1 ; -WREG32_SOC15 (OSSSYS ,0 , regIH_DOORBELL_RETRY_CAM, -ih_v7_0_setup_retry_doorbell (adev->irq.retry_cam_doorbell_index)); -/* Enable IH Retry CAM */ tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 );
Signature · 1 line
2.54.0
Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
Quoted 4 lines · Timur Kristóf
The doorbell is not working on Navi 48.
Use theIH_RETRY_CAM_ACKregister just like on Navi 3x.Signed-off-by Timur Kristóf <[email protected]>
Quoted 41 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-16@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev,return 0 ; } -static uint32_t ih_v7_0_setup_retry_doorbell (u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} -static void ih_v7_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) { -WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev);if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) { -/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih +2 ) <<1 ; -WREG32_SOC15 (OSSSYS ,0 , regIH_DOORBELL_RETRY_CAM, -ih_v7_0_setup_retry_doorbell (adev->irq.retry_cam_doorbell_index)); -/* Enable IH Retry CAM */ tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 );
RE: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
In reply to Timur KristófAMD General
Quoted 52 lines
-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Timur Kristóf
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei <[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0The doorbell is not working on Navi 48.
Use theIH_RETRY_CAM_ACKregister just like on Navi 3x.Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-10@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev, return 0; } -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} -static void ih_v7_0_retry_cam_ack (struct amdgpu_device *adev,u32 cam_index) { -WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 395
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev);if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) {
We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same. The code being removed below is for IH 7.1 and not 7.0.
Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH 7.1. I would recommend to drop this patch.
Regards,
Mukul
Quoted 14 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-4-/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev- >doorbell_index.ih + 2) << 1; - WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM, - ih_v7_0_setup_retry_doorbell(adev-Quoted 1 line
irq.retry_cam_doorbell_index));
-
/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);
tmp = REG_SET_FIELD(tmp,IH_RETRY_INT_CAM_CNTL,
ENABLE, 1);
--
2.54.0
Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
Quoted 14 lines · Mukul
AMD General
Quoted 11 lines
-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Timur Kristóf
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo
<[email protected]>; He, Siwei <[email protected]>; Yang, Philip
Quoted 66 lines
Quoted 66 lines
<[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0The doorbell is not working on Navi 48.
Use theIH_RETRY_CAM_ACKregister just like on Navi 3x.Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-9@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev, return 0; } -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} - static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
cam_index) {drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-1-WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev);if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 ))
Quoted 9 lines
Quoted 1 line
{
We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same. The code being removed below is for IH 7.1 and not 7.0.
Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH 7.1.
I would recommend to drop this patch.
Hello Mukul,
Unfortunately I couldn't figure out how to get the doorbell working on Navi 48 and in the previous thread Amir said that it also didn't work for him, and that's why I decided to try the MMIO register.
If you have an idea how to get the doorbell to work on Navi 48, I'm happy to try your suggestion.
Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to keep the code maintainable. If it's not possible then we can also have a slightly different code path for the two different IH versions here.
Thanks & best regards,
Timur
Quoted 26 lines
Quoted 24 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2-/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev-Quoted 2 lines
doorbell_index.ih + 2) << 1;
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2- WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM, - ih_v7_0_setup_retry_doorbell(adev-Quoted 2 lines
irq.retry_cam_doorbell_index));
-
/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL,ENABLE, 1);
--
2.54.0
RE: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
AMD General
Quoted 115 lines
-----Original Message-----
From: Timur Kristóf <[email protected]>
Sent: Friday, July 3, 2026 2:41 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei <[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul <[email protected]>
Subject: Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0Quoted 14 lines · Mukul
AMD General
Quoted 11 lines
-----Original Message-----
From: amd-gfx <[email protected]> On Behalf Of Timur Kristóf
Sent: Wednesday, July 1, 2026 12:17 PM
To: [email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock <[email protected]>; Shetaia, Amir <[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario <[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo<[email protected]>; He, Siwei <[email protected]>; Yang, Philip
Quoted 68 lines
Quoted 68 lines
<[email protected]>; Joshi, Mukul <[email protected]>
Cc: Timur Kristóf <[email protected]>
Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0The doorbell is not working on Navi 48.
Use theIH_RETRY_CAM_ACKregister just like on Navi 3x.Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-9@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(structamdgpu_device *adev, return 0; } -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} - static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
cam_index) {drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-1-WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct
amdgpu_device
*adev)pci_set_master(adev->pdev); if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7,1, 0))
Quoted 9 lines
Quoted 1 line
{
We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same. The code being removed below is for IH 7.1 and not 7.0.
Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH 7.1.
I would recommend to drop this patch.Hello Mukul,
Unfortunately I couldn't figure out how to get the doorbell working on Navi 48 and in the previous thread Amir said that it also didn't work for him, and that's why I decided to try the MMIO register.
Quoted 7 lines
If you have an idea how to get the doorbell to work on Navi 48, I'm happy to try your suggestion.
Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to keep the code maintainable. If it's not possible then we can also have a slightly different code path for the two different IH versions here.
Hi Timur,
As I had mentioned on the other patch, we need to enable the NBIO path to route the doorbell correctly. That’s the reason wy your doorbells are not working on Navi3x and Navi4x.
I am working on the change for Navi4x first and will send it out to you soon one I have verified it on my end.
Regards,
Mukul
Quoted 32 lines
Thanks & best regards,
TimurQuoted 26 lines
Quoted 24 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2-/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev-Quoted 2 lines
doorbell_index.ih + 2) << 1;
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2- WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM, - ih_v7_0_setup_retry_doorbell(adev-Quoted 2 lines
irq.retry_cam_doorbell_index));
-
/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL,ENABLE, 1);
--
2.54.0
Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0
Hi Timur,
For Navi4x, can you please give this change to enable the doorbell path a try in addition to your changes and use doorbells then to clear the CAM entries:
drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c +0 -0
@@ -251,7 +251,7 @@ static void nbif_v6_3_1_ih_doorbell_range(struct amdgpu_device *adev,
ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range, GDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL, S2A_DOORBELL_PORT1_RANGE_SIZE,
- 2);
+ 8);
Please let us know how it goes.
Thanks and regards,
Mukul
Quoted 162 lines · Mukul
AMD General
Quoted 113 lines
-----Original Message-----
From: Timur Kristóf<[email protected]>
Sent: Friday, July 3, 2026 2:41 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák<[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin <[email protected]>; Kuehling, Felix<[email protected]>; Lazar, Lijo <[email protected]>; He, Siwei<[email protected]>; Yang, Philip <[email protected]>; Joshi, Mukul<[email protected]>
Subject: Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0Quoted 14 lines · Mukul
AMD General
Quoted 11 lines
-----Original Message-----
From: amd-gfx<[email protected]> On Behalf Of Timur Kristóf
Sent: Wednesday, July 1, 2026 12:17 PM To:[email protected]; Deucher, Alexander <[email protected]>; Koenig, Christian <[email protected]>; Natalie Vock<[email protected]>; Shetaia, Amir<[email protected]>; Marek Olšák <[email protected]>; Limonciello, Mario<[email protected]>; Tvrtko Ursulin<[email protected]>; Kuehling, Felix <[email protected]>; Lazar, Lijo<[email protected]>; He, Siwei<[email protected]>; Yang, Philip
Quoted 68 lines
Quoted 68 lines
<[email protected]>; Joshi, Mukul<[email protected]>
Cc: Timur Kristóf<[email protected]>
Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0The doorbell is not working on Navi 48.
Use theIH_RETRY_CAM_ACKregister just like on Navi 3x.Signed-off-by Timur Kristóf<[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-9@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(structamdgpu_device *adev, return 0; } -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{ -u32 val =0 ; - - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,OFFSET , doorbell_index); - val =REG_SET_FIELD (val,IH_DOORBELL_RPTR ,ENABLE ,1 ); - -return val; -} - static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
cam_index) {drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-1-WDOORBELL32 (adev->irq.retry_cam_doorbell_index, cam_index); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_CAM_ACK, cam_index); }#define regIH_RING1_CLIENT_CFG_INDEX_V7_1 0x122@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct
amdgpu_device
*adev)pci_set_master(adev->pdev); if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7,1, 0))
Quoted 8 lines
Quoted 1 line
{
We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same. The code being removed below is for IH 7.1 and not 7.0.
Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH 7.1.
I would recommend to drop this patch.Hello Mukul,
Unfortunately I couldn't figure out how to get the doorbell working on Navi 48 and in the previous thread Amir said that it also didn't work for him, and that's why I decided to try the MMIO register.
Quoted 7 lines
If you have an idea how to get the doorbell to work on Navi 48, I'm happy to try your suggestion.
Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to keep the code maintainable. If it's not possible then we can also have a slightly different code path for the two different IH versions here.
Hi Timur,
As I had mentioned on the other patch, we need to enable the NBIO path to route the doorbell correctly. That’s the reason wy your doorbells are not working on Navi3x and Navi4x.
I am working on the change for Navi4x first and will send it out to you soon one I have verified it on my end.
Regards,
MukulQuoted 27 lines
Thanks & best regards,
TimurQuoted 22 lines
Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2-/* Allocate the doorbell for IH Retry CAM */ - adev->irq.retry_cam_doorbell_index = (adev-Quoted 2 lines
doorbell_index.ih + 2) << 1;
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-2- WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM, - ih_v7_0_setup_retry_doorbell(adev-Quoted 2 lines
irq.retry_cam_doorbell_index));
-
/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL,ENABLE, 1);
--
2.54.0
PATCH 13/14 drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs
In reply to Timur KristófThe retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +10 -0
@@ -397,6 +397,16 @@ static int ih_v6_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); +if (!(adev->flags &AMD_IS_APU )) { +/* Enable IH Retry CAM */ + tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 ); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,CAM_SIZE ,0xF ); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL, tmp); + + adev->irq.retry_cam_enabled = true; + } +/* enable interrupts */ ret =ih_v6_0_toggle_interrupts (adev, true);if (ret)
Signature · 1 line
2.54.0
Re: [PATCH 13/14] drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs
Quoted 7 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>
Quoted 25 lines
--- drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/gpu/drm/amd/amdgpu/ih_v6_0.c +10
-0@@ -397,6 +397,16 @@ static int ih_v6_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); +if (!(adev->flags &AMD_IS_APU )) { +/* Enable IH Retry CAM */ + tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 ); + tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,CAM_SIZE ,0xF ); +WREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL, tmp); + + adev->irq.retry_cam_enabled = true; + } +/* enable interrupts */ ret =ih_v6_0_toggle_interrupts (adev, true);if (ret)
PATCH 14/14 drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
In reply to Timur KristófThe retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +2 -1
@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); -if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) { +if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 ) || + !(adev->flags &AMD_IS_APU )) {/* Enable IH Retry CAM */ tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 );
Signature · 1 line
2.54.0
Christian König
maintainerRe: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
Quoted 16 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>--- drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev) · line 394
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +2
-1@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)pci_set_master (adev->pdev); -if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 )) { +if (amdgpu_ip_version (adev,OSSSYS_HWIP ,0 ) ==IP_VERSION (7 ,1 ,0 ) || + !(adev->flags &AMD_IS_APU )) {
I think the check should be the other way around.
In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.
Apart from that looks good to me,
Christian.
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0/* Enable IH Retry CAM */ tmp =RREG32_SOC15 (OSSSYS ,0 , regIH_RETRY_INT_CAM_CNTL); tmp =REG_SET_FIELD (tmp,IH_RETRY_INT_CAM_CNTL ,ENABLE ,1 );
Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
Quoted 21 lines · On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König…
Quoted 20 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.cindex291326d2ee8b..ec0919fa8254100644drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev)
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-1pci_set_master (adev->pdev); - if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,
0)) {
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 394
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-0+ if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)
||
> > + !(adev->flags & AMD_IS_APU)) {Quoted 5 lines
I think the check should be the other way around.
In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.
What do you mean by the other way around?
The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that version the retry CAM is always enabled regardless of whether it's an APU or dGPU.
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 396
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);
Quoted 1 line
Quoted 1 line
tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL, ENABLE,
1);
Christian König
maintainerRe: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
Quoted 36 lines · Timur Kristóf
Quoted 21 lines · On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König…
Quoted 20 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.cindex291326d2ee8b..ec0919fa8254100644drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev)
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-1pci_set_master (adev->pdev); - if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,0)) {
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 394
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-0+ if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)|| >> + !(adev->flags & AMD_IS_APU)) {Quoted 5 lines
I think the check should be the other way around.
In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.
Quoted 6 lines
What do you mean by the other way around?
The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that version the retry CAM is always enabled regardless of whether it's an APU or dGPU.
I just wanted to make the logic more readable. In other words:
if (!(adev->flags & AMD_IS_APU) ||
amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
...
I still need to double check why we have the exception for OSSSYS 7.1.
Could be that this is for some MI* product, but I'm not 100% sure yet.
Regards,
Christian.
Quoted 11 lines
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 396
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);Quoted 1 line
Quoted 1 line
tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL, ENABLE,1);
Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
Quoted 47 lines · On 2026. július 2., csütörtök 14:30:20 közép-európai nyári idő Christian König…
Quoted 39 lines · Timur Kristóf
Quoted 21 lines · On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian…
Quoted 20 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.cindex291326d2ee8b..ec0919fa8254100644drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev)
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-1pci_set_master (adev->pdev); - if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,0)) {
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev) · line 394
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +2
-0+ if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0) + !(adev->flags &AMD_IS_APU )) {I think the check should be the other way around.
In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.
Quoted 6 lines
What do you mean by the other way around?
The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that version the retry CAM is always enabled regardless of whether it's an APU or dGPU.
Quoted 4 lines
I just wanted to make the logic more readable. In other words:
if (!(adev->flags &
AMD_IS_APU) ||amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
Thanks, and sorry for the confusion.
I'll apply this suggestion to the next version of the series.
Quoted 4 lines
I still need to double check why we have the exception for OSSSYS 7.1.
Could be that this is for some MI* product, but I'm not 100% sure yet.
It was added in commit e06d19420118
I don't know which products have IH 7.1 because the IH block versions are not mentioned in the ASIC info table. I guess IH 7.1 goes with GFX 12.1 but I don't have any way to verify that guess.
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);Quoted 1 line
Quoted 1 line
tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL, ENABLE,1);
Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs
In reply to Christian KönigQuoted 49 lines · Christian König
Quoted 36 lines · Timur Kristóf
Quoted 21 lines · On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König…
Quoted 20 lines · Timur Kristóf
The retry CAM can filter interrupts which occur repeatedly, such as page fault interrupts when retry faults are enabled. This makes processing those interrupts much more efficient, because the CPU won't have to deal with processing the same interrupt repeatedly.
Signed-off-by Timur Kristóf <[email protected]>---
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.cindex291326d2ee8b..ec0919fa8254100644drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device *adev)
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-1pci_set_master (adev->pdev); - if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,0)) {
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 394
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +1
-0+ if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)|| >> + !(adev->flags & AMD_IS_APU)) {Quoted 5 lines
I think the check should be the other way around.
In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.
Quoted 6 lines
What do you mean by the other way around?
The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that version the retry CAM is always enabled regardless of whether it's an APU or dGPU.
I just wanted to make the logic more readable. In other words:
Quoted 7 lines
if (!(adev->flags &
AMD_IS_APU) ||amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))...
I still need to double check why we have the exception for OSSSYS 7.1.
Could be that this is for some MI* product, but I'm not 100% sure yet.
oss 7.1 is for an datacenter part.
Alex
Quoted 16 lines
Regards,
Christian.Quoted 11 lines
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c static int ih_v7_0_irq_init(struct amdgpu_device · line 396
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/ih_v7_0.c +0
-0/* Enable IH Retry CAM */ tmp = RREG32_SOC15(OSSSYS, 0,
regIH_RETRY_INT_CAM_CNTL);Quoted 1 line
Quoted 1 line
tmp = REG_SET_FIELD(tmp,
IH_RETRY_INT_CAM_CNTL, ENABLE,1);