Skip to contentSkip to conversation

drm/amdgpu: don't disable ttm buffer funcs on reset

PATCH Superseded Pierre-Eric Pelloux-Prayer 3 participants 5 messages Jun 17 – Aug 4

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 drm/amdgpu: don't disable ttm buffer funcs on reset

Access to hw is already suspended so there are no reasons to disable ttm buffer funcs.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 --
 1 file changed, 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -2
index 6608780ffef2..57e10afb4714 100644
@@ -3167,8 +3167,6 @@ static int amdgpu_device_ip_suspend(struct amdgpu_device *adev)
 		amdgpu_virt_request_full_gpu(adev, false);
 	}
 
-	amdgpu_ttm_disable_buffer_funcs(adev);
-
 	r = amdgpu_device_ip_suspend_phase1(adev);
 	if (r)
 		return r;
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 2/3 drm/amdgpu: delay ttm buffer func enablement on xgmi

When amdgpu_init_minimal_xgmi is used, SDMA engines init is delayed so amdgpu_ttm_enable_buffer_funcs must be called later.

Without this, the check for num_buffer_funcs_scheds will fail and using ttm buffer funcs later will fail.

---
v2: no need to add a new call. Instead move down the one from
amdgpu_device_reinit_after_reset after amdgpu_ib_ring_tests
---

Fixes 3a5da695c8d1 ("drm/amdgpu: only use working sdma schedulers for ttm")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -3
index 57e10afb4714..53150fe55350 100644
@@ -2479,7 +2479,8 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
 	if (r)
 		goto init_failed;
 
-	amdgpu_ttm_enable_buffer_funcs(adev);
+	if (amdgpu_ip_member_of_hwini(adev, AMD_IP_BLOCK_TYPE_SDMA))
+		amdgpu_ttm_enable_buffer_funcs(adev);
 
 	/* Don't init kfd if whole hive need to be reset during init */
 	if (adev->init_lvl->level != AMDGPU_INIT_LEVEL_MINIMAL_XGMI) {
@@ -5253,8 +5254,6 @@ int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context)
 				if (r)
 					goto out;
 
-				amdgpu_ttm_enable_buffer_funcs(tmp_adev);
-
 				r = amdgpu_device_ip_resume_phase3(tmp_adev);
 				if (r)
 					goto out;
@@ -5320,6 +5319,8 @@ int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context)
 				r = -EAGAIN;
 				goto end;
 			}
+
+			amdgpu_ttm_enable_buffer_funcs(tmp_adev);
 		}
 
 		if (r)
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 2/3] drm/amdgpu: delay ttm buffer func enablement on xgmi

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The relocated call to amdgpu_ttm_enable_buffer_funcs() silently fails due to the amdgpu_in_reset() guard, leaving TTM DMA permanently disabled. --

commit 61a1713a716e72510eef36a97d31d040637c47f6
Author: Pierre-Eric Pelloux-Prayer <[email protected]>

drm/amdgpu: delay ttm buffer func enablement on xgmi

When amdgpu_init_minimal_xgmi is used, SDMA engines init is delayed so amdgpu_ttm_enable_buffer_funcs must be called later. This commit moves the call into amdgpu_device_reinit_after_reset to avoid initialization failures.

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index 57e10afb4714d..53150fe553508 100644

[ ... ]

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context) · line 5323

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
@@ -5320,6 +5319,8 @@ int amdgpu_device_reinit_after_reset(struct amdgpu_reset_context *reset_context)
 				r = -EAGAIN;
 				goto end;
 			}
+
+			amdgpu_ttm_enable_buffer_funcs(tmp_adev);

[Severity: High]
This isn't a bug introduced by this patch, but doesn't this relocated call silently fail due to the amdgpu_in_reset() guard?

Because amdgpu_device_reinit_after_reset() executes within a hardware reset context, the in_gpu_reset flag is set. Looking at amdgpu_ttm_enable_buffer_funcs() in drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c, there is an early return guard for this:

	if (!adev->mman.initialized || amdgpu_in_reset(adev) ||
	    adev->mman.buffer_funcs_enabled || adev->gmc.is_app_apu)
		return;

Will this cause the function to silently abort, leaving TTM DMA permanently disabled and forcing all VRAM migrations to fallback to slow CPU copies?

Also, by placing the call at the very end of amdgpu_device_reinit_after_reset(), it executes after drm_client_dev_resume(). Would any VRAM migrations triggered by fbdev restore run without hardware DMA even if the early return was bypassed?

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
 		}
 
 		if (r)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#3 lore raw

PATCH v2 3/3 drm/amdgpu: remove useless enable/disable of ttm buffer funcs

In reply to Pierre-Eric Pelloux-Prayer

amdgpu_ttm_disable_buffer_funcs and amdgpu_ttm_enable_buffer_funcs should each only be called once per amdgpu_device: at startup and teardown.

Therefore the other calls are redundant and can be safely dropped.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 -----
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 1 -
 2 files changed, 6 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -5
index 53150fe55350..6abcde8a1a00 100644
@@ -3381,8 +3381,6 @@ static int amdgpu_device_ip_resume(struct amdgpu_device *adev)
 
 	r = amdgpu_device_ip_resume_phase2(adev);
 
-	amdgpu_ttm_enable_buffer_funcs(adev);
-
 	if (r)
 		return r;
 
@@ -4505,8 +4503,6 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
 	if (r)
 		goto unwind_userq;
 
-	amdgpu_ttm_disable_buffer_funcs(adev);
-
 	amdgpu_fence_driver_hw_fini(adev);
 
 	r = amdgpu_device_ip_suspend_phase2(adev);
@@ -4519,7 +4515,6 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
 	return 0;
 
 unwind_evict:
-	amdgpu_ttm_enable_buffer_funcs(adev);
 	amdgpu_fence_driver_hw_init(adev);
 
 unwind_userq:
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -1
index 6c6ab4dd6ea9..95f879e723ea 100644
@@ -2099,7 +2099,6 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 	}
 
 	/* Change the size here instead of the init above so only lpfn is affected */
-	amdgpu_ttm_disable_buffer_funcs(adev);
 #ifdef CONFIG_64BIT
 #ifdef CONFIG_X86
 	if (adev->gmc.xgmi.connected_to_cpu)
Signature · 1 line
2.43.0
Reviewed-by on v2 3/3
#4 lore raw

Re: [PATCH v2 3/3] drm/amdgpu: remove useless enable/disable of ttm buffer funcs

Quoted 5 lines · Pierre-Eric Pelloux-Prayer

amdgpu_ttm_disable_buffer_funcs and amdgpu_ttm_enable_buffer_funcs should each only be called once per amdgpu_device: at startup and teardown.

Therefore the other calls are redundant and can be safely dropped.

I think there were originally issues where SDMA was attempted to be used when it was not ready which is why we added all of these cases, but I think that should be handled properly now. Series is:

Quoted 52 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 5 -----
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 1 -
 2 files changed, 6 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -5
index 53150fe55350..6abcde8a1a00 100644
@@ -3381,8 +3381,6 @@ static int amdgpu_device_ip_resume(struct amdgpu_device *adev)
 
        r = amdgpu_device_ip_resume_phase2(adev);
 
-       amdgpu_ttm_enable_buffer_funcs(adev);
-
        if (r)
                return r;
 
@@ -4505,8 +4503,6 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
        if (r)
                goto unwind_userq;
 
-       amdgpu_ttm_disable_buffer_funcs(adev);
-
        amdgpu_fence_driver_hw_fini(adev);
 
        r = amdgpu_device_ip_suspend_phase2(adev);
@@ -4519,7 +4515,6 @@ int amdgpu_device_suspend(struct drm_device *dev, bool notify_clients)
        return 0;
 
 unwind_evict:
-       amdgpu_ttm_enable_buffer_funcs(adev);
        amdgpu_fence_driver_hw_init(adev);
 
 unwind_userq:
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +0 -2
index 6c6ab4dd6ea9..95f879e723ea 100644
@@ -2099,7 +2099,6 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
        }
 
        /* Change the size here instead of the init above so only lpfn is affected */
-       amdgpu_ttm_disable_buffer_funcs(adev);
 #ifdef CONFIG_64BIT
 #ifdef CONFIG_X86
        if (adev->gmc.xgmi.connected_to_cpu)
--

2.43.0