drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
Subsystems (3)
PATCH 1/1 drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
kfd_pre_reset will free mem_objs allocated by kfd_gtt_sa_allocate
Without this change, sriov tdr code path will never free those allocated memories and get memory leak.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
@@ -3847,6 +3847,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, {int r; +amdgpu_amdkfd_pre_reset (adev); +if (from_hypervisor) r =amdgpu_virt_request_full_gpu (adev, true);else
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
-----Original Message-----
From: Jack Zhang <[email protected]>
Sent: Thursday, April 2, 2020 3:20 PM
To: [email protected]
Cc: Zhang, Jack (Jian) <[email protected]>
Subject: [PATCH] drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
kfd_pre_reset will free mem_objs allocated by kfd_gtt_sa_allocate
Without this change, sriov tdr code path will never free those allocated memories and get memory leak.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -1
@@ -3847,6 +3847,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, {int r; +amdgpu_amdkfd_pre_reset (adev); +if (from_hypervisor) r =amdgpu_virt_request_full_gpu (adev, true);else --
2.7.4
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
[+Monk]
This looks reasonable to me. However, you're effectively reverting this commit by Monk:
a03eb637d2a5 drm/amdgpu: fix KIQ ring test fail in TDR of SRIOV
In hind-sight, Monk's commit was broken. Removing the call to pre_reset has other consequences, such as breaking notifications about reset to user mode, and probably invalidating some assumptions in kfd_post_reset. Can you coordinate with Monk to work out why his change was needed, and whether you'll need a different solution for the problem he was trying to address?
In the meanwhile, this patch is
Am 2020-04-02 um 3:20 a.m. schrieb Jack Zhang:
Quoted 15 lines
kfd_pre_resetwill freemem_objsallocated bykfd_gtt_sa_allocateWithout this change, sriov tdr code path will never free those allocated memories and get memory leak.
Signed-off-by Jack Zhang <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · line 3854
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2
-0@@ -3847,6 +3847,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, {int r; +amdgpu_amdkfd_pre_reset (adev); +if (from_hypervisor) r =amdgpu_virt_request_full_gpu (adev, true);else
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Liu, Monk
Acked-by Felix KuehlingThanks Felix
Hi Jack
I think below changes can resolve your problem , we had this on our customer branch already, it fix the memory leak, and also fix my previous bug . Can you make this change applied to gfx_v10/v9 ? thanks !
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c +2 -0
@@ -543,6 +543,8 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,uint32_t temp;struct v10_compute_mqd *m =get_mqd (mqd); +if (amdgpu_sriov_vf (adev) && adev->in_gpu_reset) +return 0 ;#if 0 unsigned long flags;int retry;
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
@@ -3653,6 +3653,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev,if (r)return r; +amdgpu_amdkfd_pre_reset (adev); +/* Resume IP prior to SMC */ r =amdgpu_device_ip_reinit_early_sriov (adev);if (r)
_____________________________________
Monk Liu|GPU Virtualization Team |AMD
-----Original Message-----
From: Kuehling, Felix <[email protected]>
Sent: Friday, April 3, 2020 1:26 AM
To: Zhang, Jack (Jian) <[email protected]>; [email protected]; Liu, Monk <[email protected]>
Subject: Re: [PATCH] drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
[+Monk]
This looks reasonable to me. However, you're effectively reverting this commit by Monk:
a03eb637d2a5 drm/amdgpu: fix KIQ ring test fail in TDR of SRIOV
In hind-sight, Monk's commit was broken. Removing the call to pre_reset has other consequences, such as breaking notifications about reset to user mode, and probably invalidating some assumptions in kfd_post_reset. Can you coordinate with Monk to work out why his change was needed, and whether you'll need a different solution for the problem he was trying to address?
In the meanwhile, this patch is
Am 2020-04-02 um 3:20 a.m. schrieb Jack Zhang:
Quoted 16 lines
kfd_pre_resetwill freemem_objsallocated bykfd_gtt_sa_allocateWithout this change, sriov tdr code path will never free those allocated memories and get memory leak.
Signed-off-by Jack Zhang <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_reset_sriov(struct · line 3854
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2
-0@@ -3847,6 +3847,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, { int r; +amdgpu_amdkfd_pre_reset (adev); +if (from_hypervisor) r =amdgpu_virt_request_full_gpu (adev, true);else
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Zhang, Jack (Jian)
Acked-by Felix KuehlingThanks Monk,
I just updated the patch and it could passed 1000 rounds TDR test.
Sent out an review email.
Regards,
Jack
-----Original Message-----
From: Liu, Monk <[email protected]>
Sent: Friday, April 3, 2020 11:38 AM
To: Kuehling, Felix <[email protected]>; Zhang, Jack (Jian) <[email protected]>; [email protected]
Subject: RE: [PATCH] drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
Thanks Felix
Hi Jack
I think below changes can resolve your problem , we had this on our customer branch already, it fix the memory leak, and also fix my previous bug . Can you make this change applied to gfx_v10/v9 ? thanks !
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.c +2 -0
@@ -543,6 +543,8 @@ static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd,uint32_t temp;struct v10_compute_mqd *m =get_mqd (mqd); +if (amdgpu_sriov_vf (adev) && adev->in_gpu_reset) +return 0 ;#if 0 unsigned long flags;int retry;
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
@@ -3653,6 +3653,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev,if (r)return r; +amdgpu_amdkfd_pre_reset (adev); +/* Resume IP prior to SMC */ r =amdgpu_device_ip_reinit_early_sriov (adev);if (r)
_____________________________________
Monk Liu|GPU Virtualization Team |AMD
-----Original Message-----
From: Kuehling, Felix <[email protected]>
Sent: Friday, April 3, 2020 1:26 AM
To: Zhang, Jack (Jian) <[email protected]>; [email protected]; Liu, Monk <[email protected]>
Subject: Re: [PATCH] drm/amdgpu/sriov add amdgpu_amdkfd_pre_reset in gpu reset
[+Monk]
This looks reasonable to me. However, you're effectively reverting this commit by Monk:
a03eb637d2a5 drm/amdgpu: fix KIQ ring test fail in TDR of SRIOV
In hind-sight, Monk's commit was broken. Removing the call to pre_reset has other consequences, such as breaking notifications about reset to user mode, and probably invalidating some assumptions in kfd_post_reset. Can you coordinate with Monk to work out why his change was needed, and whether you'll need a different solution for the problem he was trying to address?
In the meanwhile, this patch is
Am 2020-04-02 um 3:20 a.m. schrieb Jack Zhang:
Quoted 16 lines
kfd_pre_resetwill freemem_objsallocated bykfd_gtt_sa_allocateWithout this change, sriov tdr code path will never free those allocated memories and get memory leak.
Signed-off-by Jack Zhang <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_device_reset_sriov(struct · line 3854
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2
-0@@ -3847,6 +3847,8 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, { int r; +amdgpu_amdkfd_pre_reset (adev); +if (from_hypervisor) r =amdgpu_virt_request_full_gpu (adev, true);else
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx