tests/amdgpu/amd_queue_reset: Fix random hang during queue reset stress
PATCH 1/1 tests/amdgpu/amd_queue_reset: Fix random hang during queue reset stress
A concurrent "good" background job can be collateral-cancelled by the queue reset and return an error the good-job path did not tolerate, tripping igt_assert_eq() and aborting the background process. Its death breaks the fixed NUM_CHILD_PROCESSES barrier, so the survivors block forever in sem_wait().
Tolerate every reset-induced error on good jobs: unify the two asymmetric allow-lists in amdgpu_test_exec_cs_helper() and add -EHWPOISON in run_background(), so the process never aborts and the barrier cannot deadlock.
--- lib/amdgpu/amd_command_submission.c | 14 ++++++++++---- tests/amdgpu/amd_queue_reset.c | 10 +++++++++- 2 files changed, 19 insertions(+), 5 deletions(-)
lib/amdgpu/amd_command_submission.c +10 -4
@@ -119,8 +119,11 @@ int amdgpu_test_exec_cs_helper(amdgpu_device_handle device, unsigned int ip_typeif (expect_failure)igt_info ("amdgpu_cs_submit %d PID %d\n" , r,getpid ());else { -/* we allow ECANCELED, ENODATA or -EHWPOISON for good jobs temporally */ -if (r != -ECANCELED && r != -ENODATA && r != -EHWPOISON ) +/* a good job may be collateral-cancelled by a concurrent +* queue reset; tolerate every reset-induced error +*/ +if (r != -ECANCELED && r != -ENODATA && r != -EHWPOISON && + r != -ETIME )igt_assert_eq (r,0 ); }
@@ -142,8 +145,11 @@ int amdgpu_test_exec_cs_helper(amdgpu_device_handle device, unsigned int ip_typeigt_info ("EXPECT FAILURE amdgpu_cs_query_fence_status %d\n" "expired %d PID %d\n" , r, expired,getpid ()); }else { -/* we allow ECANCELED or ENODATA for good jobs temporally */ -if (r != -ECANCELED && r != -ENODATA && r != -ETIME ) +/* a good job may be collateral-cancelled by a concurrent +* queue reset; tolerate every reset-induced error +*/ +if (r != -ECANCELED && r != -ENODATA && r != -ETIME && + r != -EHWPOISON )igt_assert_eq (r,0 ); } }
tests/amdgpu/amd_queue_reset.c +9 -1
@@ -840,7 +840,15 @@ run_background(amdgpu_device_handle device, struct shmbuf *sh_mem,//if entire gpu reset then stop back ground jobs break ; } -if (r != -ECANCELED && r != -ETIME && r != -ENODATA ) +/* +* A "good" background job can be collateral-cancelled by +* the concurrent queue reset triggered by the bad job. +* Tolerate every reset-induced error so the helper never +* aborts this process - its death would break the +* NUM_CHILD_PROCESSES barrier and hang the whole test. +*/ +if (r != -ECANCELED && r != -ETIME && r != -ENODATA && + r != -EHWPOISON )igt_assert_eq (r,0 );//igt_assert_eq(err_codes.err_code_wait_for_fence, job.reset_err_result); counter++;
Signature · 1 line
2.49.0
LGTM
Reviewed-by Vitaly Prosyak <[email protected]>
Quoted 69 lines · Jesse Zhang
A concurrent "good" background job can be collateral-cancelled by the queue reset and return an error the good-job path did not tolerate, tripping
igt_assert_eq()and aborting the background process. Its death breaks the fixedNUM_CHILD_PROCESSESbarrier, so the survivors block forever insem_wait().Tolerate every reset-induced error on good jobs: unify the two asymmetric allow-lists in
amdgpu_test_exec_cs_helper()and add-EHWPOISONinrun_background(), so the process never aborts and the barrier cannot deadlock.Signed-off-by Jesse Zhang <[email protected]>--- lib/amdgpu/amd_command_submission.c | 14 ++++++++++---- tests/amdgpu/amd_queue_reset.c | 10 +++++++++- 2 files changed, 19 insertions(+), 5 deletions(-)lib/amdgpu/amd_command_submission.c +10
-4@@ -119,8 +119,11 @@ int amdgpu_test_exec_cs_helper(amdgpu_device_handle device, unsigned int ip_typeif (expect_failure)igt_info ("amdgpu_cs_submit %d PID %d\n" , r,getpid ());else { -/* we allow ECANCELED, ENODATA or -EHWPOISON for good jobs temporally */ -if (r != -ECANCELED && r != -ENODATA && r != -EHWPOISON ) +/* a good job may be collateral-cancelled by a concurrent +* queue reset; tolerate every reset-induced error +*/ +if (r != -ECANCELED && r != -ENODATA && r != -EHWPOISON && + r != -ETIME )igt_assert_eq (r,0 ); }@@ -142,8 +145,11 @@ int amdgpu_test_exec_cs_helper(amdgpu_device_handle device, unsigned int ip_typeigt_info ("EXPECT FAILURE amdgpu_cs_query_fence_status %d\n" "expired %d PID %d\n" , r, expired,getpid ()); }else { -/* we allow ECANCELED or ENODATA for good jobs temporally */ -if (r != -ECANCELED && r != -ENODATA && r != -ETIME ) +/* a good job may be collateral-cancelled by a concurrent +* queue reset; tolerate every reset-induced error +*/ +if (r != -ECANCELED && r != -ENODATA && r != -ETIME && + r != -EHWPOISON )igt_assert_eq (r,0 ); } }tests/amdgpu/amd_queue_reset.c +9
-1@@ -840,7 +840,15 @@ run_background(amdgpu_device_handle device, struct shmbuf *sh_mem,//if entire gpu reset then stop back ground jobs break ; } -if (r != -ECANCELED && r != -ETIME && r != -ENODATA ) +/* +* A "good" background job can be collateral-cancelled by +* the concurrent queue reset triggered by the bad job. +* Tolerate every reset-induced error so the helper never +* aborts this process - its death would break the +* NUM_CHILD_PROCESSES barrier and hang the whole test. +*/ +if (r != -ECANCELED && r != -ETIME && r != -ENODATA && + r != -EHWPOISON )igt_assert_eq (r,0 );//igt_assert_eq(err_codes.err_code_wait_for_fence, job.reset_err_result); counter++;
Patchwork
CI✗ Fi.CI.BAT: failure for tests/amdgpu/amd_queue_reset: Fix random hang during queue reset stress
In reply to Jesse Zhang== Series Details ==
== Series Details ==
Series: tests/amdgpu/amd_queue_reset: Fix random hang during queue reset stress
URL : https://patchwork.freedesktop.org/series/170103/
State : failure
== Summary ==
Series 170103 revision 1 was fully merged or fully failed: no git log