Skip to contentSkip to conversation

test/interrupt: account for race with callback

PATCH Applied Aaron Conole 3 participants 4 messages Aug 8, 2019 – Sep 13, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 test/interrupt: account for race with callback

Because the eal interrupt framework can race when invoking the callback and a separate unregister call, the test needs to accommodate the chance that the two collide. Do this by checking the return value of unregister against the race-condition flag (EAGAIN).

Fixes f1a6c22424ce ("app/test: update interrupts test")

---
NOTE: it's difficult to reproduce this race. I tried a bit, but have
only seen it sporadically. In Travis environment, the CPU resource can be very limited and this test is quite racy.

 app/test/test_interrupts.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
app/test/test_interrupts.c +7 -3
index d8c2d8124..233b14a70 100644
@@ -370,9 +370,13 @@ test_interrupt_full_path_check(enum test_interrupt_handle_type intr_type)
 		rte_delay_ms(TEST_INTERRUPT_CHECK_INTERVAL);
 
 	rte_delay_ms(TEST_INTERRUPT_CHECK_INTERVAL);
-	if (rte_intr_callback_unregister(&test_intr_handle,
-			test_interrupt_callback, &test_intr_handle) < 0)
-		return -1;
+	while ((count =
+		rte_intr_callback_unregister(&test_intr_handle,
+					     test_interrupt_callback,
+					     &test_intr_handle)) < 0) {
+		if (count != -EAGAIN)
+			return -1;
+	}
 
 	if (flag == 0) {
 		printf("callback has not been called\n");
Signature · 1 line
2.21.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 7 lines · Aaron Conole

Because the eal interrupt framework can race when invoking the callback and a separate unregister call, the test needs to accommodate the chance that the two collide. Do this by checking the return value of unregister against the race-condition flag (EAGAIN).

Fixes f1a6c22424ce ("app/test: update interrupts test")

Not too sure about this tag, but anyway, this is old enough to apply to every stable releases we have.
Cc: [email protected] ?

Quoted 5 lines

---
NOTE: it's difficult to reproduce this race. I tried a bit, but have
only seen it sporadically. In Travis environment, the CPU resource can be very limited and this test is quite racy.

Managed to reproduce it:

# time (log=/tmp/$$.log; while true; do echo interrupt_autotest
|taskset -c 0-1 ./build-gcc-static/app/test/dpdk-test -l 0-1 >$log
2>&1; grep -q 'Test OK' $log || break; done; cat $log; rm -f $log)
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:15d7 net_e1000_em
APP: HPET is not enabled, using TSC as default timer
RTE>>interrupt_autotest
Check unknown valid interrupt full path
Check valid UIO interrupt full path
Check valid device event interrupt full path
count=-11 Resource temporarily unavailable
failure occurred during checking valid device event interrupt full path
Clearing for interrupt tests
Test Failed
RTE>>
real    0m38.081s
user    0m35.836s
sys    0m2.171s
Quoted 26 lines
 app/test/test_interrupts.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
app/test/test_interrupts.c +7 -4
index d8c2d8124..233b14a70 100644
@@ -370,9 +370,13 @@ test_interrupt_full_path_check(enum test_interrupt_handle_type intr_type)
                rte_delay_ms(TEST_INTERRUPT_CHECK_INTERVAL);
 
        rte_delay_ms(TEST_INTERRUPT_CHECK_INTERVAL);
-       if (rte_intr_callback_unregister(&test_intr_handle,
-                       test_interrupt_callback, &test_intr_handle) < 0)
-               return -1;
+       while ((count =
+               rte_intr_callback_unregister(&test_intr_handle,
+                                            test_interrupt_callback,
+                                            &test_intr_handle)) < 0) {
+               if (count != -EAGAIN)
+                       return -1;
+       }
 
        if (flag == 0) {
                printf("callback has not been called\n");
--
Quoted 2 lines

2.21.0

With this patch, my loop has been running for more than 10 minutes now.

--
David Marchand

#2 lore raw

09/08/2019 11:18, David Marchand:

Quoted 47 lines · Aaron Conole
Quoted 7 lines

Because the eal interrupt framework can race when invoking the callback and a separate unregister call, the test needs to accommodate the chance that the two collide. Do this by checking the return value of unregister against the race-condition flag (EAGAIN).

Fixes f1a6c22424ce ("app/test: update interrupts test")

Not too sure about this tag, but anyway, this is old enough to apply to every stable releases we have.
Cc: [email protected] ?

Quoted 5 lines

---
NOTE: it's difficult to reproduce this race. I tried a bit, but have
only seen it sporadically. In Travis environment, the CPU resource can be very limited and this test is quite racy.

Managed to reproduce it:

# time (log=/tmp/$$.log; while true; do echo interrupt_autotest
|taskset -c 0-1 ./build-gcc-static/app/test/dpdk-test -l 0-1 >$log
2>&1; grep -q 'Test OK' $log || break; done; cat $log; rm -f $log)
EAL: Detected 8 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:00:1f.6 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 8086:15d7 net_e1000_em
APP: HPET is not enabled, using TSC as default timer
RTE>>interrupt_autotest
Check unknown valid interrupt full path
Check valid UIO interrupt full path
Check valid device event interrupt full path
count=-11 Resource temporarily unavailable
failure occurred during checking valid device event interrupt full path
Clearing for interrupt tests
Test Failed
RTE>>
real    0m38.081s
user    0m35.836s
sys    0m2.171s
Quoted 2 lines

With this patch, my loop has been running for more than 10 minutes now.

Applied, thanks

#3 lore raw
In reply to Aaron Conole
Quoted 8 lines · Aaron Conole

Because the eal interrupt framework can race when invoking the callback and a separate unregister call, the test needs to accommodate the chance that the two collide. Do this by checking the return value of unregister against the race-condition flag (EAGAIN).

Fixes f1a6c22424ce ("app/test: update interrupts test")

---

Ping. Still see these failures.