use after on interrupt thread during EAL cleanup
https://bugs.dpdk.org/show_bug.cgi?id=1683
Bug ID: 1683
Summary: use after on interrupt thread during EAL cleanup
Product: DPDK
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: core
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
This was caught with ASan in a CI run in my GHA:
+ devtools/test-null.sh
EAL: Detected CPU lcores: 4
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /run/user/1001/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
testpmd: create a new mbuf pool <mb_pool_0>: n=2048, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc Interactive-mode selected
Auto-start selected
Configuring Port 0 (socket 0)
...
Shutting down port 1...
=================================================================
==46768==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000008e44 at pc 0x5613ec543091 bp 0x7f87065fd1d0 sp 0x7f87065fd1c8
READ of size 4 at 0x606000008e44 thread T1
#0 0x5613ec543090 in rte_intr_fd_get
/home/runner/work/dpdk/dpdk/build/../lib/eal/common/eal_common_interrupts.c:210:22
#1 0x5613ec5a350b in eal_alarm_callback
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_alarm.c:125:19
#2 0x5613ec5acef1 in eal_intr_process_interrupts
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_interrupts.c:1026:5
#3 0x5613ec5acef1 in eal_intr_handle_interrupts
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_interrupts.c:1100:7
#4 0x5613ec5aba06 in eal_intr_thread_main
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_interrupts.c:1172:3
#5 0x7f870b294ac2 in start_thread nptl/./nptl/pthread_create.c:442:8
#6 0x7f870b32684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:810x606000008e44 is located 4 bytes inside of 64-byte region [0x606000008e40,0x606000008e80)
freed by thread T0 here:
#0 0x5613eb24ba32 in free
(/home/runner/work/dpdk/dpdk/build/app/dpdk-testpmd+0xb8ca32) (BuildId:
1c1d93dcb4fec8b525f3ed4b97885f75ba17fcb1)
#1 0x5613ec5a1b7f in rte_eal_cleanup
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal.c:1333:2
#2 0x5613eb3bf7bc in main
/home/runner/work/dpdk/dpdk/build/../app/test-pmd/testpmd.c:4583:8
#3 0x7f870b229d8f in __libc_start_call_maincsu/../sysdeps/nptl/libc_start_call_main.h:58:16
previously allocated by thread T0 here:
#0 0x5613eb24bec8 in __interceptor_calloc
(/home/runner/work/dpdk/dpdk/build/app/dpdk-testpmd+0xb8cec8) (BuildId:
1c1d93dcb4fec8b525f3ed4b97885f75ba17fcb1)
#1 0x5613ec542b56 in rte_intr_instance_alloc
/home/runner/work/dpdk/dpdk/build/../lib/eal/common/eal_common_interrupts.c:51:17
#2 0x5613ec5a26ed in rte_eal_alarm_init
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_alarm.c:66:16
#3 0x5613ec59f5a3 in rte_eal_init
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal.c:1011:6
#4 0x5613eb3be5b3 in main
/home/runner/work/dpdk/dpdk/build/../app/test-pmd/testpmd.c:4369:9
#5 0x7f870b229d8f in __libc_start_call_maincsu/../sysdeps/nptl/libc_start_call_main.h:58:16
Thread T1 created by T0 here:
#0 0x5613eb23515c in __interceptor_pthread_create
(/home/runner/work/dpdk/dpdk/build/app/dpdk-testpmd+0xb7615c) (BuildId:
1c1d93dcb4fec8b525f3ed4b97885f75ba17fcb1)
#1 0x5613ec59d50c in rte_thread_create
/home/runner/work/dpdk/dpdk/build/../lib/eal/unix/rte_thread.c:199:8
#2 0x5613ec56011b in rte_thread_create_control
/home/runner/work/dpdk/dpdk/build/../lib/eal/common/eal_common_thread.c:308:8
#3 0x5613ec56096c in rte_thread_create_internal_control
/home/runner/work/dpdk/dpdk/build/../lib/eal/common/eal_common_thread.c:358:9
#4 0x5613ec5ab811 in rte_eal_intr_init
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal_interrupts.c:1200:8
#5 0x5613ec59f58a in rte_eal_init
/home/runner/work/dpdk/dpdk/build/../lib/eal/linux/eal.c:1006:6
#6 0x5613eb3be5b3 in main
/home/runner/work/dpdk/dpdk/build/../app/test-pmd/testpmd.c:4369:9
#7 0x7f870b229d8f in __libc_start_call_maincsu/../sysdeps/nptl/libc_start_call_main.h:58:16
SUMMARY: AddressSanitizer: heap-use-after-free /home/runner/work/dpdk/dpdk/build/../lib/eal/common/eal_common_interrupts.c:210:22 in rte_intr_fd_get
Shadow bytes around the buggy address:
0x0c0c7fff9170: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa 0x0c0c7fff9180: fd fd fd fd fd fd fd fa fa fa fa fa 00 00 00 00 0x0c0c7fff9190: 00 00 06 fa fa fa fa fa fd fd fd fd fd fd fd fa 0x0c0c7fff91a0: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa 0x0c0c7fff91b0: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
=>0x0c0c7fff91c0: fd fd fd fa fa fa fa fa[fd]fd fd fd fd fd fd fd
0x0c0c7fff91d0: fa fa fa fa 00 00 00 00 00 00 00 00 fa fa fa fa 0x0c0c7fff91e0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd 0x0c0c7fff91f0: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd 0x0c0c7fff9200: fa fa fa fa 00 00 00 00 00 00 00 00 fa fa fa fa 0x0c0c7fff9210: 00 00 00 00 00 00 00 00 fa fa fa fa 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb
==46768==ABORTING
From a quick reading of the cleanup code, the reason is probably that the interrupt thread was not killed before releasing the interrupt handler in rte_eal_alarm_cleanup() call.
There may be a need for killing the interrupt thread or adding some synchronisation point.
This issue probably affects all OS implementations.
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
https://bugs.dpdk.org/show_bug.cgi?id=1683
David Marchand ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |RESOLVED
Resolution|--- |FIXED--- Comment #7 from David Marchand ([email protected]) --- Fixed in commits cf1937a96dcf ("eal/freebsd: unregister alarm callback before free") and d84bf0d9aeb4 ("eal/linux: unregister alarm callback before free").
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.