amt.sh crashes smcrouted
Subsystems (2)
Jakub Kicinski
maintainerHi Taehee Yoo!
We started running amt tests in the netdev CI, and it looks like it hangs - or at least it doesn't produce any output for long enough for the test runner to think it hung.
While looking at the logs, however, I see:
[ 3.361660] smcrouted[294]: segfault at 7fff480c95f3 ip 00000000004034e4 sp 00007fff480b9410 error 6 in smcrouted[402000+a000] likely on CPU 3 (core 3, socket 0) [ 3.361812] Code: 74 24 38 89 ef e8 4c 33 00 00 44 0f b7 f8 66 39 84 24 e2 01 00 00 75 09 45 85 ed 0f 85 ed 01 00 00 48 8b 44 24 38 0f b6 40 33 <42> 88 84 3c e4 01 00 00 48 8b 3b 48 8d 54 24 38 48 8d 74 24 50 e8
https://netdev-3.bots.linux.dev/vmksft-net/results/577882/4-amt-sh/
So I think the cause may be a bug in smcroute.
We use smcroute build from latest git
# cd smcroute/ # git log -1 --oneline cd25930 .github: use same CFLAGS for both configure runs # smcroute -v SMCRoute v2.5.6
Could you check if you can repro this crash?
Quoted 1 line · Jakub Kicinski
Hi Jakub,
Thanks a lot for the report!
Quoted 15 lines
Hi Taehee Yoo!
We started running amt tests in the netdev CI, and it looks like it hangs - or at least it doesn't produce any output for long enough for the test runner to think it hung.
While looking at the logs, however, I see:
[ 3.361660] smcrouted[294]: segfault at 7fff480c95f3 ip 00000000004034e4 sp 00007fff480b9410 error 6 in smcrouted[402000+a000] likely on CPU 3 (core 3, socket 0) [ 3.361812] Code: 74 24 38 89 ef e8 4c 33 00 00 44 0f b7 f8 66 39 84 24 e2 01 00 00 75 09 45 85 ed 0f 85 ed 01 00 00 48 8b 44 24 38 0f b6 40 33 <42> 88 84 3c e4 01 00 00 48 8b 3b 48 8d 54 24 38 48 8d 74 24 50 e8https://netdev-3.bots.linux.dev/vmksft-net/results/577882/4-amt-sh/
So I think the cause may be a bug in smcroute.
Quoted 8 lines
We use smcroute build from latest git
# cd smcroute/ # git log -1 --oneline cd25930 .github: use same CFLAGS for both configure runs # smcroute -v SMCRoute v2.5.6Could you check if you can repro this crash?
I tried to reproduce the latest version of the smcrouted crash, but I couldn't reproduce it.
I'm sure this crash is the reason for the failure of your case. But the real bug of this phenomenon is the amt.sh doesn't have timeout logic. If the smcrouted did crash or it couldn't finish in time something in it, it should print FAIL and then quit this test, but it waits forever.
I will send a patch for it.
Thank you so much for taking care of it.
Taehee Yoo