iwd 1.23 crashes in eapol_handle_pkt_1_of_4()
A message that is not in the archive (sent off-list or not yet received): [email protected]
Toke Høiland-Jørgensen <toke at toke.dk>
Hi
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes. I rebuilt iwd with debug symbols enabled and got the following backtrace out of 'coredumpctl gdb':
Core was generated by `/usr/lib/iwd/iwd'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00005555f0ba8b3d in eapol_handle_ptk_1_of_4 (sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) at src/eapol.c:1236
1236 handshake_event(sm->handshake,
(gdb) bt
#0 0x00005555f0ba8b3d in eapol_handle_ptk_1_of_4 (sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) at src/eapol.c:1236
#1 0x00005555f0bab118 in eapol_key_handle (unencrypted=<optimized out>, frame=<optimized out>, sm=0x5555f2b4a920) at src/eapol.c:2343
#2 eapol_rx_packet (proto=<optimized out>, from=<optimized out>, frame=<optimized out>, unencrypted=<optimized out>, user_data=0x5555f2b4a920) at src/eapol.c:2665
#3 0x00005555f0bac497 in __eapol_rx_packet (ifindex=62, src=src(a)entry=0x5555f2b62574 "x\212 J\207\267", proto=proto(a)entry=34958, frame=frame(a)entry=0x5555f2b62588 "\002\003",
len=len(a)entry=121, noencrypt=noencrypt(a)entry=false) at src/eapol.c:3017
#4 0x00005555f0b8c617 in netdev_control_port_frame_event (netdev=0x5555f2b64450, msg=0x5555f2b62588) at src/netdev.c:5574
#5 netdev_unicast_notify (msg=msg(a)entry=0x5555f2b619a0, user_data=<optimized out>) at src/netdev.c:5613
#6 0x00007f60084c9a51 in dispatch_unicast_watches (msg=0x5555f2b619a0, id=<optimized out>, genl=0x5555f2b3fc80) at ell/genl.c:954
#7 process_unicast (nlmsg=0x7fff61abeac0, genl=0x5555f2b3fc80) at ell/genl.c:973
#8 received_data (io=<optimized out>, user_data=0x5555f2b3fc80) at ell/genl.c:1098
#9 0x00007f60084c61bd in io_callback (fd=<optimized out>, events=1, user_data=0x5555f2b3fd20) at ell/io.c:120#10 0x00007f60084c536d in l_main_iterate (timeout=<optimized out>) at ell/main.c:478
#11 0x00007f60084c543e in l_main_run () at ell/main.c:525
#12 l_main_run () at ell/main.c:507
#13 0x00007f60084c5670 in l_main_run_with_signal (callback=callback(a)entry=0x5555f0b89150 <signal_handler>, user_data=user_data(a)entry=0x0) at ell/main.c:647 #14 0x00005555f0b886a4 in main (argc=<optimized out>, argv=<optimized out>) at src/main.c:532
Does anyone have any ideas? :)
Thanks!
-Toke
Diederik de Haas
Quoted 4 lines · On 2 February 2022 13:05:56 CET Toke Høiland-Jørgensen wrote:
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes.
Does anyone have any ideas? :)
According to https://bugs.debian.org/1004607, iwd 1.23 needs a newer version of ELL (version 0.48), so maybe you need to upgrade ELL too?
Toke Høiland-Jørgensen <toke at toke.dk>
Quoted 8 lines · Diederik de Haas
Quoted 4 lines · On 2 February 2022 13:05:56 CET Toke Høiland-Jørgensen wrote:
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes.
Does anyone have any ideas? :)
According to https://bugs.debian.org/1004607, iwd 1.23 needs a newer version of ELL (version 0.48), so maybe you need to upgrade ELL too?
Nope, that's not it - already running ell v0.48 :)
-Toke
Denis Kenzior
PATCH 1/1 handshake: Do not crash if handshake is destroyed
In reply to Toke Høiland-Jørgensen <toke at toke.dk>Commit 4d2176df2985 ("handshake: Allow event handler to free handshake") introduced a re-entrancy guard so that handshake_state objects that are destroyed as a result of the event do not cause a crash. It rightly used a temporary object to store the passed in handshake. Unfortunately this caused variable shadowing which resulted in crashes fixed by commit d22b174a7318 ("handshake: use _hs directly in handshake_event"). However, since the temporary was no longer used, this fix itself caused a crash:
#0 0x00005555f0ba8b3d in eapol_handle_ptk_1_of_4 (sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) at src/eapol.c:1236
1236 handshake_event(sm->handshake,
(gdb) bt
#0 0x00005555f0ba8b3d in eapol_handle_ptk_1_of_4 (sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) at src/eapol.c:1236 #1 0x00005555f0bab118 in eapol_key_handle (unencrypted=<optimized out>, frame=<optimized out>, sm=0x5555f2b4a920) at src/eapol.c:2343 #2 eapol_rx_packet (proto=<optimized out>, from=<optimized out>, frame=<optimized out>, unencrypted=<optimized out>, user_data=0x5555f2b4a920) at src/eapol.c:2665 #3 0x00005555f0bac497 in __eapol_rx_packet (ifindex=62, src=src(a)entry=0x5555f2b62574 "x\212 J\207\267", proto=proto(a)entry=34958, frame=frame(a)entry=0x5555f2b62588 "\002\003", len=len(a)entry=121, noencrypt=noencrypt(a)entry=false) at src/eapol.c:3017 #4 0x00005555f0b8c617 in netdev_control_port_frame_event (netdev=0x5555f2b64450, msg=0x5555f2b62588) at src/netdev.c:5574 #5 netdev_unicast_notify (msg=msg(a)entry=0x5555f2b619a0, user_data=<optimized out>) at src/netdev.c:5613 #6 0x00007f60084c9a51 in dispatch_unicast_watches (msg=0x5555f2b619a0, id=<optimized out>, genl=0x5555f2b3fc80) at ell/genl.c:954 #7 process_unicast (nlmsg=0x7fff61abeac0, genl=0x5555f2b3fc80) at ell/genl.c:973 #8 received_data (io=<optimized out>, user_data=0x5555f2b3fc80) at ell/genl.c:1098 #9 0x00007f60084c61bd in io_callback (fd=<optimized out>, events=1, user_data=0x5555f2b3fd20) at ell/io.c:120
#10 0x00007f60084c536d in l_main_iterate (timeout=<optimized out>) at ell/main.c:478
#11 0x00007f60084c543e in l_main_run () at ell/main.c:525
#12 l_main_run () at ell/main.c:507
#13 0x00007f60084c5670 in l_main_run_with_signal (callback=callback(a)entry=0x5555f0b89150 <signal_handler>, user_data=user_data(a)entry=0x0) at ell/main.c:647 #14 0x00005555f0b886a4 in main (argc=<optimized out>, argv=<optimized out>) at src/main.c:532
This happens when the driver does not support rekeying, which causes iwd to attempt a disconnect and re-connect. The disconnect action is taken during the event callback and destroys the underlying eapol state machine. Since a temporary isn't used, attempting to dereference sm->handshake results in a crash.
Fix this by introducing a UNIQUE_ID macro which should prevent shadowing and using a temporary variable as originally intended.
--- src/handshake.h | 38 +++++++++++++++++++++----------------- src/util.h | 4 ++++ 2 files changed, 25 insertions(+), 17 deletions(-)
src/handshake.h +21 -17
@@ -164,23 +164,27 @@ struct handshake_state {handshake_event_func_t event_func; }; -#define handshake_event (_hs, event, ...) \ - (__extension__ ({ \ - bool freed = false; \ - \ - if ((_hs)->event_func && !(_hs)->in_event) { \ - (_hs)->in_event = true; \ - (_hs)->event_func((_hs), event, (_hs)->user_data, \ - ##__VA_ARGS__); \ - \ - if (!(_hs)->in_event) { \ - handshake_state_free((_hs)); \ - freed = true; \ - } else \ - (_hs)->in_event = false; \ - } \ - freed; \ - })) +#define HSID (x) UNIQUE_ID(handshake_, x) + +#define handshake_event (_hs, event, ...) \ + ({ \ + bool HSID(freed) = false; \ + typeof(_hs) HSID(hs) = (_hs); \ + \ + if (HSID(hs)->event_func && !HSID(hs)->in_event) { \ + HSID(hs)->in_event = true; \ + HSID(hs)->event_func(HSID(hs), (event), \ + HSID(hs)->user_data, \ + ##__VA_ARGS__); \ + \ + if (!HSID(hs)->in_event) { \ + handshake_state_free(HSID(hs)); \ + HSID(freed) = true; \ + } else \ + HSID(hs)->in_event = false; \ + } \ + HSID(freed); \ + })void handshake_state_free (struct handshake_state *s);
src/util.h +4 -0
@@ -26,6 +26,10 @@#include <stdint.h> #include <unistd.h> +#define ___PASTE (a, b) a ## b +#define __PASTE (a, b) ___PASTE(a, b) +#define UNIQUE_ID (x, id) __PASTE(__unique_prefix_, __PASTE(x, id)) +#define align_len (len, boundary) (((len)+(boundary)-1) & ~((boundary)-1))#define MAC "%02x:%02x:%02x:%02x:%02x:%02x"
Signature · 1 line
2.32.0
Toke Høiland-Jørgensen <toke at toke.dk>
Tested-by Toke Høiland-Jørgensen on v1 1/1Re: [PATCH] handshake: Do not crash if handshake is destroyed
Quoted 35 lines · Denis Kenzior
Commit
4d2176df2985("handshake: Allow event handler to free handshake") introduced a re-entrancy guard so thathandshake_stateobjects that are destroyed as a result of the event do not cause a crash. It rightly used a temporary object to store the passed in handshake. Unfortunately this caused variable shadowing which resulted in crashes fixed by commitd22b174a7318("handshake: use_hsdirectly inhandshake_event"). However, since the temporary was no longer used, this fix itself caused a crash:#0
0x00005555f0ba8b3dineapol_handle_ptk_1_of_4(sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) atsrc/eapol.c:1236
1236 handshake_event(sm->handshake,
(gdb) bt#0 0x00005555f0ba8b3d in eapol_handle_ptk_1_of_4 (sm=sm(a)entry=0x5555f2b4a920, ek=0x5555f2b62588, ek(a)entry=0x16, unencrypted=unencrypted(a)entry=false) at src/eapol.c:1236 #1 0x00005555f0bab118 in eapol_key_handle (unencrypted=<optimized out>, frame=<optimized out>, sm=0x5555f2b4a920) at src/eapol.c:2343 #2 eapol_rx_packet (proto=<optimized out>, from=<optimized out>, frame=<optimized out>, unencrypted=<optimized out>, user_data=0x5555f2b4a920) at src/eapol.c:2665 #3 0x00005555f0bac497 in __eapol_rx_packet (ifindex=62, src=src(a)entry=0x5555f2b62574 "x\212 J\207\267", proto=proto(a)entry=34958, frame=frame(a)entry=0x5555f2b62588 "\002\003", len=len(a)entry=121, noencrypt=noencrypt(a)entry=false) at src/eapol.c:3017 #4 0x00005555f0b8c617 in netdev_control_port_frame_event (netdev=0x5555f2b64450, msg=0x5555f2b62588) at src/netdev.c:5574 #5 netdev_unicast_notify (msg=msg(a)entry=0x5555f2b619a0, user_data=<optimized out>) at src/netdev.c:5613 #6 0x00007f60084c9a51 in dispatch_unicast_watches (msg=0x5555f2b619a0, id=<optimized out>, genl=0x5555f2b3fc80) at ell/genl.c:954 #7 process_unicast (nlmsg=0x7fff61abeac0, genl=0x5555f2b3fc80) at ell/genl.c:973 #8 received_data (io=<optimized out>, user_data=0x5555f2b3fc80) at ell/genl.c:1098 #9 0x00007f60084c61bd in io_callback (fd=<optimized out>, events=1, user_data=0x5555f2b3fd20) at ell/io.c:120#10
0x00007f60084c536dinl_main_iterate(timeout=<optimized out>) atell/main.c:478
#110x00007f60084c543einl_main_run() atell/main.c:525
#12l_main_run() atell/main.c:507
#130x00007f60084c5670inl_main_run_with_signal(callback=callback(a)entry=0x5555f0b89150 <signal_handler>, user_data=user_data(a)entry=0x0) atell/main.c:647#140x00005555f0b886a4in main (argc=<optimized out>, argv=<optimized out>) atsrc/main.c:532This happens when the driver does not support rekeying, which causes iwd to attempt a disconnect and re-connect. The disconnect action is taken during the event callback and destroys the underlying eapol state machine. Since a temporary isn't used, attempting to dereference
sm->handshakeresults in a crash.
Quoted 6 lines
Fix this by introducing a
UNIQUE_IDmacro which should prevent shadowing and using a temporary variable as originally intended.Fixes d22b174a7318 ("handshake: use _hs directly in handshake_event")Fixes 4d2176df2985 ("handshake: Allow event handler to free handshake")Reported-By Toke Høiland-Jørgensen <toke(a)toke.dk>
With this patch, the crashes are gone - thanks!
Denis Kenzior
Hi Toke,
Quoted 6 lines · Toke Høiland-Jørgensen
Hi
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes. I rebuilt iwd with debug symbols enabled and got the following backtrace out of 'coredumpctl gdb':
*hangs his head in shame*. So I think we screwed up the same macro 2x now.
Can you please try the patch I just sent out?
Regards,
-Denis
Toke Høiland-Jørgensen <toke at toke.dk>
Quoted 11 lines · Denis Kenzior
Hi Toke,
Quoted 6 lines · Toke Høiland-Jørgensen
Hi
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes. I rebuilt iwd with debug symbols enabled and got the following backtrace out of 'coredumpctl gdb':
*hangs his head in shame*. So I think we screwed up the same macro 2x now.
Heh, yeah, did see there was a commit touching that macro, but didn't look closer.
Quoted 1 line
Can you please try the patch I just sent out?
Can do - thanks! Did a build with it incorporated and am running that now. Usually crashes once every hour (with striking regularity); will leave it running overnight and see if the crashes are gone by tomorrow :)
-Toke
Toke Høiland-Jørgensen <toke at toke.dk>
Quoted 17 lines · Toke Høiland-Jørgensen
Quoted 11 lines · Denis Kenzior
Hi Toke,
Quoted 6 lines · Toke Høiland-Jørgensen
Hi
Since updating to iwd 1.23 (from Arch packages), I've started seeing periodic crashes. I rebuilt iwd with debug symbols enabled and got the following backtrace out of 'coredumpctl gdb':
*hangs his head in shame*. So I think we screwed up the same macro 2x now.
Heh, yeah, did see there was a commit touching that macro, but didn't look closer.
Quoted 6 lines
Quoted 1 line
Can you please try the patch I just sent out?
Can do - thanks! Did a build with it incorporated and am running that now. Usually crashes once every hour (with striking regularity); will leave it running overnight and see if the crashes are gone by tomorrow :)
Alright, so no crashes - thanks for the swift fix!
And just for future reference, in case anyone else runs into the same
issue: The crashes were initially taking the kernel down with them, which has now also been fixed:
https://lore.kernel.org/r/20220201130951.22093-1-johannes(a)sipsolutions.net
-Toke