iwd 1.21 stuck on connecting state
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Hi,
I've recently started noticing on my laptop that iwd gets stuck on 'connecting' state when I boot a **second time** right after poweroff. The fix is quite easy. Just disconnect and connect again from iwctl.
I recall having a similar issue not long ago. Maybe a regression was introduced. For the record, router and connection settings are the same as before.
Arch Linux with kernel 5.16-zen1.
Thanks!
James Prestwood
Hi,
Quoted 5 lines · On Thu, 2022-01-13 at 11:27 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Hi,
I've recently started noticing on my laptop that iwd gets stuck on 'connecting' state when I boot a **second time** right after poweroff. The fix is quite easy. Just disconnect and connect again from iwctl.
If its repeatable could you get an IWD log with debugging enabled, as well as an iwmon trace?
https://iwd.wiki.kernel.org/debugging
Quoted 11 lines
I recall having a similar issue not long ago. Maybe a regression was introduced. For the record, router and connection settings are the same as before.
Arch Linux with kernel 5.16-zen1.
Thanks!
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Hi James,
I've gone through debugging and this is what I have: https://termbin.com/y9vq
iwd.service
# /usr/lib/systemd/system/iwd.service [Unit] Description=Wireless service After=network-pre.target Before=network.target Wants=network.target
[Service]
Type=dbus
BusName=net.connman.iwd
ExecStart=/usr/lib/iwd/iwd
NotifyAccess=main
LimitNPROC=1
Restart=on-failure
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_NET_BIND_SERVICE PrivateTmp=true
NoNewPrivileges=true
DevicePolicy=closed
DeviceAllow=/dev/rfkill rw
ProtectHome=yes
ProtectSystem=strict
ProtectControlGroups=yes
ProtectKernelModules=yes
ConfigurationDirectory=iwd
StateDirectory=iwd
StateDirectoryMode=0700
[Install]
WantedBy=multi-user.target
# /etc/systemd/system/iwd.service.d/override.conf [Service] RuntimeDirectory=resolvconf ReadWritePaths=/etc/resolv.conf
/etc/iwd/main.conf
[General]
EnableNetworkConfiguration=true
AddressRandomization=once
[Network]
NameResolvingService=resolvconf
I've also tried to collect iwmon logs but they are huge, like almost 2 MB. Should I filter some of the info as the wiki shows (--nortnl --nowiphy --noscan).
Thanks!
James Prestwood
Hi,
Since you have EnableNetworkConfiguration=true IWD is performing DHCP and waiting for that to succeed. I see further you tried again and got an IP so for some reason your DHCP server was not acting properly in the 'stuck in connecting case'.
You could try setting IWD_DHCP_DEBUG=1 as an environment variable and try again. That may give more insight into what DHCP is waiting for.
Thanks,
James
Quoted 59 lines · On Fri, 2022-01-14 at 20:08 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Hi James,
I've gone through debugging and this is what I have:
https://termbin.com/y9vqiwd.service
# /usr/lib/systemd/system/iwd.service [Unit] Description=Wireless service After=network-pre.target Before=network.target Wants=network.target[Service]
Type=dbus
BusName=net.connman.iwd
ExecStart=/usr/lib/iwd/iwd
NotifyAccess=main
LimitNPROC=1
Restart=on-failure
CapabilityBoundingSet=CAP_NET_ADMINCAP_NET_RAWCAP_NET_BIND_SERVICEPrivateTmp=true
NoNewPrivileges=true
DevicePolicy=closed
DeviceAllow=/dev/rfkill rw
ProtectHome=yes
ProtectSystem=strict
ProtectControlGroups=yes
ProtectKernelModules=yes
ConfigurationDirectory=iwd
StateDirectory=iwd
StateDirectoryMode=0700[Install]
WantedBy=multi-user.target# /etc/systemd/system/iwd.service.d/override.conf [Service] RuntimeDirectory=resolvconf ReadWritePaths=/etc/resolv.conf
/etc/iwd/main.conf[General]
EnableNetworkConfiguration=true
AddressRandomization=once[Network]
NameResolvingService=resolvconfI've also tried to collect iwmon logs but they are huge, like almost 2 MB. Should I filter some of the info as the wiki shows (--nortnl -- nowiphy --noscan).
Thanks!
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
These are the logs with IWD_DHCP_DEBUG=1.
On first attempt I didn't manage to get a connection due to DHCP, but after stopping/starting it worked.
James Prestwood
Hi,
It looks like the DHCP server sent some packet that we did not accept. There isn't enough debug printing to know what exactly so we'll need to get a network capture PCAP and see what data it sent. You can use tcpdump/wireshark to get this. Just ensure you see DHCP packets in the capture.
Something like:
tcpdump -i <iface> port 67 or port 68 -s 0 -w dhcp.pcap
Thanks,
James
Quoted 9 lines · On Tue, 2022-01-18 at 11:43 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
These are the logs with IWD_DHCP_DEBUG=1.
On first attempt I didn't manage to get a connection due to DHCP, but after stopping/starting it worked.
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Hi again,
I managed to trigger the problem while dumping tcp packets. Here is the result:
# ~ tcpdump -i wlan0 port 67 or port 68 -s 0 -w dhcp.pcap
tcpdump: listening on wlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes ^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel
# ~ tcpdump -r dhcp.pcap
reading from file dhcp.pcap, link-type EN10MB (Ethernet), snapshot length 262144
18:33:36.492009 IP _gateway.bootps > 192.168.1.35.bootpc: BOOTP/DHCP, Reply, length 284
18:33:36.492212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:33:38.606330 IP _gateway.bootps > 192.168.1.36.bootpc: BOOTP/DHCP, Reply, length 284
18:33:40.021633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:33:48.860117 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:05.360324 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:14.092945 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:14.705123 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:16.217430 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
18:34:16.217613 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:16.313729 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
18:34:16.320877 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
So I stopped / started iwd.service and checked that connection was lost (no ping). Then started dumping packets while iwctl was constantly reporting 'connecting' state. From there I just disconnected and connected again successfully and aborted tcpdump.
James Prestwood
Quoted 41 lines · On Thu, 2022-01-27 at 17:51 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Hi again,
I managed to trigger the problem while dumping tcp packets. Here is the result:
# ~ tcpdump -i wlan0 port 67 or port 68 -s 0 -w dhcp.pcap
tcpdump: listening on wlan0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C
12 packets captured
12 packets received by filter
0 packets dropped by kernel# ~ tcpdump -r dhcp.pcap
reading from file dhcp.pcap, link-type EN10MB (Ethernet), snapshot length 262144
18:33:36.492009 IP _gateway.bootps > 192.168.1.35.bootpc: BOOTP/DHCP, Reply, length 284
18:33:36.492212 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:33:38.606330 IP _gateway.bootps > 192.168.1.36.bootpc: BOOTP/DHCP, Reply, length 284
18:33:40.021633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:33:48.860117 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:05.360324 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:14.092945 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:14.705123 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:16.217430 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
18:34:16.217613 IP 0.0.0.0.bootpc > 255.255.255.255.bootps:
BOOTP/DHCP, Request from xx:xx:xx:xx:xx:xx (oui Unknown), length 300 18:34:16.313729 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
18:34:16.320877 IP _gateway.bootps > host.bootpc: BOOTP/DHCP, Reply, length 284
Quoted 4 lines
So I stopped / started iwd.service and checked that connection was lost (no ping). Then started dumping packets while iwctl was constantly reporting 'connecting' state. From there I just disconnected and connected again successfully and aborted tcpdump.
Could you also attach the actual pcap file for inspection? Something in the packet is not validating but we have no way of knowing without seeing the actual packet contents.
Thanks,
James
Quoted 3 lines
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Sorry, didn't know that. I reproduced the problem again and uploaded the file here: https://transfer.sh/get/jiEvOg/dhcp.pcap
Hope this helps!
James Prestwood
Quoted 4 lines · On Sat, 2022-01-29 at 20:03 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Sorry, didn't know that. I reproduced the problem again and uploaded the file here: https://transfer.sh/get/jiEvOg/dhcp.pcap
Hope this helps!
Thanks for getting this. For some reason your DHCP server is sending two offers, and IWD is ignoring the second:
- DHCP offers 192.168.1.33
- IWD responds with a request for .33
- 2 seconds go by and DHCP offers 192.168.1.35
- IWD ignores offer as its already waiting for a reply from its original request
- IWD continues to retransmit requests for .35 but the server ignores them.
It seems the server wants .35 as this is what works later when (I assume) you restarted IWD.
A quick fix could be that we automatically restart DHCP after some timeout (same effect as you restarting IWD). The proper fix would be to handle this multiple offer case.
Out of curiosity what kind of network hardware are you using? Are you just using an off the shelf router or some custom openwrt/hostapd setup?
Thanks,
James
Quoted 3 lines
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
Denis Kenzior
Hi James,
Quoted 8 lines
- DHCP offers 192.168.1.33
- IWD responds with a request for .33
- 2 seconds go by and DHCP offers 192.168.1.35
- IWD ignores offer as its already waiting for a reply from its original request
- IWD continues to retransmit requests for .35 but the server ignores them.
Quoted 6 lines
It seems the server wants .35 as this is what works later when (I assume) you restarted IWD.
A quick fix could be that we automatically restart DHCP after some timeout (same effect as you restarting IWD). The proper fix would be to handle this multiple offer case.
The second .35 offer seems to come from the same DHCP server as the first one. So we may be able to handle this case by assuming the current offer supersedes the last one we have from this server and restart the request.
Regards,
-Denis
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Quoted 3 lines
Out of curiosity what kind of network hardware are you using? Are you just using an off the shelf router or some custom openwrt/hostapd setup?
This is a router from my Internet provider. It's been with me for some years now but it's not the first time I experience a problem similar to this one with iwd.
Please take a look at this other issue from September 2021 also posted by me [1], just in case they both could share a root cause. Responses are not displayed on a single thread but they are easy to find on that month.
Thanks.
--
[1] https://lists.01.org/hyperkitty/list/iwd(a)lists.01.org/thread/K6FJL6B7UL3Z5OUPP4OEPFZPN6YNHNYC/
James Prestwood
Hi,
Quoted 4 lines · On Sat, 2022-01-29 at 20:03 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Sorry, didn't know that. I reproduced the problem again and uploaded the file here: https://transfer.sh/get/jiEvOg/dhcp.pcap
Hope this helps!
Below is a patch for ELL [1]. If its possible for you to apply it and rebuild both IWD/ELL manually to test that would be great. The issue should be solved but you will see an additional print when you normally would see IWD hang and never get connected:
"Server sent another offer, using it instead"
Note: IWD_DHCP_DEBUG has to be enabled to see this.
If you are able to get a log showing this behavior we can mark this as fixed, otherwise we can wait till this patch is in a release and see if it fixed the issue then. Thanks for your help so far.
[1]
https://lists.01.org/hyperkitty/list/ell(a)lists.01.org/thread/DJHXIOGGOCGGFUD2E3N2XVXCUAO2X2UP/
Quoted 3 lines
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Hi.
Quoted 4 lines
Below is a patch for ELL [1]. If its possible for you to apply it and rebuild both IWD/ELL manually to test that would be great. The issue should be solved but you will see an additional print when you normally would see IWD hang and never get connected:
I'm trying to apply the patch but I got several errors. I'm just copying and pasting it on a new file.
This is how I'm applying it: patch -d ell -p1 -i ../dhcp.patch
First it complained about being malformed:
patching file ell/dhcp.c
patch: **** malformed patch at line 72: void *userdata,
Just fixed it putting lines 71 and 72 together. Then all 3 hunks failed:
patching file ell/dhcp.c
Hunk #1 FAILED at 757.
Hunk #2 FAILED at 766.
Hunk #3 FAILED at 832.
3 out of 3 hunks FAILED -- saving rejects to file ell/dhcp.c.rej
I'm not used diff patching so probably I'm doing something wrong.
Quoted 2 lines
[1]
https://lists.01.org/hyperkitty/list/ell(a)lists.01.org/thread/DJHXIOGGOCGG...
James Prestwood
Hi,
This actually got merged an hour or two ago:
https://git.kernel.org/pub/scm/libs/ell/ell.git/commit/?id=b02c2034d8f05b4d731196de51a31cf1ef282e1f
Quoted 35 lines · On Thu, 2022-02-03 at 16:05 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Hi.
Quoted 5 lines
Below is a patch for ELL [1]. If its possible for you to apply it and rebuild both IWD/ELL manually to test that would be great. The issue should be solved but you will see an additional print when you normally
would see IWD hang and never get connected:I'm trying to apply the patch but I got several errors. I'm just copying and pasting it on a new file.
This is how I'm applying it: patch -d ell -p1 -i
../dhcp.patchFirst it complained about being malformed:
patching file
ell/dhcp.c
patch: **** malformed patch at line 72: void *userdata,Just fixed it putting lines 71 and 72 together. Then all 3 hunks failed:
patching file
ell/dhcp.c
Hunk #1 FAILED at 757.
Hunk #2 FAILED at 766.
Hunk #3 FAILED at 832.
3 out of 3 hunks FAILED -- saving rejects to file ell/dhcp.c.rejI'm not used diff patching so probably I'm doing something wrong.
Quoted 3 lines
[1]
https://lists.01.org/hyperkitty/list/ell(a)lists.01.org/thread/DJHXIOGGOCGG
..._______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org
54d22547-afc0-4701-b1fc-5a2a2e411646 at simplelogin.co
Quoted 1 line
This actually got merged an hour or two ago
I noticed right after posting. Classic hehe.
Issue seems solved. I get connectivity and according to logs iwd is discarding previous DHCP requests:
[DHCPv4] dhcp_client_send_discover:346
[DHCPv4] l_dhcp_client_start:1210 Entering state: DHCP_STATE_SELECTING [DHCPv4] dhcp_client_timeout_resend:526
[DHCPv4] dhcp_client_send_discover:346
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_offer:762
[DHCPv4] dhcp_client_handle_offer:805 Entering state: DHCP_STATE_REQUESTING [DHCPv4] dhcp_client_send_request:396
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_offer:762
[DHCPv4] dhcp_client_receive_offer:786 Server sent another offer, using it instead [DHCPv4] dhcp_client_handle_offer:805 Entering state: DHCP_STATE_REQUESTING [DHCPv4] dhcp_client_send_request:396
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_ack:684
[DHCPv4] dhcp_client_rx_message:905 Entering state: DHCP_STATE_BOUND [DHCPv4] dhcp_client_rx_message:939 T1 expiring in 21600784 ms
[DHCPv4] l_acd_start:447 Skipping probes and sending announcements
[DHCPv4] announce_wait_timeout:166 No conflicts found for 192.168.1.39, announcing address [DHCPv4] acd_send_packet:146 sending packet with target IP 192.168.1.39
Would you create a release soon?
Thanks!
James Prestwood
Quoted 7 lines · On Thu, 2022-02-03 at 20:10 +0000, 54d22547-afc0-4701-b1fc-5a2a2e411646(a)simpl…
Quoted 1 line
This actually got merged an hour or two ago
I noticed right after posting. Classic hehe.
Issue seems solved. I get connectivity and according to logs iwd is discarding previous DHCP requests:
Quoted 14 lines
[DHCPv4] dhcp_client_send_discover:346
[DHCPv4] l_dhcp_client_start:1210 Entering state:DHCP_STATE_SELECTING
[DHCPv4] dhcp_client_timeout_resend:526
[DHCPv4] dhcp_client_send_discover:346
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_offer:762
[DHCPv4] dhcp_client_handle_offer:805 Entering state:DHCP_STATE_REQUESTING
[DHCPv4] dhcp_client_send_request:396
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_offer:762
[DHCPv4] dhcp_client_receive_offer:786 Server sent another offer, using it instead
Perfect, glad to see it worked.
Quoted 13 lines
[DHCPv4] dhcp_client_handle_offer:805 Entering state:
DHCP_STATE_REQUESTING
[DHCPv4] dhcp_client_send_request:396
[DHCPv4] dhcp_client_rx_message:832
[DHCPv4] dhcp_client_receive_ack:684
[DHCPv4] dhcp_client_rx_message:905 Entering state:DHCP_STATE_BOUND[DHCPv4] dhcp_client_rx_message:939 T1 expiring in 21600784 ms [DHCPv4] l_acd_start:447 Skipping probes and sending announcements [DHCPv4] announce_wait_timeout:166 No conflicts found for 192.168.1.39, announcing address
[DHCPv4] acd_send_packet:146 sending packet with target IP 192.168.1.39
Quoted 1 line
Would you create a release soon?
Yes, I think this is happening very soon.
Quoted 5 lines
Thanks!
_______________________________________________
iwd mailing list -- iwd(a)lists.01.org
To unsubscribe send an email to iwd-leave(a)lists.01.org