Weird DHCP related problems with net-next

Hi,

I am observing a strange problem on net-next (not observed with net, bisection in progress) where the initial DHCP configuration using busybox's udhcpc is able to configure the local interface address and DNS serer, but not the default gateway. Restarting udhcpc a second time does not exhibit this problem.

Quoted 7 lines · Florian Fainelli

Hi,

I am observing a strange problem on net-next (not observed with net, bisection in progress) where the initial DHCP configuration using busybox's udhcpc is able to configure the local interface address and DNS serer, but not the default gateway. Restarting udhcpc a second time does not exhibit this problem.

Hi Florian

I've seen something similar, but different, again with DSA involved, on a WiFi Access point. I have debian, and i'm using isc dhcp. It gets an address, sets the address on the interface, but does not add the interface route to the routing table. Not sure about default route, i would have to go check that.

Hi Andrew,

Quoted 11 lines · Andrew Lunn

Quoted 7 lines · Florian Fainelli

Hi,

I am observing a strange problem on net-next (not observed with net, bisection in progress) where the initial DHCP configuration using busybox's udhcpc is able to configure the local interface address and DNS serer, but not the default gateway. Restarting udhcpc a second time does not exhibit this problem.

Hi Florian

Quoted 5 lines

I've seen something similar, but different, again with DSA involved, on a WiFi Access point. I have debian, and i'm using isc dhcp. It gets an address, sets the address on the interface, but does not add the interface route to the routing table. Not sure about default route, i would have to go check that.

Interesting, did you also observe this with 'net', or just with 'net-next'?

Contrary to what I reported above, this is only an issue with SYSTEMPORT/DSA/SF2, I could not reproduce this GENET or the Asix driver,

Note that I am observing this on ARM (Andrew probably is as well),

Hi Andrew,

Quoted 9 lines · Andrew Lunn

Quoted 7 lines · Florian Fainelli

Hi,

I am observing a strange problem on net-next (not observed with net, bisection in progress) where the initial DHCP configuration using busybox's udhcpc is able to configure the local interface address and DNS serer, but not the default gateway. Restarting udhcpc a second time does not exhibit this problem.

Quoted 7 lines

Hi Florian

I've seen something similar, but different, again with DSA involved, on a WiFi Access point. I have debian, and i'm using isc dhcp. It gets an address, sets the address on the interface, but does not add the interface route to the routing table. Not sure about default route, i would have to go check that.

Interesting, did you also observe this with 'net', or just with 'net-next'?

Contrary to what I reported above, this is only an issue with SYSTEMPORT/DSA/SF2, I could not reproduce this GENET or the Asix driver, I was just conflating two different systems here.

My bisection seems to point at this commit:

58c2cb16b116d7feace621bd6b647bbabacfa225 ("switchdev: convert fib_ipv4_add/del over to switchdev_port_obj_add/del")

And indeed, hacking a bit the kernel to remove the SWITCHDEV/DSA dependencies to leave just DSA makes thing work again.

Scott, Jiri, any clues? I can instrument the kernel a bit more to help find what is the problem here. Note that I am observing this on ARM (Andrew probably is as well), where uninitialized stack variables are potentially garbage.

I see the problem now, DSA does not implement a port_obj_add callback, so when net/ipv4/fib_trie.c::switchdev_fib_ipv4_add() gets to call switchdev_port_obj_add, we return -EOPNOTSUPP, and take the error path in fib_table_insert thus not inserting the route for this interface.

Now when I restart the DHCP client, we end-up inserting the default route which is correct, still figuring out what is different here, probably the deletion of the routes by the DHCP client script first is the different condition.

At any rate, since switchdev_fib_ipv4_add() returns something that make us take an error path in the fib_trie, something like this seems to fix it for me but I am not well versed enough into the IPv4 routing code to be 100% confident this is the right fix. Also, there are other callers of switchdev_port_obj_add() but a quick look seems to make them safe as they are only called for "offloading" capable hardware.

Quoted 4 lines · Florian Fainelli

I see the problem now, DSA does not implement a port_obj_add callback, so when net/ipv4/fib_trie.c::switchdev_fib_ipv4_add() gets to call switchdev_port_obj_add, we return -EOPNOTSUPP, and take the error path in fib_table_insert thus not inserting the route for this interface.

Yup, that's the problem.

Quoted 4 lines

Now when I restart the DHCP client, we end-up inserting the default route which is correct, still figuring out what is different here, probably the deletion of the routes by the DHCP client script first is the different condition.

After the first failure, ipv4.fib_offload_disabled is set, so the next time switchdev_fib_ipv4_add() just returns 0 and the route is installed. That explains the one-off behavior.

Quoted 6 lines

At any rate, since switchdev_fib_ipv4_add() returns something that make us take an error path in the fib_trie, something like this seems to fix it for me but I am not well versed enough into the IPv4 routing code to be 100% confident this is the right fix. Also, there are other callers of switchdev_port_obj_add() but a quick look seems to make them safe as they are only called for "offloading" capable hardware.

Your fix looks good to me. The other users of switchdev_port_obj_add() want to return -EOPNOTSUPP to user, so it's just this one case for IPv4 fib insert/del where we'll want to treat no support silently. Are you going to resend as patch for net-next, or should I?