Skip to contentSkip to conversation

neighbour: add RTNL_FLAG_DUMP_SPLIT_NLM_DONE to RTM_GETNEIGH

PATCH In review Maciej Żenczykowski 4 participants 5 messages Jun 15, 2024 – Mar 20

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 neighbour: add RTNL_FLAG_DUMP_SPLIT_NLM_DONE to RTM_GETNEIGH

without this Android's net test, available at:
https://cs.android.com/android/platform/superproject/main/+/main:kernel/tests/net/test/
run via:
/...aosp-tests.../net/test/run_net_test.sh --builder neighbour_test.py
fails with:
TypeError: NLMsgHdr requires a bytes object of length 16, got 4

Fixes 7e4975f7e7fb ("neighbour: fix neigh_dump_info() return value")
Cc Jakub Kicinski <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc David S. Miller <[email protected]>
---
 net/core/neighbour.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/core/neighbour.c +1 -1
index 45fd88405b6b..e1d12c6ac5b6 100644
@@ -3892,7 +3892,7 @@ static int __init neigh_init(void)
 	rtnl_register(PF_UNSPEC, RTM_NEWNEIGH, neigh_add, NULL, 0);
 	rtnl_register(PF_UNSPEC, RTM_DELNEIGH, neigh_delete, NULL, 0);
 	rtnl_register(PF_UNSPEC, RTM_GETNEIGH, neigh_get, neigh_dump_info,
-		      RTNL_FLAG_DUMP_UNLOCKED);
+		      RTNL_FLAG_DUMP_UNLOCKED | RTNL_FLAG_DUMP_SPLIT_NLM_DONE);
 
 	rtnl_register(PF_UNSPEC, RTM_GETNEIGHTBL, NULL, neightbl_dump_info,
 		      0);
Signature · 1 line
2.45.2.627.g7a2c4fd464-goog
#1 lore raw

For the other patch, I've tracked down:
32affa5578f0 ("fib: rules: no longer hold RTNL in fib_nl_dumprule()")
which causes half the regression.

But... I haven't figured out what causes the final half (or third depending on how you look at it).

I've also spent quite a while trying to figure out what exactly is going wrong in the python netlink parsing code.
The code leaves a *lot* to be desired...

Turns out it doesn't honour the nlmsghdr.length field of NLMSG_DONE messages, so it only reads the header (16 bytes) instead of the kernel generated 20=16+4 NULL bytes. I'm not sure why those extra 4 bytes are there, but they are... (anyone know?)
This results in a leftover 4 bytes, which then fail to parse as another nlmsghdr (because it also effectively ignores that it's a DONE and continues parsing).
Which explains the failure:
TypeError: NLMsgHdr requires a bytes object of length 16, got 4

Fixing the parsing, results in things hanging, because we ignore the DONE.

Fixing that... causes more issues (or I'm still confused about how the rest works, it's hard to follow, complicated by python's lack of types and some apparently dead code).

Ultimately I think the right answer is to simply fix the horribly broken netlink parser, which only ever worked by (more-or-less) chance. We have plenty of time (months) to fix it in time for the next release of Android after 15/V, which will be the first one to support a kernel newer than 6.6 LTS anyway.

Furthermore, the python netlink parser is only used in the test framework, while the non-test code itself uses C++& java netlink parsers (that I have not yet looked at) but is likely to either work or contain entirely different classes of bugs ;-)

maintainer
#2 lore raw
Quoted 6 lines · On Sun, 16 Jun 2024 10:09:07 +0200 Maciej Żenczykowski wrote:

For the other patch, I've tracked down:
32affa5578f0 ("fib: rules: no longer hold RTNL in fib_nl_dumprule()")
which causes half the regression.

But... I haven't figured out what causes the final half (or third depending on how you look at it).

To be completely honest I also have a fix queued for the other case, since it was reported already a month ago. But I "forgot" to send it. I had these tags on it:

Fixes 4ce5dc9316de ("inet: switch inet_dump_fib() to RCU protection")
Fixes 5fc68320c1fb ("ipv6: remove RTNL protection from inet6_dump_fib()")

LMK if you want to resend yours or I should send mine, because Ilya pinged the old thread this morning..

Quoted 8 lines

I've also spent quite a while trying to figure out what exactly is going wrong in the python netlink parsing code.
The code leaves a *lot* to be desired...

Turns out it doesn't honour the nlmsghdr.length field of NLMSG_DONE messages, so it only reads the header (16 bytes) instead of the kernel generated 20=16+4 NULL bytes. I'm not sure why those extra 4 bytes are there, but they are... (anyone know?)

They are the error code. Just like in a netlink ack. And similarly extack attrs may follow. Main difference with ack off the top of my head is that DONE never echos the request.

Quoted 18 lines

This results in a leftover 4 bytes, which then fail to parse as another nlmsghdr (because it also effectively ignores that it's a DONE and continues parsing).
Which explains the failure:
TypeError: NLMsgHdr requires a bytes object of length 16, got 4

Fixing the parsing, results in things hanging, because we ignore the DONE.

Fixing that... causes more issues (or I'm still confused about how the rest works, it's hard to follow, complicated by python's lack of types and some apparently dead code).

Ultimately I think the right answer is to simply fix the horribly broken netlink parser, which only ever worked by (more-or-less) chance. We have plenty of time (months) to fix it in time for the next release of Android after 15/V, which will be the first one to support a kernel newer than 6.6 LTS anyway.

Quoted 4 lines

Furthermore, the python netlink parser is only used in the test framework, while the non-test code itself uses C++& java netlink parsers (that I have not yet looked at) but is likely to either work or contain entirely different classes of bugs ;-)

We do have: tools/net/ynl/lib/ynl.py in the tree, FWIW. It's BSD-licensed, feel free to lift it / some of it. It's designed for the netlink YAML specs but the basics like message / attr parsing should work.

#3 lore raw

PATCH v3 1/1 neighbour: add RTNL_FLAG_DUMP_SPLIT_NLM_DONE to RTM_GETNEIGH

From: Maciej Żenczykowski <[email protected]>

without this Android's net test, available at:
https://cs.android.com/android/platform/superproject/main/+/main:kernel/tests/net/test/
run via:
/...aosp-tests.../net/test/run_net_test.sh --builder neighbour_test.py
fails with:
TypeError: NLMsgHdr requires a bytes object of length 16, got 4

Fixes 7e4975f7e7fb ("neighbour: fix neigh_dump_info() return value")
Cc Jakub Kicinski <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc David S. Miller <[email protected]>

---
It seems like this fix slipped through the cracks, so I'm resending it.

v3: Rebased on top of mainline and fixed the conflicts as v2 from Maciej
no longer applied cleanly. Verified the fix works as explained un the commit message. Added tested and sob tags.

 net/core/neighbour.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/core/neighbour.c +2 -1
index c56a4e7bf790..bdee4cf110bd 100644
@@ -3953,7 +3953,8 @@ static const struct rtnl_msg_handler neigh_rtnl_msg_handlers[] __initconst = {
 	{.msgtype = RTM_NEWNEIGH, .doit = neigh_add},
 	{.msgtype = RTM_DELNEIGH, .doit = neigh_delete},
 	{.msgtype = RTM_GETNEIGH, .doit = neigh_get, .dumpit = neigh_dump_info,
-	 .flags = RTNL_FLAG_DOIT_UNLOCKED | RTNL_FLAG_DUMP_UNLOCKED},
+	 .flags = RTNL_FLAG_DOIT_UNLOCKED | RTNL_FLAG_DUMP_UNLOCKED |
+		  RTNL_FLAG_DUMP_SPLIT_NLM_DONE},
 	{.msgtype = RTM_GETNEIGHTBL, .dumpit = neightbl_dump_info,
 	 .flags = RTNL_FLAG_DUMP_UNLOCKED},
 	{.msgtype = RTM_SETNEIGHTBL, .doit = neightbl_set,
Signature · 1 line
2.53.0.1018.g2bb0e51243-goog
#4 lore raw

This didn't really slip through; rather, the netlink parsing code in Android's net test is buggy and needs a rework...

Quoted 44 lines · Carlos Llamas

From: Maciej Żenczykowski <[email protected]>

without this Android's net test, available at:
https://cs.android.com/android/platform/superproject/main/+/main:kernel/tests/net/test/
run via:
/...aosp-tests.../net/test/run_net_test.sh --builder neighbour_test.py
fails with:
TypeError: NLMsgHdr requires a bytes object of length 16, got 4

Fixes 7e4975f7e7fb ("neighbour: fix neigh_dump_info() return value")
Cc Jakub Kicinski <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc David S. Miller <[email protected]>

---
It seems like this fix slipped through the cracks, so I'm resending it.

v3: Rebased on top of mainline and fixed the conflicts as v2 from Maciej
no longer applied cleanly. Verified the fix works as explained un the commit message. Added tested and sob tags.

 net/core/neighbour.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/core/neighbour.c +2 -2
index c56a4e7bf790..bdee4cf110bd 100644
@@ -3953,7 +3953,8 @@ static const struct rtnl_msg_handler neigh_rtnl_msg_handlers[] __initconst = {
        {.msgtype = RTM_NEWNEIGH, .doit = neigh_add},
        {.msgtype = RTM_DELNEIGH, .doit = neigh_delete},
        {.msgtype = RTM_GETNEIGH, .doit = neigh_get, .dumpit = neigh_dump_info,
-        .flags = RTNL_FLAG_DOIT_UNLOCKED | RTNL_FLAG_DUMP_UNLOCKED},
+        .flags = RTNL_FLAG_DOIT_UNLOCKED | RTNL_FLAG_DUMP_UNLOCKED |
+                 RTNL_FLAG_DUMP_SPLIT_NLM_DONE},
        {.msgtype = RTM_GETNEIGHTBL, .dumpit = neightbl_dump_info,
         .flags = RTNL_FLAG_DUMP_UNLOCKED},
        {.msgtype = RTM_SETNEIGHTBL, .doit = neightbl_set,
--

2.53.0.1018.g2bb0e51243-goog