Skip to contentSkip to conversation

rtnetlink: Honour NLM_F_ECHO flag in rtnl_{new, set, del}link

PATCH Superseded Hangbin Liu 3 participants 10 messages Sep 27, 2022 – Sep 29, 2022

Subsystems (2)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 rtnetlink: Honour NLM_F_ECHO flag in rtnl_{new, set, del}link

Netlink messages are used for communicating between user and kernel space. When user space configures the kernel with netlink messages, it can set the NLM_F_ECHO flag to request the kernel to send the applied configuration back to the caller. This allows user space to retrieve configuration information that are filled by the kernel (either because these parameters can only be set by the kernel or because user space let the kernel choose a default value).

The kernel has support this feature in some places like RTM_{NEW, DEL}ADDR, RTM_{NEW, DEL}ROUTE. This patch handles NLM_F_ECHO flag and send link info back after rtnl_{new, set, del}link.

---
v3:
1) Fix group parameter in rtnl_notify.
2) Use helper rtmsg_ifinfo_build_skb() instead re-write a new one.

v2:
1) Rename rtnl_echo_link_info() to rtnl_link_notify().
2) Remove IFLA_LINK_NETNSID and IFLA_EXT_MASK, which do not fit here.
3) Add NLM_F_ECHO in rtnl_dellink. But we can't re-use the rtnl_link_notify() helper as we need to get the link info before rtnl_delete_link().

---
 include/linux/rtnetlink.h |  2 +-
 net/core/dev.c            |  2 +-
 net/core/rtnetlink.c      | 47 ++++++++++++++++++++++++++++++---------
 3 files changed, 38 insertions(+), 13 deletions(-)
include/linux/rtnetlink.h +1 -1
index ae2c6a3cec5d..3534701cdcc5 100644
@@ -24,7 +24,7 @@ void rtmsg_ifinfo_newnet(int type, struct net_device *dev, unsigned int change,
 struct sk_buff *rtmsg_ifinfo_build_skb(int type, struct net_device *dev,
 				       unsigned change, u32 event,
 				       gfp_t flags, int *new_nsid,
-				       int new_ifindex);
+				       int new_ifindex, u32 pid, u32 seq);
 void rtmsg_ifinfo_send(struct sk_buff *skb, struct net_device *dev,
 		       gfp_t flags);
 
net/core/dev.c +1 -1
index d66c73c1c734..fb2603bd07a9 100644
@@ -10862,7 +10862,7 @@ void unregister_netdevice_many(struct list_head *head)
 		if (!dev->rtnl_link_ops ||
 		    dev->rtnl_link_state == RTNL_LINK_INITIALIZED)
 			skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, dev, ~0U, 0,
-						     GFP_KERNEL, NULL, 0);
+						     GFP_KERNEL, NULL, 0, 0, 0);
 
 		/*
 		 *	Flush the unicast and multicast chains
net/core/rtnetlink.c +36 -11

151 lines not on this page · raw message

Signature · 1 line
2.37.2
maintainer
#1 lore raw

net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412

Quoted 13 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6 -0
@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,
 		if (err)
 			goto out_unregister;
 	}
+
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);
+
 out:
 	if (link_net)
 		put_net(link_net);

I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.

#2 lore raw
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6 -0
@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,
 		if (err)
 			goto out_unregister;
 	}
+
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);
+
Quoted 8 lines

net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412

Quoted 3 lines
net/core/rtnetlink.c +0 -0
 out:
 	if (link_net)
 		put_net(link_net);

I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.

I think __dev_notify_flags() only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.

Thanks
Hangbin

#3 lore raw
Quoted 21 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6 -0
@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,
 		if (err)
 			goto out_unregister;
 	}
+
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);
+
Quoted 8 lines

net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412

Quoted 3 lines
net/core/rtnetlink.c +0 -0
 out:
 	if (link_net)
 		put_net(link_net);

I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.

Quoted 3 lines

I think __dev_notify_flags() only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.

In rntl_configure_link(), dev->rtnl_link_state is RTNL_LINK_INITIALIZING on device cretation, so __dev_notify_flags() is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to make rtnl_notify() send the unicast message together with the multicast ones.

Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.

Quoted 3 lines

Thanks
Hangbin

#4 lore raw
Quoted 26 lines · Guillaume Nault
Quoted 21 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6 -0
@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,
 		if (err)
 			goto out_unregister;
 	}
+
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);
+
Quoted 8 lines

net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412

Quoted 3 lines
net/core/rtnetlink.c +0 -0
 out:
 	if (link_net)
 		put_net(link_net);

I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.

Quoted 3 lines

I think __dev_notify_flags() only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.

Quoted 5 lines

In rntl_configure_link(), dev->rtnl_link_state is RTNL_LINK_INITIALIZING on device cretation, so __dev_notify_flags() is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to make rtnl_notify() send the unicast message together with the multicast ones.

To update __dev_notify_flags() with nlmsghdr, we also need to update rtnl_configure_link(), which is called by some drivers.

Quoted 4 lines

Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.

And the __dev_notify_flags() is only used when the dev flag changed.

It looks no much change if we call it when create new link:
rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags()

But when set link, it is only called when flag changed do_setlink() -> dev_change_flags() -> __dev_notify_flags().

Unless you want to omit the ECHO message when setting link.

At latest, when call rtnl_delete_link(), there is no way to call __dev_notify_flags(). So we still need to use the current way.

As a summarize, we need to change a lot of code if we use __dev_notify_flags() to notify user, while we can only use it in one place. This looks not worth.

WDYT?

Thanks
Hangbin

maintainer
#5 lore raw
Quoted 14 lines · On Thu, 29 Sep 2022 11:10:36 +0800 Hangbin Liu wrote:
Quoted 3 lines

Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.

And the __dev_notify_flags() is only used when the dev flag changed.

It looks no much change if we call it when create new link:
rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags()

But when set link, it is only called when flag changed do_setlink() -> dev_change_flags() -> __dev_notify_flags().

Unless you want to omit the ECHO message when setting link.

Quoted 7 lines

At latest, when call rtnl_delete_link(), there is no way to call __dev_notify_flags(). So we still need to use the current way.

As a summarize, we need to change a lot of code if we use __dev_notify_flags() to notify user, while we can only use it in one place. This looks not worth.

WDYT?

There needs to be a clear use case if you want to add notifications. Plumbing ECHO to existing notifications is just good hygiene, if you want to add new notifications you'd need to provide a real use case.

I don't buy the "a lot code changed" BTW, you can make dev_change_flags() a wrapper and add dev_change_flags_nlh() or whatnot which will take the extra argument.

#6 lore raw
In reply to Hangbin Liu
Quoted 27 lines · Hangbin Liu
Quoted 26 lines · Guillaume Nault
Quoted 24 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6 -0
@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,
 		if (err)
 			goto out_unregister;
 	}
+
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);
+
Quoted 8 lines

net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412

Quoted 3 lines
net/core/rtnetlink.c +0 -0
 out:
 	if (link_net)
 		put_net(link_net);

I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.

I think __dev_notify_flags() only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.

Quoted 8 lines
Quoted 5 lines

In rntl_configure_link(), dev->rtnl_link_state is RTNL_LINK_INITIALIZING on device cretation, so __dev_notify_flags() is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to make rtnl_notify() send the unicast message together with the multicast ones.

To update __dev_notify_flags() with nlmsghdr, we also need to update rtnl_configure_link(), which is called by some drivers.

There's just a handful of virtual net devices that call rtnl_configure_link(). It should be easy to modify its prototype and adjust these external callers.

Quoted 6 lines
Quoted 3 lines

Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.

And the __dev_notify_flags() is only used when the dev flag changed.

Quoted 7 lines

It looks no much change if we call it when create new link:
rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags()

But when set link, it is only called when flag changed do_setlink() -> dev_change_flags() -> __dev_notify_flags().

Unless you want to omit the ECHO message when setting link.

For SET operations, we may send one, many or even zero notifications. I agree with Jackub that we shouldn't add specific notifications just for NLM_F_ECHO. That means, either we echo all the existing notifications (or none if the device hasn't been modified), or we leave NLM_F_ECHO unimplemented for SET operations.

As I said in my previous reply, I can't see any use case for ECHO on SET operations, so I don't mind if we skip it.

Quoted 2 lines

At latest, when call rtnl_delete_link(), there is no way to call __dev_notify_flags(). So we still need to use the current way.

Not everything has to be done with __dev_notify_flags(). For DEL operations notifications are sent in netdev_unregister_many(). Given the overwhelming number of callers, modifying its prototype isn't going to be practical, but you can use a wrapper.

Something like (very rough idea):

-void unregister_netdevice_many(struct list_head *head)
+void unregister_netdevice_many_notify(struct list_head *head, u32 portid,
+ const struct nlmsghdr *nlh)
{
[...]

                 if (!dev->rtnl_link_ops ||
                     dev->rtnl_link_state == RTNL_LINK_INITIALIZED)

- skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, dev, ~0U, 0,
- GFP_KERNEL, NULL, 0);
+ skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, portid,
+ nlh->nlmsg_seq, dev, ~0U,
+ 0, GFP_KERNEL, NULL, 0);

[...]
if (skb)
- rtmsg_ifinfo_send(skb, dev, GFP_KERNEL);
+ rtmsg_ifinfo_send(skb, portid, dev, nlh, GFP_KERNEL);

[...]
}

+void unregister_netdevice_many(struct list_head *head) +{
+ unregister_netdevice_many_notify(head, 0, NULL);
+}

Quoted 4 lines

As a summarize, we need to change a lot of code if we use __dev_notify_flags() to notify user, while we can only use it in one place. This looks not worth.

WDYT?

Using __dev_notify_flags() is only for NEW operations. SET and DEL have to be handled differently.

To summarise, the objective is to reuse the existing notifications. In practice that means just doing the required plumbing to pass the correct parameters to the existing nlmsg_notify() calls (and ensure we set the correct portid and sequence number in the netlink message header).

Quoted 3 lines

Thanks
Hangbin

#7 lore raw
In reply to Hangbin Liu

Hi Jakub,

net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018

Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4 -0
@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);

BTW, in do_setlink() I planed to use RTM_SETLINK. But I found iproute2 use RTM_NEWLINK to set links. And I saw an old doc[1] said

"""
- RTM_SETLINK does not follow the usual rtnetlink conventions and ignores all netlink flags

The RTM_NEWLINK message type is a superset of RTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """

So I just use RTM_NEWLINK for the notification. Do you think if we should use RTM_SETLINK?

[1] https://lwn.net/Articles/236919/

Thanks
Hangbin

#8 lore raw
Quoted 18 lines · Hangbin Liu

Hi Jakub,

net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018

Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4 -0
@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);

BTW, in do_setlink() I planed to use RTM_SETLINK. But I found iproute2 use RTM_NEWLINK to set links. And I saw an old doc[1] said

"""
- RTM_SETLINK does not follow the usual rtnetlink conventions and ignores all netlink flags

Quoted 6 lines

The RTM_NEWLINK message type is a superset of RTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """

So I just use RTM_NEWLINK for the notification. Do you think if we should use RTM_SETLINK?

This problem should go away once you switch to reusing the existing notifications (instead of adding new ones).

Quoted 5 lines

[1] https://lwn.net/Articles/236919/

Thanks
Hangbin

maintainer
#9 lore raw
In reply to Hangbin Liu
Quoted 14 lines · On Wed, 28 Sep 2022 11:22:11 +0800 Hangbin Liu wrote:

Hi Jakub,

net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018

Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4 -0
@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb,
 		}
 	}
 
+	nskb = rtmsg_ifinfo_build_skb(RTM_NEWLINK, dev, 0, 0, GFP_KERNEL, NULL,
+				      0, pid, nlh->nlmsg_seq);
+	if (nskb)
+		rtnl_notify(nskb, dev_net(dev), pid, RTNLGRP_LINK, nlh, GFP_KERNEL);  

BTW, in do_setlink() I planed to use RTM_SETLINK. But I found iproute2 use RTM_NEWLINK to set links. And I saw an old doc[1] said

Quoted 7 lines

"""
- RTM_SETLINK does not follow the usual rtnetlink conventions and ignores all netlink flags

The RTM_NEWLINK message type is a superset of RTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """

Interesting, so we actually do use this "NEW as SET" thing.

Quoted 2 lines

So I just use RTM_NEWLINK for the notification. Do you think if we should use RTM_SETLINK?

FWIW I think it's typical for rtnl / classic netlink to generate "new object" notification whenever object is changed, rather than a notification about just the change.