rtnetlink: Honour NLM_F_ECHO flag in rtnl_{new, set, del}link
Subsystems (2)
PATCH v3 1/1 rtnetlink: Honour NLM_F_ECHO flag in rtnl_{new, set, del}link
Netlink messages are used for communicating between user and kernel space. When user space configures the kernel with netlink messages, it can set the NLM_F_ECHO flag to request the kernel to send the applied configuration back to the caller. This allows user space to retrieve configuration information that are filled by the kernel (either because these parameters can only be set by the kernel or because user space let the kernel choose a default value).
The kernel has support this feature in some places like RTM_{NEW, DEL}ADDR, RTM_{NEW, DEL}ROUTE. This patch handles NLM_F_ECHO flag and send link info back after rtnl_{new, set, del}link.
---
v3:
1) Fix group parameter in rtnl_notify.
2) Use helper rtmsg_ifinfo_build_skb() instead re-write a new one.
v2:
1) Rename rtnl_echo_link_info() to rtnl_link_notify().
2) Remove IFLA_LINK_NETNSID and IFLA_EXT_MASK, which do not fit here.
3) Add NLM_F_ECHO in rtnl_dellink. But we can't re-use the rtnl_link_notify() helper as we need to get the link info before rtnl_delete_link().
--- include/linux/rtnetlink.h | 2 +- net/core/dev.c | 2 +- net/core/rtnetlink.c | 47 ++++++++++++++++++++++++++++++--------- 3 files changed, 38 insertions(+), 13 deletions(-)
include/linux/rtnetlink.h +1 -1
@@ -24,7 +24,7 @@ void rtmsg_ifinfo_newnet(int type, struct net_device *dev, unsigned int change,struct sk_buff *rtmsg_ifinfo_build_skb (int type,struct net_device *dev,unsigned change,u32 event,gfp_t flags,int *new_nsid, -int new_ifindex); +int new_ifindex,u32 pid,u32 seq);void rtmsg_ifinfo_send (struct sk_buff *skb,struct net_device *dev,gfp_t flags);
net/core/dev.c +1 -1
@@ -10862,7 +10862,7 @@ void unregister_netdevice_many(struct list_head *head)if (!dev->rtnl_link_ops || dev->rtnl_link_state ==RTNL_LINK_INITIALIZED ) skb =rtmsg_ifinfo_build_skb (RTM_DELLINK , dev, ~0U ,0 , -GFP_KERNEL ,NULL ,0 ); +GFP_KERNEL ,NULL ,0 ,0 ,0 );/* * Flush the unicast and multicast chains
net/core/rtnetlink.c +36 -11
151 lines not on this page · raw message
Signature · 1 line
2.37.2
Jakub Kicinski
maintainernet/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412
Quoted 13 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6
-0@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,if (err) goto out_unregister; } + + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL ); + out:if (link_net)put_net (link_net);
I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags() sends a notification, already.
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6
-0@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,if (err) goto out_unregister; } + + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL ); +
Quoted 8 lines
net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412
Quoted 3 lines
net/core/rtnetlink.c +0
-0out:if (link_net)put_net (link_net);I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()sends a notification, already.
I think __dev_notify_flags() only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.
Thanks
Hangbin
Quoted 21 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6
-0@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,if (err) goto out_unregister; } + + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL ); +Quoted 8 lines
net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412
Quoted 3 lines
net/core/rtnetlink.c +0
-0out:if (link_net)put_net (link_net);I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()sends a notification, already.
Quoted 3 lines
I think
__dev_notify_flags()only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.
In rntl_configure_link(), dev->rtnl_link_state is RTNL_LINK_INITIALIZING on device cretation, so __dev_notify_flags() is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to make rtnl_notify() send the unicast message together with the multicast ones.
Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.
Quoted 3 lines
Thanks
Hangbin
Quoted 26 lines · Guillaume Nault
Quoted 21 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6
-0@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,if (err) goto out_unregister; } + + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL ); +Quoted 8 lines
net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412
Quoted 3 lines
net/core/rtnetlink.c +0
-0out:if (link_net)put_net (link_net);I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()sends a notification, already.Quoted 3 lines
I think
__dev_notify_flags()only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.
Quoted 5 lines
In
rntl_configure_link(),dev->rtnl_link_stateisRTNL_LINK_INITIALIZINGon device cretation, so__dev_notify_flags()is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to makertnl_notify()send the unicast message together with the multicast ones.
To update __dev_notify_flags() with nlmsghdr, we also need to update rtnl_configure_link(), which is called by some drivers.
Quoted 4 lines
Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.
And the __dev_notify_flags() is only used when the dev flag changed.
It looks no much change if we call it when create new link:rtnl_newlink_create() -> rtnl_configure_link() -> __dev_notify_flags()
But when set link, it is only called when flag changed do_setlink() -> dev_change_flags() -> __dev_notify_flags().
Unless you want to omit the ECHO message when setting link.
At latest, when call rtnl_delete_link(), there is no way to call __dev_notify_flags(). So we still need to use the current way.
As a summarize, we need to change a lot of code if we use __dev_notify_flags() to notify user, while we can only use it in one place. This looks not worth.
WDYT?
Thanks
Hangbin
Jakub Kicinski
maintainerQuoted 14 lines · On Thu, 29 Sep 2022 11:10:36 +0800 Hangbin Liu wrote:
Quoted 3 lines
Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.
And the
__dev_notify_flags()is only used when the dev flag changed.It looks no much change if we call it when create new link:
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()But when set link, it is only called when flag changed
do_setlink()->dev_change_flags()->__dev_notify_flags().Unless you want to omit the ECHO message when setting link.
Quoted 7 lines
At latest, when call
rtnl_delete_link(), there is no way to call__dev_notify_flags(). So we still need to use the current way.As a summarize, we need to change a lot of code if we use
__dev_notify_flags()to notify user, while we can only use it in one place. This looks not worth.WDYT?
There needs to be a clear use case if you want to add notifications. Plumbing ECHO to existing notifications is just good hygiene, if you want to add new notifications you'd need to provide a real use case.
I don't buy the "a lot code changed" BTW, you can make dev_change_flags() a wrapper and add dev_change_flags_nlh() or whatnot which will take the extra argument.
Quoted 27 lines · Hangbin Liu
Quoted 26 lines · Guillaume Nault
Quoted 24 lines · Hangbin Liu
Quoted 11 lines · Jakub Kicinski
Quoted 10 lines · On Tue, 27 Sep 2022 12:13:03 +0800 Hangbin Liu wrote:
net/core/rtnetlink.c +6
-0@@ -3382,6 +3401,12 @@ static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm,if (err) goto out_unregister; } + + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL ); +Quoted 8 lines
net/core/rtnetlink.c static int rtnl_newlink_create(struct sk_buff *skb, struct ifinfomsg *ifm, · line 3412
Quoted 3 lines
net/core/rtnetlink.c +0
-0out:if (link_net)put_net (link_net);I'm surprised you're adding new notifications. Does the kernel not already notify about new links? I thought
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()sends a notification, already.I think
__dev_notify_flags()only sends notification when dev flag changed. On the other hand, the notification is sent via multicast, while this patch is intend to unicast the notification to the user space.
Quoted 8 lines
Quoted 5 lines
In
rntl_configure_link(),dev->rtnl_link_stateisRTNL_LINK_INITIALIZINGon device cretation, so__dev_notify_flags()is called with gchanges=~0 and notification should be always sent. It's just a matter of passing the portid and the nlmsghdr down the call chain to makertnl_notify()send the unicast message together with the multicast ones.To update
__dev_notify_flags()with nlmsghdr, we also need to updatertnl_configure_link(), which is called by some drivers.
There's just a handful of virtual net devices that call rtnl_configure_link(). It should be easy to modify its prototype and adjust these external callers.
Quoted 6 lines
Quoted 3 lines
Now for device modification, I'm not sure there's a use case for unicast notifications. The caller already knows which values it asked to modify, so ECHO doesn't bring much value compared to a simple ACK.
And the
__dev_notify_flags()is only used when the dev flag changed.
Quoted 7 lines
It looks no much change if we call it when create new link:
rtnl_newlink_create()->rtnl_configure_link()->__dev_notify_flags()But when set link, it is only called when flag changed
do_setlink()->dev_change_flags()->__dev_notify_flags().Unless you want to omit the ECHO message when setting link.
For SET operations, we may send one, many or even zero notifications. I agree with Jackub that we shouldn't add specific notifications just for NLM_F_ECHO. That means, either we echo all the existing notifications (or none if the device hasn't been modified), or we leave NLM_F_ECHO unimplemented for SET operations.
As I said in my previous reply, I can't see any use case for ECHO on SET operations, so I don't mind if we skip it.
Quoted 2 lines
At latest, when call
rtnl_delete_link(), there is no way to call__dev_notify_flags(). So we still need to use the current way.
Not everything has to be done with __dev_notify_flags(). For DEL operations notifications are sent in netdev_unregister_many(). Given the overwhelming number of callers, modifying its prototype isn't going to be practical, but you can use a wrapper.
Something like (very rough idea):
-void unregister_netdevice_many(struct list_head *head)
+void unregister_netdevice_many_notify(struct list_head *head, u32 portid,
+ const struct nlmsghdr *nlh)
{
[...]
if (!dev->rtnl_link_ops ||
dev->rtnl_link_state == RTNL_LINK_INITIALIZED)- skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, dev, ~0U, 0,
- GFP_KERNEL, NULL, 0);
+ skb = rtmsg_ifinfo_build_skb(RTM_DELLINK, portid,
+ nlh->nlmsg_seq, dev, ~0U,
+ 0, GFP_KERNEL, NULL, 0);
[...]
if (skb)
- rtmsg_ifinfo_send(skb, dev, GFP_KERNEL);
+ rtmsg_ifinfo_send(skb, portid, dev, nlh, GFP_KERNEL);
[...]
}
+void unregister_netdevice_many(struct list_head *head) +{
+ unregister_netdevice_many_notify(head, 0, NULL);
+}
Quoted 4 lines
As a summarize, we need to change a lot of code if we use
__dev_notify_flags()to notify user, while we can only use it in one place. This looks not worth.WDYT?
Using __dev_notify_flags() is only for NEW operations. SET and DEL have to be handled differently.
To summarise, the objective is to reuse the existing notifications. In practice that means just doing the required plumbing to pass the correct parameters to the existing nlmsg_notify() calls (and ensure we set the correct portid and sequence number in the netlink message header).
Quoted 3 lines
Thanks
Hangbin
Hi Jakub,
net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018
Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4
-0@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb, } } + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL );
BTW, in do_setlink() I planed to use RTM_SETLINK. But I found iproute2 use RTM_NEWLINK to set links. And I saw an old doc[1] said
"""
- RTM_SETLINK does not follow the usual rtnetlink conventions and ignores all netlink flags
The RTM_NEWLINK message type is a superset of RTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """
So I just use RTM_NEWLINK for the notification. Do you think if we should use RTM_SETLINK?
[1] https://lwn.net/Articles/236919/
Thanks
Hangbin
Quoted 18 lines · Hangbin Liu
Hi Jakub,
net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018
Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4
-0@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb, } } + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL );BTW, in
do_setlink()I planed to useRTM_SETLINK. But I found iproute2 useRTM_NEWLINKto set links. And I saw an old doc[1] said"""
-RTM_SETLINKdoes not follow the usual rtnetlink conventions and ignores all netlink flags
Quoted 6 lines
The
RTM_NEWLINKmessage type is a superset ofRTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """So I just use
RTM_NEWLINKfor the notification. Do you think if we should useRTM_SETLINK?
This problem should go away once you switch to reusing the existing notifications (instead of adding new ones).
Quoted 5 lines
[1] https://lwn.net/Articles/236919/
Thanks
Hangbin
Jakub Kicinski
maintainerQuoted 14 lines · On Wed, 28 Sep 2022 11:22:11 +0800 Hangbin Liu wrote:
Hi Jakub,
net/core/rtnetlink.c static int do_setlink(const struct sk_buff *skb, · line 3018
Quoted 8 lines · Hangbin Liu
net/core/rtnetlink.c +4
-0@@ -3009,6 +3012,11 @@ static int do_setlink(const struct sk_buff *skb, } } + nskb =rtmsg_ifinfo_build_skb (RTM_NEWLINK , dev,0 ,0 ,GFP_KERNEL ,NULL , +0 , pid, nlh->nlmsg_seq); +if (nskb) +rtnl_notify (nskb,dev_net (dev), pid,RTNLGRP_LINK , nlh,GFP_KERNEL );BTW, in
do_setlink()I planed to useRTM_SETLINK. But I found iproute2 useRTM_NEWLINKto set links. And I saw an old doc[1] said
Quoted 7 lines
"""
-RTM_SETLINKdoes not follow the usual rtnetlink conventions and ignores all netlink flagsThe
RTM_NEWLINKmessage type is a superset ofRTM_SETLINK, it allows to change both driver specific and generic attributes of the device. """
Interesting, so we actually do use this "NEW as SET" thing.
Quoted 2 lines
So I just use
RTM_NEWLINKfor the notification. Do you think if we should useRTM_SETLINK?
FWIW I think it's typical for rtnl / classic netlink to generate "new object" notification whenever object is changed, rather than a notification about just the change.