net: sched: calls synchronize_net() only when needed
Subsystems (3)
PATCH 1/1 net: sched: calls synchronize_net() only when needed
dev_deactivate_many() role is to remove the qdiscs of a network device.
When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
Most virtual devices do not have a qdisc (if we exclude noqueue ones).
We can call the expensive synchronize_net() only if needed.
Note that dev_deactivate_many() does not have to deal with qdisc-less dev_queue_xmit, as an old comment was claiming.
--- net/sched/sch_generic.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
net/sched/sch_generic.c +11 -11
@@ -1277,15 +1277,17 @@ static void qdisc_deactivate(struct Qdisc *qdisc)static void dev_deactivate_queue (struct net_device *dev,struct netdev_queue *dev_queue, -void *_qdisc_default) +void *_sync_needed) { -struct Qdisc *qdisc_default = _qdisc_default; +bool *sync_needed = _sync_needed;struct Qdisc *qdisc; qdisc =rtnl_dereference (dev_queue->qdisc);if (qdisc) { +if (qdisc->enqueue) + *sync_needed = true;qdisc_deactivate (qdisc); -rcu_assign_pointer (dev_queue->qdisc, qdisc_default); +rcu_assign_pointer (dev_queue->qdisc, &noop_qdisc); } }
@@ -1352,24 +1354,22 @@ static bool some_qdisc_is_busy(struct net_device *dev)*/ void dev_deactivate_many (struct list_head *head) { +bool sync_needed = false;struct net_device *dev;list_for_each_entry (dev, head, close_list) {netdev_for_each_tx_queue (dev, dev_deactivate_queue, - &noop_qdisc); + &sync_needed);if (dev_ingress_queue (dev))dev_deactivate_queue (dev,dev_ingress_queue (dev), - &noop_qdisc); + &sync_needed);netdev_watchdog_down (dev); } -/* Wait for outstanding qdisc-less dev_queue_xmit calls or -* outstanding qdisc enqueuing calls. -* This is avoided if all devices are in dismantle phase : -* Caller will call synchronize_net() for us -*/ -synchronize_net (); +/* Wait for outstanding qdisc enqueuing calls. */ +if (sync_needed) +synchronize_net ();list_for_each_entry (dev, head, close_list) {netdev_for_each_tx_queue (dev, dev_reset_queue,NULL );
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Quoted 9 lines · Eric Dumazet
dev_deactivate_many()role is to remove the qdiscs of a network device.When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
Most virtual devices do not have a qdisc (if we exclude noqueue ones).
Such as? To me, most virtual devices use noqueue:
$ git grep IFF_NO_QUEUE -- drivers/net/ drivers/net/amt.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bareudp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bonding/bond_main.c: bond_dev->priv_flags |= IFF_BONDING | IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/caif/caif_serial.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/dummy.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ethernet/netronome/nfp/nfp_net_repr.c: netdev->priv_flags |= IFF_NO_QUEUE | IFF_DISABLE_NETPOLL; drivers/net/geneve.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/gtp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ipvlan/ipvlan_main.c: dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/ipvlan/ipvtap.c: dev->priv_flags &= ~IFF_NO_QUEUE; drivers/net/loopback.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/macsec.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/macvlan.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/net_failover.c: failover_dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/netdevsim/netdev.c: IFF_NO_QUEUE; drivers/net/netkit.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/nlmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/pfcp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/team/team_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/veth.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vrf.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vsockmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vxlan/vxlan_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wan/hdlc_fr.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireguard/device.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireless/virtual/mac80211_hwsim.c: dev->priv_flags |= IFF_NO_QUEUE;
And noqueue_qdisc_ops sets ->enqueue to noop_enqueue():
struct Qdisc_ops noqueue_qdisc_ops __read_mostly = {
.id = "noqueue",
.priv_size = 0,
.init = noqueue_init,
.enqueue = noop_enqueue,
.dequeue = noop_dequeue,
.peek = noop_dequeue,
.owner = THIS_MODULE,};
Thanks.
Quoted 47 lines · Cong Wang
Quoted 9 lines · Eric Dumazet
dev_deactivate_many()role is to remove the qdiscs of a network device.When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
Most virtual devices do not have a qdisc (if we exclude noqueue ones).
Such as? To me, most virtual devices use noqueue:
$ git grep IFF_NO_QUEUE -- drivers/net/ drivers/net/amt.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bareudp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bonding/bond_main.c: bond_dev->priv_flags |= IFF_BONDING | IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/caif/caif_serial.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/dummy.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ethernet/netronome/nfp/nfp_net_repr.c: netdev->priv_flags |= IFF_NO_QUEUE | IFF_DISABLE_NETPOLL; drivers/net/geneve.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/gtp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ipvlan/ipvlan_main.c: dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/ipvlan/ipvtap.c: dev->priv_flags &= ~IFF_NO_QUEUE; drivers/net/loopback.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/macsec.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/macvlan.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/net_failover.c: failover_dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/netdevsim/netdev.c: IFF_NO_QUEUE; drivers/net/netkit.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/nlmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/pfcp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/team/team_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/veth.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vrf.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vsockmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vxlan/vxlan_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wan/hdlc_fr.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireguard/device.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireless/virtual/mac80211_hwsim.c: dev->priv_flags |= IFF_NO_QUEUE;And
noqueue_qdisc_opssets ->enqueue tonoop_enqueue():struct
Qdisc_opsnoqueue_qdisc_ops__read_mostly= {
Quoted 8 lines
.id = "noqueue", .priv_size = 0, .init = noqueue_init, .enqueue = noop_enqueue, .dequeue = noop_dequeue, .peek = noop_dequeue, .owner = THIS_MODULE,};
Sure, but please a look at :
static int noqueue_init(struct Qdisc *qdisc, struct nlattr *opt, struct netlink_ext_ack *extack)
{
/* register_qdisc() assigns a default of noop_enqueue if unset,
* but __dev_queue_xmit() treats noqueue only as such
* if this is NULL - so clear it here. */
qdisc->enqueue = NULL;
return 0;
}Quoted 62 lines · Eric Dumazet
Quoted 47 lines · Cong Wang
Quoted 9 lines · Eric Dumazet
dev_deactivate_many()role is to remove the qdiscs of a network device.When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
Most virtual devices do not have a qdisc (if we exclude noqueue ones).
Such as? To me, most virtual devices use noqueue:
$ git grep IFF_NO_QUEUE -- drivers/net/ drivers/net/amt.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bareudp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bonding/bond_main.c: bond_dev->priv_flags |= IFF_BONDING | IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/caif/caif_serial.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/dummy.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ethernet/netronome/nfp/nfp_net_repr.c: netdev->priv_flags |= IFF_NO_QUEUE | IFF_DISABLE_NETPOLL; drivers/net/geneve.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/gtp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ipvlan/ipvlan_main.c: dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/ipvlan/ipvtap.c: dev->priv_flags &= ~IFF_NO_QUEUE; drivers/net/loopback.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/macsec.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/macvlan.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/net_failover.c: failover_dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/netdevsim/netdev.c: IFF_NO_QUEUE; drivers/net/netkit.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/nlmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/pfcp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/team/team_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/veth.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vrf.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vsockmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vxlan/vxlan_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wan/hdlc_fr.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireguard/device.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireless/virtual/mac80211_hwsim.c: dev->priv_flags |= IFF_NO_QUEUE;And
noqueue_qdisc_opssets ->enqueue tonoop_enqueue():struct
Qdisc_opsnoqueue_qdisc_ops__read_mostly= {Quoted 8 lines
.id = "noqueue", .priv_size = 0, .init = noqueue_init, .enqueue = noop_enqueue, .dequeue = noop_dequeue, .peek = noop_dequeue, .owner = THIS_MODULE,};
Sure, but please a look at :
static int
noqueue_init(struct Qdisc *qdisc, struct nlattr *opt, struct netlink_ext_ack *extack)
Quoted 7 lines
{
/* register_qdisc() assigns a default of noop_enqueue if unset,
* but __dev_queue_xmit() treats noqueue only as such
* if this is NULL - so clear it here. */
qdisc->enqueue = NULL;
return 0;
}How can we proceed on this patch ?
I can remove the "(if we exclude noqueue ones)" part if this is confusing.
Paolo Abeni
maintainerQuoted 71 lines · Eric Dumazet
Quoted 62 lines · Eric Dumazet
Quoted 47 lines · Cong Wang
Quoted 9 lines · Eric Dumazet
dev_deactivate_many()role is to remove the qdiscs of a network device.When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
Most virtual devices do not have a qdisc (if we exclude noqueue ones).
Such as? To me, most virtual devices use noqueue:
$ git grep IFF_NO_QUEUE -- drivers/net/ drivers/net/amt.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bareudp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/bonding/bond_main.c: bond_dev->priv_flags |= IFF_BONDING | IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/caif/caif_serial.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/dummy.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ethernet/netronome/nfp/nfp_net_repr.c: netdev->priv_flags |= IFF_NO_QUEUE | IFF_DISABLE_NETPOLL; drivers/net/geneve.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/gtp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/ipvlan/ipvlan_main.c: dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/ipvlan/ipvtap.c: dev->priv_flags &= ~IFF_NO_QUEUE; drivers/net/loopback.c: dev->priv_flags |= IFF_LIVE_ADDR_CHANGE | IFF_NO_QUEUE; drivers/net/macsec.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/macvlan.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/net_failover.c: failover_dev->priv_flags |= IFF_UNICAST_FLT | IFF_NO_QUEUE; drivers/net/netdevsim/netdev.c: IFF_NO_QUEUE; drivers/net/netkit.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/nlmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/pfcp.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/team/team_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/veth.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vrf.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vsockmon.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/vxlan/vxlan_core.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wan/hdlc_fr.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireguard/device.c: dev->priv_flags |= IFF_NO_QUEUE; drivers/net/wireless/virtual/mac80211_hwsim.c: dev->priv_flags |= IFF_NO_QUEUE;And
noqueue_qdisc_opssets ->enqueue tonoop_enqueue():struct
Qdisc_opsnoqueue_qdisc_ops__read_mostly= {Quoted 8 lines
.id = "noqueue", .priv_size = 0, .init = noqueue_init, .enqueue = noop_enqueue, .dequeue = noop_dequeue, .peek = noop_dequeue, .owner = THIS_MODULE,};
Sure, but please a look at :
static int
noqueue_init(struct Qdisc *qdisc, struct nlattr *opt, struct netlink_ext_ack *extack)Quoted 7 lines
{ /* register_qdisc() assigns a default of noop_enqueue if unset, * but __dev_queue_xmit() treats noqueue only as such * if this is NULL - so clear it here. */ qdisc->enqueue = NULL; return 0; }
Quoted 3 lines
How can we proceed on this patch ?
I can remove the "(if we exclude noqueue ones)" part if this is confusing.
I personally interpret the lack of reply from Cong to your previous message as agreement/understanding. I'll drop the blamed sentence from the commit message when applying the patch.
BTW, in the long run I think it would be nice to remove the noqueue_init hack - AFAICS no in-tree kernel has NULL enqueue/dequeue/peek CB, so the related checks in register_qdisc() could possibly be dropped.
/P
This patch was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Hello:
This patch was applied to netdev/net-next.git (main) by Paolo Abeni <[email protected]>:
Quoted 8 lines · On Thu, 9 Jan 2025 17:18:50 +0000 you wrote:
dev_deactivate_many()role is to remove the qdiscs of a network device.When/if a qdisc is dismantled, an rcu grace period is needed to make sure all outstanding qdisc enqueue are done before we proceed with a qdisc reset.
[...]
Here is the summary with links:
- [net-next] net: sched: calls synchronize_net() only when needed
https://git.kernel.org/netdev/net-next/c/88df16f851ad
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html