net: dqs: introduce NETIF_F_NO_BQL device feature
Subsystems (4)
PATCH 1/1 net: dqs: introduce NETIF_F_NO_BQL device feature
From: Jason Xing <[email protected]>
Since commit 74293ea1c4db6 ("net: sysfs: Do not create sysfs for non BQL device") limits the non-BQL driver not creating byte_queue_limits directory, I found there is one exception, namely, virtio-net driver, which should also be limited in netdev_uses_bql().
I decided to introduce a NO_BQL bit in device feature because
1) it can help us limit virtio-net driver for now.
2) if we found another non-BQL driver, we can take it into account.
3) we can replace all the driver meeting those two statements in netdev_uses_bql() in future.
For now, I would like to make the first step to use this new bit for dqs use instead of replacing/applying all the non-BQL drivers.
After this patch, 1) there is no byte_queue_limits directory in virtio-net driver. 2) running ethtool -k eth1 shows "no-bql: on [fixed]".
--- drivers/net/virtio_net.c | 2 +- include/linux/netdev_features.h | 3 ++- net/core/net-sysfs.c | 2 +- net/ethtool/common.c | 1 + 4 files changed, 5 insertions(+), 3 deletions(-)
drivers/net/virtio_net.c +1 -1
@@ -5634,7 +5634,7 @@ static int virtnet_probe(struct virtio_device *vdev)IFF_TX_SKB_NO_LINEAR ; dev->netdev_ops = &virtnet_netdev; dev->stat_ops = &virtnet_stat_ops; - dev->features =NETIF_F_HIGHDMA ; + dev->features =NETIF_F_HIGHDMA |NETIF_F_NO_BQL ; dev->ethtool_ops = &virtnet_ethtool_ops;SET_NETDEV_DEV (dev, &vdev->dev);
include/linux/netdev_features.h +2 -1
@@ -14,7 +14,6 @@ typedef u64 netdev_features_t;enum {NETIF_F_SG_BIT ,/* Scatter/gather IO. */ NETIF_F_IP_CSUM_BIT ,/* Can checksum TCP/UDP over IPv4. */ - __UNUSED_NETIF_F_1,NETIF_F_HW_CSUM_BIT ,/* Can checksum all the packets. */ NETIF_F_IPV6_CSUM_BIT ,/* Can checksum TCP/UDP over IPV6 */ NETIF_F_HIGHDMA_BIT ,/* Can DMA to high memory. */
@@ -91,6 +90,7 @@ enum {NETIF_F_HW_HSR_FWD_BIT ,/* Offload HSR forwarding */ NETIF_F_HW_HSR_DUP_BIT ,/* Offload HSR duplication */ +NETIF_F_NO_BQL_BIT ,/* non-BQL driver */ /* * Add your fresh new feature above and remember to update * netdev_features_strings[] in net/ethtool/common.c and maybe
@@ -168,6 +168,7 @@ enum {#define NETIF_F_HW_HSR_TAG_RM __NETIF_F(HW_HSR_TAG_RM)#define NETIF_F_HW_HSR_FWD __NETIF_F(HW_HSR_FWD)#define NETIF_F_HW_HSR_DUP __NETIF_F(HW_HSR_DUP) +#define NETIF_F_NO_BQL __NETIF_F(NO_BQL)/* Finds the next feature with the highest number of the range of start-1 till 0. */
net/core/net-sysfs.c +1 -1
@@ -1764,7 +1764,7 @@ static const struct kobj_type netdev_queue_ktype = {static bool netdev_uses_bql (const struct net_device *dev) { -if (dev->features &NETIF_F_LLTX || +if (dev->features & (NETIF_F_LLTX |NETIF_F_NO_BQL ) || dev->priv_flags &IFF_NO_QUEUE )return false;
net/ethtool/common.c +1 -0
@@ -74,6 +74,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = { [NETIF_F_HW_HSR_TAG_RM_BIT] = "hsr-tag-rm-offload", [NETIF_F_HW_HSR_FWD_BIT] = "hsr-fwd-offload", [NETIF_F_HW_HSR_DUP_BIT] = "hsr-dup-offload", + [NETIF_F_NO_BQL_BIT] = "no-bql", }; const char
Signature · 1 line
2.37.3
Quoted 14 lines · Jason Xing
From: Jason Xing <[email protected]>
Since commit
74293ea1c4db6("net: sysfs: Do not create sysfs for non BQL device") limits the non-BQL driver not creatingbyte_queue_limitsdirectory, I found there is one exception, namely, virtio-net driver, which should also be limited innetdev_uses_bql().I decided to introduce a
NO_BQLbit in device feature because
1) it can help us limit virtio-net driver for now.
2) if we found another non-BQL driver, we can take it into account.
3) we can replace all the driver meeting those two statements innetdev_uses_bql()in future.
Quoted 7 lines
For now, I would like to make the first step to use this new bit for dqs use instead of replacing/applying all the non-BQL drivers.
After this patch, 1) there is no
byte_queue_limitsdirectory in virtio-net driver. 2) running ethtool -k eth1 shows "no-bql: on [fixed]".Signed-off-by Jason Xing <[email protected]>
I do not think we want to consume a precious bit from dev->features for something like that.
dev->features should be reserved for bits we used in the fast path, for instance netif_skb_features(). It is good to have free bits for future fast path use.
(I think Vladimir was trying to make some room, this was a discussion we had last year)
I do not see the reason to report to ethtool the 'nobql bit' :
If a driver opts-out, then the bql sysfs files will not be there, user space can see the absence of the files.
Quoted 24 lines · Eric Dumazet
Quoted 14 lines · Jason Xing
From: Jason Xing <[email protected]>
Since commit
74293ea1c4db6("net: sysfs: Do not create sysfs for non BQL device") limits the non-BQL driver not creatingbyte_queue_limitsdirectory, I found there is one exception, namely, virtio-net driver, which should also be limited innetdev_uses_bql().I decided to introduce a
NO_BQLbit in device feature because
1) it can help us limit virtio-net driver for now.
2) if we found another non-BQL driver, we can take it into account.
3) we can replace all the driver meeting those two statements innetdev_uses_bql()in future.Quoted 7 lines
For now, I would like to make the first step to use this new bit for dqs use instead of replacing/applying all the non-BQL drivers.
After this patch, 1) there is no
byte_queue_limitsdirectory in virtio-net driver. 2) running ethtool -k eth1 shows "no-bql: on [fixed]".Signed-off-by Jason Xing <[email protected]>
Quoted 8 lines
I do not think we want to consume a precious bit from
dev->featuresfor something like that.
dev->featuresshould be reserved for bits we used in the fast path, for instancenetif_skb_features(). It is good to have free bits for future fast path use.(I think Vladimir was trying to make some room, this was a discussion we had last year)
Thanks for your reminder. When I was trying to introduce one new bit, I noticed an overflow warning when compiling.
Quoted 4 lines
I do not see the reason to report to ethtool the 'nobql bit' :
If a driver opts-out, then the bql sysfs files will not be there, user space can see the absence of the files.
The reason is that I just followed the comment to force myself to report to ethtool. Now I see.
It seems not that easy to consider all the non-BQL drivers. Let me think more about it.
Thanks,
Jason
Jakub Kicinski
maintainerQuoted 2 lines · On Mon, 10 Jun 2024 07:55:55 +0800 Jason Xing wrote:
Quoted 2 lines
(I think Vladimir was trying to make some room, this was a discussion we had last year)
s/Vladimir/Olek/ ?
Quoted 7 lines
Thanks for your reminder. When I was trying to introduce one new bit, I noticed an overflow warning when compiling.
Quoted 3 lines
I do not see the reason to report to ethtool the 'nobql bit' :
If a driver opts-out, then the bql sysfs files will not be there, user space can see the absence of the files.
Quoted 5 lines
The reason is that I just followed the comment to force myself to report to ethtool. Now I see.
It seems not that easy to consider all the non-BQL drivers. Let me think more about it.
All Eric was saying, AFAIU, is that you can for example add a bit in somewhere towards the end of struct nedevice, no need to pack this info into feature bits.
BTW the Fixes tag is a bit of an exaggeration here. The heuristic in netdev_uses_bql() is best effort, its fine to miss some devices.
Quoted 17 lines · Jakub Kicinski
Quoted 2 lines · On Mon, 10 Jun 2024 07:55:55 +0800 Jason Xing wrote:
Quoted 2 lines
(I think Vladimir was trying to make some room, this was a discussion we had last year)
s/Vladimir/Olek/ ?
Quoted 10 lines
Thanks for your reminder. When I was trying to introduce one new bit, I noticed an overflow warning when compiling.
Quoted 3 lines
I do not see the reason to report to ethtool the 'nobql bit' :
If a driver opts-out, then the bql sysfs files will not be there, user space can see the absence of the files.The reason is that I just followed the comment to force myself to report to ethtool. Now I see.
Quoted 6 lines
Quoted 2 lines
It seems not that easy to consider all the non-BQL drivers. Let me think more about it.
All Eric was saying, AFAIU, is that you can for example add a bit in somewhere towards the end of struct nedevice, no need to pack this info into feature bits.
Oh, thanks for pointing this out.
I would like to add a new bit field in the enum netdev_priv_flags because it is better that it's grouped into an existing enum for future compatibility.
Quoted 3 lines
BTW the Fixes tag is a bit of an exaggeration here. The heuristic in
netdev_uses_bql()is best effort, its fine to miss some devices.
I see.
Thanks,
Jason
Quoted 13 lines · Sun, Jun 09, 2024 at 03:17:32PM CEST, [email protected] wrote:
From: Jason Xing <[email protected]>
Since commit
74293ea1c4db6("net: sysfs: Do not create sysfs for non BQL device") limits the non-BQL driver not creatingbyte_queue_limitsdirectory, I found there is one exception, namely, virtio-net driver, which should also be limited innetdev_uses_bql().I decided to introduce a
NO_BQLbit in device feature because
1) it can help us limit virtio-net driver for now.
2) if we found another non-BQL driver, we can take it into account.
3) we can replace all the driver meeting those two statements innetdev_uses_bql()in future.
Quoted 5 lines
For now, I would like to make the first step to use this new bit for dqs use instead of replacing/applying all the non-BQL drivers.
After this patch, 1) there is no
byte_queue_limitsdirectory in virtio-net driver. 2) running ethtool -k eth1 shows "no-bql: on [fixed]".
Wait, you introduce this flag only for the sake of virtio_net driver, don't you. Since there is currently an attempt to implement bql in virtio_net, wouldn't it make this flag obsolete? Can't you wait?
Quoted 79 lines
Signed-off-by Jason Xing <[email protected]>---
drivers/net/virtio_net.c | 2 +- include/linux/netdev_features.h | 3 ++- net/core/net-sysfs.c | 2 +- net/ethtool/common.c | 1 + 4 files changed, 5 insertions(+), 3 deletions(-)drivers/net/virtio_net.c +1
-1@@ -5634,7 +5634,7 @@ static int virtnet_probe(struct virtio_device *vdev)IFF_TX_SKB_NO_LINEAR ; dev->netdev_ops = &virtnet_netdev; dev->stat_ops = &virtnet_stat_ops; - dev->features =NETIF_F_HIGHDMA ; + dev->features =NETIF_F_HIGHDMA |NETIF_F_NO_BQL ; dev->ethtool_ops = &virtnet_ethtool_ops;SET_NETDEV_DEV (dev, &vdev->dev);include/linux/netdev_features.h +2
-1@@ -14,7 +14,6 @@ typedef u64 netdev_features_t;enum {NETIF_F_SG_BIT ,/* Scatter/gather IO. */ NETIF_F_IP_CSUM_BIT ,/* Can checksum TCP/UDP over IPv4. */ - __UNUSED_NETIF_F_1,NETIF_F_HW_CSUM_BIT ,/* Can checksum all the packets. */ NETIF_F_IPV6_CSUM_BIT ,/* Can checksum TCP/UDP over IPV6 */ NETIF_F_HIGHDMA_BIT ,/* Can DMA to high memory. */ @@ -91,6 +90,7 @@ enum {NETIF_F_HW_HSR_FWD_BIT ,/* Offload HSR forwarding */ NETIF_F_HW_HSR_DUP_BIT ,/* Offload HSR duplication */ +NETIF_F_NO_BQL_BIT ,/* non-BQL driver */ /* * Add your fresh new feature above and remember to update * netdev_features_strings[] in net/ethtool/common.c and maybe @@ -168,6 +168,7 @@ enum {#define NETIF_F_HW_HSR_TAG_RM __NETIF_F(HW_HSR_TAG_RM)#define NETIF_F_HW_HSR_FWD __NETIF_F(HW_HSR_FWD)#define NETIF_F_HW_HSR_DUP __NETIF_F(HW_HSR_DUP) +#define NETIF_F_NO_BQL __NETIF_F(NO_BQL)/* Finds the next feature with the highest number of the range of start-1 till 0. */ net/core/net-sysfs.c +1
-1@@ -1764,7 +1764,7 @@ static const struct kobj_type netdev_queue_ktype = {static bool netdev_uses_bql (const struct net_device *dev) { -if (dev->features &NETIF_F_LLTX || +if (dev->features & (NETIF_F_LLTX |NETIF_F_NO_BQL ) || dev->priv_flags &IFF_NO_QUEUE )return false;net/ethtool/common.c +1
-0@@ -74,6 +74,7 @@ const char netdev_features_strings[NETDEV_FEATURE_COUNT][ETH_GSTRING_LEN] = { [NETIF_F_HW_HSR_TAG_RM_BIT] = "hsr-tag-rm-offload", [NETIF_F_HW_HSR_FWD_BIT] = "hsr-fwd-offload", [NETIF_F_HW_HSR_DUP_BIT] = "hsr-dup-offload", + [NETIF_F_NO_BQL_BIT] = "no-bql", };const char
Signature · 1 line
2.37.3
Quoted 18 lines · Jiri Pirko
Quoted 16 lines · Sun, Jun 09, 2024 at 03:17:32PM CEST, [email protected] wrote:
From: Jason Xing <[email protected]>
Since commit
74293ea1c4db6("net: sysfs: Do not create sysfs for non BQL device") limits the non-BQL driver not creatingbyte_queue_limitsdirectory, I found there is one exception, namely, virtio-net driver, which should also be limited innetdev_uses_bql().I decided to introduce a
NO_BQLbit in device feature because
1) it can help us limit virtio-net driver for now.
2) if we found another non-BQL driver, we can take it into account.
3) we can replace all the driver meeting those two statements innetdev_uses_bql()in future.For now, I would like to make the first step to use this new bit for dqs use instead of replacing/applying all the non-BQL drivers.
Quoted 6 lines
Quoted 2 lines
After this patch, 1) there is no
byte_queue_limitsdirectory in virtio-net driver. 2) running ethtool -k eth1 shows "no-bql: on [fixed]".Wait, you introduce this flag only for the sake of
virtio_netdriver, don't you. Since there is currently an attempt to implement bql invirtio_net, wouldn't it make this flag obsolete? Can't you wait?
I admit the way of introducing a new flag is not a good way to go, but I cannot figure out a better way. I'm still thinking. Virtio_net driver, I think, is not the only non-BQL we miss in dqs.
You can keep trying sending BQL patches for the virtio_net driver but they don't conflict with the current patch. Can you guarantee you can smoothly introduce BQL for virtio_net in a few days? I have no confidence in your recent action about blindly deleting old features and introducing a new one without considering the real world, I mean, the world outside of the community. They are thousands and hundreds of servers running outside. You have to prove deleting an old one doesn't harm the users instead of asking users to prove the old feature is useful. It's very weird. The key reason is about whether it breaks userspace compatibility and how you handle the compatibility. For example, I don't think kernel developers can delete some old proc files even though they are buggy/inaccurate. I'm just saying what I'm worried about. Let the maintainers decide finally.
+ Jason Wang, + Michael S. Tsirkin
Let's get back to this patch, I think actually it is a bug we need to fix since netdev_uses_bql() decided to limit non-BQL drivers. I'm still thinking, as I said...