tap/bridge: Dropping NETIF_F_GSO/NETIF_F_SG
Subsystems (3)
BTW, I just noticed that net-next spits out many of the following when I run any VMs:
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Features changed: 0x401b4849 -> 0x40004040 device msttap0 entered promiscuous mode
br0: Dropping NETIF_F_GSO since no SG feature.
br0: port 1(msttap0) entering forwarding state
br0: port 1(msttap0) entering forwarding state
tap0: Features changed: 0x40004040 -> 0x40024849
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Features changed: 0x40024849 -> 0x40004040
br0: Dropping NETIF_F_GSO since no SG feature.
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Features changed: 0x40004040 -> 0x401b4849
tap0: Dropping NETIF_F_SG since no checksum feature.
tap0: Dropping NETIF_F_GSO since no SG feature.
tap0: Features changed: 0x401b4849 -> 0x40004040
br0: Dropping NETIF_F_GSO since no SG feature.
My problem is not primarily with warnings:
will that linearize all packets and disable GSO for tap and bridge? If yes it can't be good for performance...
Signature · 1 line
MST
Quoted 29 lines · Michael S. Tsirkin
BTW, I just noticed that net-next spits out many of the following when I run any VMs:
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040device msttap0 entered promiscuous mode
br0: DroppingNETIF_F_GSOsince no SG feature.
br0: port 1(msttap0) entering forwarding state
br0: port 1(msttap0) entering forwarding state
tap0: Features changed:0x40004040->0x40024849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40024849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40004040->0x401b4849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
Quoted 5 lines
My problem is not primarily with warnings:
will that linearize all packets and disable GSO for tap and bridge? If yes it can't be good for performance...
I think so. So the question is why is checksum off?
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 30 lines · Herbert Xu
Quoted 29 lines · Michael S. Tsirkin
BTW, I just noticed that net-next spits out many of the following when I run any VMs:
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040device msttap0 entered promiscuous mode
br0: DroppingNETIF_F_GSOsince no SG feature.
br0: port 1(msttap0) entering forwarding state
br0: port 1(msttap0) entering forwarding state
tap0: Features changed:0x40004040->0x40024849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40024849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40004040->0x401b4849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
Quoted 6 lines
Quoted 5 lines
My problem is not primarily with warnings:
will that linearize all packets and disable GSO for tap and bridge? If yes it can't be good for performance...
I think so. So the question is why is checksum off?
Whatever application is creating the tap0 device is not calling ioctl(TUNSETOFFLOAD) with TUN_F_CSUM set. This is userspace bug/feature exposed by recent changes to netdev features handling.
Best Regards,
Michał Mirosław
Quoted 4 lines · Michał Mirosław
Whatever application is creating the tap0 device is not calling ioctl(TUNSETOFFLOAD) with
TUN_F_CSUMset. This is userspace bug/feature exposed by recent changes to netdev features handling.
OK, then user-space needs to be fixed.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 36 lines · Michał Mirosław
Quoted 35 lines · Herbert Xu
Quoted 34 lines · Michael S. Tsirkin
BTW, I just noticed that net-next spits out many of the following when I run any VMs:
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040device msttap0 entered promiscuous mode
br0: DroppingNETIF_F_GSOsince no SG feature.
br0: port 1(msttap0) entering forwarding state
br0: port 1(msttap0) entering forwarding state
tap0: Features changed:0x40004040->0x40024849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40024849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40004040->0x401b4849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.My problem is not primarily with warnings:
will that linearize all packets and disable GSO for tap and bridge? If yes it can't be good for performance...
Quoted 8 lines
Quoted 1 line
I think so. So the question is why is checksum off?
Whatever application is creating the tap0 device is not calling ioctl(TUNSETOFFLOAD) with
TUN_F_CSUMset. This is userspace bug/feature exposed by recent changes to netdev features handling.Best Regards,
Michał Mirosław
No, I think it's not a bug in userspace. Here is what it does:
/* Check if our kernel supports TUNSETOFFLOAD */
if (ioctl(fd, TUNSETOFFLOAD, 0) != 0 && errno == EINVAL) {
return;
}
if (csum) {
offload |= TUN_F_CSUM;
if (tso4)
offload |= TUN_F_TSO4;
if (tso6)
offload |= TUN_F_TSO6;
if ((tso4 || tso6) && ecn)
offload |= TUN_F_TSO_ECN;
if (ufo)
offload |= TUN_F_UFO;
}
if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) {
offload &= ~TUN_F_UFO;
if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) {
fprintf(stderr, "TUNSETOFFLOAD ioctl() failed: %s\n",
strerror(errno));
}
}The first call is just to check that ioctl works.
When checking for ioctl in this way, userspace clears checksum. This will clear SG and thus GSO; later userspace enables checksum. checksum is on but SG is by now disabled so GSO gets cleared again too.
It's also likely a problem that
userspace can trigger warnings in log for what used to be a legal way to check for ioctl and/or disable checksum offloading, but that is more minor.
Signature · 1 line
MST
Michael S. Tsirkin wrote, at 05/05/2011 04:44 PM:
Quoted 79 lines · Michał Mirosław
Quoted 36 lines · Herbert Xu
Quoted 35 lines · Michael S. Tsirkin
Quoted 34 lines
BTW, I just noticed that net-next spits out many of the following when I run any VMs:
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040device msttap0 entered promiscuous mode
br0: DroppingNETIF_F_GSOsince no SG feature.
br0: port 1(msttap0) entering forwarding state
br0: port 1(msttap0) entering forwarding state
tap0: Features changed:0x40004040->0x40024849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40024849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x40004040->0x401b4849
tap0: DroppingNETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040
br0: DroppingNETIF_F_GSOsince no SG feature.My problem is not primarily with warnings:
will that linearize all packets and disable GSO for tap and bridge? If yes it can't be good for performance...
Quoted 8 lines
Quoted 1 line
I think so. So the question is why is checksum off?
Whatever application is creating the tap0 device is not calling ioctl(TUNSETOFFLOAD) with
TUN_F_CSUMset. This is userspace bug/feature exposed by recent changes to netdev features handling.Best Regards,
Michał MirosławNo, I think it's not a bug in userspace. Here is what it does:
/* Check if our kernel supports TUNSETOFFLOAD */ if (ioctl(fd, TUNSETOFFLOAD, 0) != 0 && errno == EINVAL) { return; } if (csum) { offload |= TUN_F_CSUM; if (tso4) offload |= TUN_F_TSO4; if (tso6) offload |= TUN_F_TSO6; if ((tso4 || tso6) && ecn) offload |= TUN_F_TSO_ECN; if (ufo) offload |= TUN_F_UFO; } if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) { offload &= ~TUN_F_UFO; if (ioctl(fd, TUNSETOFFLOAD, offload) != 0) { fprintf(stderr, "TUNSETOFFLOAD ioctl() failed: %s\n", strerror(errno)); } }The first call is just to check that ioctl works.
When checking for ioctl in this way, userspace clears checksum. This will clear SG and thus GSO; later userspace enables checksum. checksum is on but SG is by now disabled so GSO gets cleared again too.
Quoted 4 lines
It's also likely a problem that
userspace can trigger warnings in log for what used to be a legal way to check for ioctl and/or disable checksum offloading, but that is more minor.
Maybe it's a kernel bug. Can you try following changes?
TUN_F_TSO4, TUN_F_TSO6, TUN_F_TSO_ECN, TUN_F_UFO these features are depend on NETIF_F_SG. If NETIF_F_SG is not set, these features are not be enabled and warnings are printed in netdev_fix_features().
drivers/net/tun.c +1 -1
@@ -1178,7 +1178,7 @@ static int set_offload(struct tun_struct *tun, unsigned long arg)u32 features =0 ;if (arg &TUN_F_CSUM ) { - features |=NETIF_F_HW_CSUM ; + features |=NETIF_F_HW_CSUM |NETIF_F_SG ; arg &= ~TUN_F_CSUM ;if (arg & (TUN_F_TSO4 |TUN_F_TSO6 )) {
Signature · 3 lines
Best Regards ----- Shan Wei
Quoted 4 lines · Shan Wei
TUN_F_TSO4,TUN_F_TSO6,TUN_F_TSO_ECN,TUN_F_UFOthese features are depend onNETIF_F_SG. IfNETIF_F_SGis not set, these features are not be enabled and warnings are printed innetdev_fix_features().
No, when the user turns off checksum offload everything should be turned off as well. However, when it's turned on, we shouldn't enable everything automatically.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 2 lines · Michael S. Tsirkin
BTW, I just noticed that net-next spits out
many of the following when I run any VMs:
[...]
Quoted 1 line
tap0: Features changed:
0x40004040->0x401b4849
Before this message, userspace called ioctl(TIOCSETOFFLOAD) turning offloads on.
Quoted 3 lines
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040
And then it probably called ioctl(TIOCSETOFFLOAD) again, disabling them.
Best Regards,
Michał Mirosław
Michał Mirosław wrote, at 05/05/2011 11:26 PM:
Quoted 6 lines · Michael S. Tsirkin
Quoted 2 lines
BTW, I just noticed that net-next spits out
many of the following when I run any VMs:[...]
Quoted 1 line
tap0: Features changed:
0x40004040->0x401b4849
Quoted 8 lines
Before this message, userspace called ioctl(TIOCSETOFFLOAD) turning offloads on.
Quoted 3 lines
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040And then it probably called ioctl(TIOCSETOFFLOAD) again, disabling them.
[263958.167861] tap1: Dropping NETIF_F_SG since no checksum feature. [263958.167866] tap1: Dropping NETIF_F_GSO since no SG feature. [263958.167871] tap1: Features changed: 0x401b4849 -> 0x40004040
See same warning message using tunctl to create a tap device on net-next tree. But on RHEL6, no warning message.
strace shows no TUNSETOFFLOAD to be used.
Seems that checksum feature is not set when creating tap0 interface.
#strace tunctl -u root -t tap1
open("/dev/net/tun", O_RDWR) = 3
ioctl(3, TUNSETIFF, 0x7fff789ada10) = 0
ioctl(3, TUNSETOWNER, 0) = 0
ioctl(3, TUNSETPERSIST, 0x1) = 0fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f66aeed9000 write(1, "Set 'tap1' persistent and owned "..., 41Set 'tap1' persistent and owned by uid 0 ) = 41exit_group(0) = ?
Signature · 3 lines
Best Regards ----- Shan Wei
Quoted 9 lines · Michał Mirosław
Quoted 2 lines · Michael S. Tsirkin
BTW, I just noticed that net-next spits out
many of the following when I run any VMs:[...]
Quoted 1 line
tap0: Features changed:
0x40004040->0x401b4849Before this message, userspace called ioctl(TIOCSETOFFLOAD) turning offloads on.
Quoted 8 lines
Quoted 3 lines
tap0: Dropping
NETIF_F_SGsince no checksum feature.
tap0: DroppingNETIF_F_GSOsince no SG feature.
tap0: Features changed:0x401b4849->0x40004040And then it probably called ioctl(TIOCSETOFFLOAD) again, disabling them.
Best Regards,
Michał Mirosław
I'd have to look at this some more, but I know qemu-kvm supports offloads with current kernels.
There seems to be some kernel behaviour change here.
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines · Herbert Xu
Quoted 4 lines · Shan Wei
TUN_F_TSO4,TUN_F_TSO6,TUN_F_TSO_ECN,TUN_F_UFOthese features are depend onNETIF_F_SG. IfNETIF_F_SGis not set, these features are not be enabled and warnings are printed innetdev_fix_features().No, when the user turns off checksum offload everything should be turned off as well. However, when it's turned on, we shouldn't enable everything automatically.
Cheers,
So how is NETIF_F_SG supposed to be enabled then?
In upstream kernels userspace can disable checksum offloading then re-enable and get SG set back. userspace came to depend on this behaviour so I think changing this is a regression.
Quoted 8 lines
Signature · 7 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · Michael S. Tsirkin
So how is
NETIF_F_SGsupposed to be enabled then?
It should either be enabled at device creation time, or whatever user-space entity managing the device creation should enable it along with checksumming and anything else applicable.
Quoted 3 lines
In upstream kernels userspace can disable checksum offloading then re-enable and get SG set back. userspace came to depend on this behaviour so I think changing this is a regression.
Can you point me to the relevant code in the upstream kernel? I'm not aware of any automatic SG enabling for network devices in general when you enable checksum offloading.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 7 lines · Herbert Xu
Quoted 2 lines · Michael S. Tsirkin
So how is
NETIF_F_SGsupposed to be enabled then?It should either be enabled at device creation time, or whatever user-space entity managing the device creation should enable it along with checksumming and anything else applicable.
There's no interface for userspace to enable it: userspace only has an ioctl to enable/disable checksum offloading. SG is an implementation detail.
Quoted 9 lines
Quoted 3 lines
In upstream kernels userspace can disable checksum offloading then re-enable and get SG set back. userspace came to depend on this behaviour so I think changing this is a regression.
Can you point me to the relevant code in the upstream kernel? I'm not aware of any automatic SG enabling for network devices in general when you enable checksum offloading.
Cheers,
I think what happens _SG is enabled at device creation time and then upstream just keeps it on always, even when user clears CSUM. With net-next code changed so that _SG gets cleared when CSUM 'gets cleared. But then it does not get reenabled when CSUM gets reenabled.
Quoted 4 lines
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 12 lines · Herbert Xu
Quoted 2 lines · Michael S. Tsirkin
So how is
NETIF_F_SGsupposed to be enabled then?It should either be enabled at device creation time, or whatever user-space entity managing the device creation should enable it along with checksumming and anything else applicable.
Quoted 3 lines
In upstream kernels userspace can disable checksum offloading then re-enable and get SG set back. userspace came to depend on this behaviour so I think changing this is a regression.
Quoted 5 lines
Can you point me to the relevant code in the upstream kernel? I'm not aware of any automatic SG enabling for network devices in general when you enable checksum offloading.
Cheers,
By the way with kvm we let an unpriveledged application control netdev flags through and ioctl. It's probably not a good idea to print out stuff on flag change unconditionally as that will let that application fill up the system log.
Quoted 4 lines
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 4 lines · Michael S. Tsirkin
There's no interface for userspace to enable it: userspace only has an ioctl to enable/disable checksum offloading. SG is an implementation detail.
Yes there is: ethtool -K
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 9 lines · Herbert Xu
Quoted 4 lines · Michael S. Tsirkin
There's no interface for userspace to enable it: userspace only has an ioctl to enable/disable checksum offloading. SG is an implementation detail.
Yes there is: ethtool -K
Cheers,
Ok. You are right of course, I was confused.
So it's just an info message: in the end if userspace DTRT features get enabled properly. Functionally, there is no regression.
Both bridge and tun trigger the messages ATM. I note that dev.c takes pains to avoid these messages:
Avoid warning from netdev_fix_features() for GSO without SG
Should all drivers do this: clear _SG if they clear checksum? What about _GSO? If we start doing this, this spills info about flag dependencies out to drivers.
Signature · 1 line
MST
Quoted 7 lines · Michael S. Tsirkin
Both bridge and tun trigger the messages ATM. I note that
dev.ctakes pains to avoid these messages:
Avoid warning fromnetdev_fix_features()for GSO without SGShould all drivers do this: clear
_SGif they clear checksum? What about_GSO? If we start doing this, this spills info about flag dependencies out to drivers.
Drivers don't need to do this at all since the network stack will automatically clear SG and related features if checksum is disabled.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
2011/5/16 Michael S. Tsirkin <[email protected]>:
Quoted 11 lines · Herbert Xu
Quoted 6 lines · Michael S. Tsirkin
Quoted 4 lines
There's no interface for userspace to enable it: userspace only has an ioctl to enable/disable checksum offloading. SG is an implementation detail.
Yes there is: ethtool -K
Ok. You are right of course, I was confused.
So it's just an info message: in the end if userspace DTRT features get enabled properly. Functionally, there is no regression.
I proposed changing those messages to DEBUG level some time ago.Those dependencies are constant, so they could be moved to Documentation/networking/.
Quoted 6 lines
Both bridge and tun trigger the messages ATM. I note that
dev.ctakes pains to avoid these messages:
Avoid warning fromnetdev_fix_features()for GSO without SGShould all drivers do this: clear
_SGif they clear checksum? What about_GSO? If we start doing this, this spills info about flag dependencies out to drivers.
Drivers should not care about those dependencies.
Best Regards,
Michał Mirosław
Quoted 6 lines · Herbert Xu
Quoted 5 lines · Michael S. Tsirkin
Both bridge and tun trigger the messages ATM. I note that
dev.ctakes pains to avoid these messages:
Avoid warning fromnetdev_fix_features()for GSO without SG
Quoted 8 lines
Quoted 2 lines
Should all drivers do this: clear
_SGif they clear checksum? What about_GSO? If we start doing this, this spills info about flag dependencies out to drivers.Drivers don't need to do this at all since the network stack will automatically clear SG and related features if checksum is disabled.
Cheers,
Yes, but then we get annoying info messages every time. Change them all to debug or remove completely?
Quoted 4 lines
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 3 lines · Michael S. Tsirkin
Yes, but then we get annoying info messages every time. Change them all to debug or remove completely?
If you are getting warning messages then it probably means that your driver is doing something wrong.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
2011/5/16 Herbert Xu <[email protected]>:
Quoted 5 lines · Michael S. Tsirkin
Quoted 2 lines
Yes, but then we get annoying info messages every time. Change them all to debug or remove completely?
If you are getting warning messages then it probably means that your driver is doing something wrong.
In this case - no. Those messages inform that driver supports more feature combinations than network core and the feature set is reduced because of that.
Best Regards,
Michał Mirosław
Quoted 4 lines · Michał Mirosław
In this case - no. Those messages inform that driver supports more feature combinations than network core and the feature set is reduced because of that.
The driver should never even claim to support SG/TSO/UFO if it does not support checksum. That is the point of the warning.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
David Miller
maintainerFrom: Herbert Xu <[email protected]>
Date: Tue, 17 May 2011 08:46:58 +1000
Quoted 8 lines · Michał Mirosław
Quoted 4 lines
In this case - no. Those messages inform that driver supports more feature combinations than network core and the feature set is reduced because of that.
The driver should never even claim to support SG/TSO/UFO if it does not support checksum. That is the point of the warning.
Well the check has to exist somewhere.
Currently userspace can configure tun/tap into whatever set of offloads it likes.
We're warning when the user asks for something that needs to be corrected. So the only thing you can suggest is to duplicate these changes in the tun/tap driver.
But if we do that, and error on bad combinations instead of fixing them up, we know from this discussion that existing virtualization setups and tools are going to stop working.
Quoted 6 lines · David Miller
Well the check has to exist somewhere.
Currently userspace can configure tun/tap into whatever set of offloads it likes.
Quoted 7 lines
We're warning when the user asks for something that needs to be corrected. So the only thing you can suggest is to duplicate these changes in the tun/tap driver.
But if we do that, and error on bad combinations instead of fixing them up, we know from this discussion that existing virtualization setups and tools are going to stop working.
Yeah the tun driver is simply busted. We should never have allowed user-space to tweak the feature bits like this. Instead they should have gone through the ethtool interface like everyone else, or at least use the same underlying calls as ethtool.
Actually, I think we can still do that, and apply the same rules as ethtool with respect to automatically turning things on/off.
AFAICS the current set_offload in tun.c does not call anything that verifies/fixes up the settings. If you change the feature bits after registering the tun device it may never get fixed up at all.
Allowing an unprivileged user to tweak feature bits directly with no verification is just wrong.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 20 lines · Herbert Xu
Quoted 6 lines · David Miller
Well the check has to exist somewhere.
Currently userspace can configure tun/tap into whatever set of offloads it likes.
Quoted 7 lines
We're warning when the user asks for something that needs to be corrected. So the only thing you can suggest is to duplicate these changes in the tun/tap driver.
But if we do that, and error on bad combinations instead of fixing them up, we know from this discussion that existing virtualization setups and tools are going to stop working.
Yeah the tun driver is simply busted. We should never have allowed user-space to tweak the feature bits like this. Instead they should have gone through the ethtool interface like everyone else, or at least use the same underlying calls as ethtool.
Quoted 7 lines
Actually, I think we can still do that, and apply the same rules as ethtool with respect to automatically turning things on/off.
AFAICS the current
set_offloadintun.cdoes not call anything that verifies/fixes up the settings. If you change the feature bits after registering the tun device it may never get fixed up at all.
Hmm, we get the warnings about bits dropped on each set_offload call:
netdev_update_features is called,
that calls netdev_fix_features
No?
Quoted 4 lines
Allowing an unprivileged user to tweak feature bits directly with no verification is just wrong.
Cheers,
But we do verify bits, and only allow the user
to tweak these ones:
#define TUN_USER_FEATURES (NETIF_F_HW_CSUM|NETIF_F_TSO_ECN|NETIF_F_TSO|
\
NETIF_F_TSO6|NETIF_F_UFO)No?
Quoted 4 lines
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 7 lines · Michael S. Tsirkin
Hmm, we get the warnings about bits dropped on each
set_offloadcall:
netdev_update_featuresis called,
that callsnetdev_fix_featuresNo?
I presume this has changed recently as the current Linus tree does not contain a call to netdev_update_features, it only calls netdev_features_change.
If so then this does ensure that the feature bits will be sane.
I still think the warnings should be retained at their previous level in the call paths other than set_offload. All we have to do is add a parameter to netdev_update_features and co.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 13 lines · Herbert Xu
Quoted 7 lines · Michael S. Tsirkin
Hmm, we get the warnings about bits dropped on each
set_offloadcall:
netdev_update_featuresis called,
that callsnetdev_fix_featuresNo?
I presume this has changed recently as the current Linus tree does not contain a call to
netdev_update_features, it only callsnetdev_features_change.
Quoted 7 lines
If so then this does ensure that the feature bits will be sane.
I still think the warnings should be retained at their previous level in the call paths other than
set_offload. All we have to do is add a parameter tonetdev_update_featuresand co.Cheers,
You mean like bool quiet?
Signature · 1 line
MST
Quoted 2 lines · Michael S. Tsirkin
You mean like bool quiet?
Yes when tun calls netdev_update_features we expect it to contain bogus combinations so all warnings should disappear.
Everybody should still need maintain sanity.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
2011/5/17 Herbert Xu <[email protected]>:
Quoted 6 lines · Michał Mirosław
Quoted 3 lines
In this case - no. Those messages inform that driver supports more feature combinations than network core and the feature set is reduced because of that.
The driver should never even claim to support SG/TSO/UFO if it does not support checksum. That is the point of the warning.
Not really. The dependency of SG on checksum is in network core code only, not in the hardware. For TSO/UFO they imply hw checksumming for the respective protocol, but still theres no point in duplicating features dependencies in driver code.
In the tun/tap case, by using TUNSETOFFLOAD userspace advertises that it is willing to receive unchecksummed/TSOed packets from kernel (it should use TUN_VNET_HDR for this to work correctly unless it doesn't forward the packets). After the conversion, those offloads can be disabled by using ethtool even if userspace does support them. Previously you were able to push offloaded packets to apllications not prepared for them.
Best Regards,
Michał Mirosław
Quoted 7 lines · W dniu 17 maja 2011 10:08 użytkownik Michał Mirosław <[email protected]> napisał:
In the tun/tap case, by using TUNSETOFFLOAD userspace advertises that it is willing to receive unchecksummed/TSOed packets from kernel (it should use
TUN_VNET_HDRfor this to work correctly unless it doesn't forward the packets). After the conversion, those offloads can be disabled by using ethtool even if userspace does support them. Previously you were able to push offloaded packets to apllications not prepared for them.
And there is a bug in the logic implementing this. I'll send a fix in a minute.
Best Regards,
Michał Mirosław
PATCH 1/1 net: tuntap: Fix tun_net_fix_features()
In reply to Michał Mirosławtun->set_features are meant to limit not force the features.
--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/net/tun.c +1 -1
@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }static const struct net_device_ops tun_netdev_ops = {
Signature · 1 line
1.7.2.5
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 22 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +1
-1@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }static const struct net_device_ops tun_netdev_ops = {Signature · 1 line
1.7.2.5
One thing that this will do though: previously, if ethtool disables offloads, then an application enables them, the application will have the last say.
With this patch, the most conservative approach wins. Right?
If we want to have the existing behaviour
I think the following would do this (untested). What do you think?
drivers/net/tun.c +2 -0
@@ -1199,6 +1199,8 @@ static int set_offload(struct tun_struct *tun, unsigned long arg)return -EINVAL ; tun->set_features = features; + tun->dev->features &=TUN_USER_FEATURES ; + tun->dev->features |= (features &TUN_USER_FEATURES );netdev_update_features (tun->dev);return 0 ;
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 20 lines · Michael S. Tsirkin
Quoted 19 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +1
-1@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }
Quoted 9 lines
Quoted 3 lines
drivers/net/tun.c +0
-0static const struct net_device_ops tun_netdev_ops = {Signature · 1 line
1.7.2.5One thing that this will do though: previously, if ethtool disables offloads, then an application enables them, the application will have the last say. With this patch, the most conservative approach wins. Right?
Exactly.
On device creation, wanted_features default to all offloads enabled, so unless an admin changes the flags, the application controls what is enabled. This matters only when using persistent tun/tap and admin and user are two different people. If the admin is using queues and doesn't want to handle e.g. TSO packets (I'm not sure if they are properly accounted in all queuing disciplines), then the feature should not be enabled by user.
Quoted 8 lines
If we want to have the existing behaviour
I think the following would do this (untested). What do you think?drivers/net/tun.c +0
-0
drivers/net/tun.c static int set_offload(struct tun_struct *tun, unsigned long arg) · line 1204
Quoted 6 lines
drivers/net/tun.c +2
-0@@ -1199,6 +1199,8 @@ static int set_offload(struct tun_struct *tun, unsigned long arg)return -EINVAL ; tun->set_features = features; + tun->dev->features &=TUN_USER_FEATURES ; + tun->dev->features |= (features &TUN_USER_FEATURES );netdev_update_features (tun->dev);
tun->dev->features will be recalculated by netdev_update_features() anyway. For this to work as you described it would need to alter wanted_features. I don't like the idea that something other than one of ethtool_ops is changing this field, as it then becomes something else that what the admin wants (even if that is not what he gets).
Best Regards,
Michał Mirosław
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 62 lines · Michał Mirosław
Quoted 20 lines · Michael S. Tsirkin
Quoted 19 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +1
-1@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }Quoted 9 lines
Quoted 3 lines
drivers/net/tun.c +0
-0static const struct net_device_ops tun_netdev_ops = {Signature · 1 line
1.7.2.5One thing that this will do though: previously, if ethtool disables offloads, then an application enables them, the application will have the last say. With this patch, the most conservative approach wins. Right?
Exactly.
On device creation,
wanted_featuresdefault to all offloads enabled, so unless an admin changes the flags, the application controls what is enabled. This matters only when using persistent tun/tap and admin and user are two different people. If the admin is using queues and doesn't want to handle e.g. TSO packets (I'm not sure if they are properly accounted in all queuing disciplines), then the feature should not be enabled by user.Quoted 8 lines
If we want to have the existing behaviour
I think the following would do this (untested). What do you think?drivers/net/tun.c +0
-0drivers/net/tun.c static int set_offload(struct tun_struct *tun, unsigned long arg) · line 1204
Quoted 6 lines
drivers/net/tun.c +2
-0@@ -1199,6 +1199,8 @@ static int set_offload(struct tun_struct *tun, unsigned long arg)return -EINVAL ; tun->set_features = features; + tun->dev->features &=TUN_USER_FEATURES ; + tun->dev->features |= (features &TUN_USER_FEATURES );netdev_update_features (tun->dev);
tun->dev->featureswill be recalculated bynetdev_update_features()anyway. For this to work as you described it would need to alterwanted_features. I don't like the idea that something other than one ofethtool_opsis changing this field, as it then becomes something else that what the admin wants (even if that is not what he gets).
Quoted 2 lines
Best Regards,
Michał Mirosław
Yes, with virtualization admin and the app are two different people usually. The device doesn't have to be persistent though I think - what limits this to persistent devices?
I agree this behaviour seems more consistent, I just hope this change does not break any setups.
Signature · 1 line
MST
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Michał Mirosław
Quoted 2 lines · Michael S. Tsirkin
Quoted 1 line · Michał Mirosław
tun->set_featuresare meant to limit not force the features.
[...]
Quoted 15 lines
Quoted 15 lines
Quoted 5 lines
One thing that this will do though: previously, if ethtool disables offloads, then an application enables them, the application will have the last say. With this patch, the most conservative approach wins. Right?
Exactly.
On device creation,
wanted_featuresdefault to all offloads enabled, so unless an admin changes the flags, the application controls what is enabled. This matters only when using persistent tun/tap and admin and user are two different people. If the admin is using queues and doesn't want to handle e.g. TSO packets (I'm not sure if they are properly accounted in all queuing disciplines), then the feature should not be enabled by user.
[...]
Quoted 3 lines
Yes, with virtualization admin and the app are two different people usually. The device doesn't have to be persistent though I think - what limits this to persistent devices?
Hmm. Nothing really. I just forgot about the virtualization case. You usually will change the offloads just after device creation unless you're testing or debugging something.
Quoted 2 lines
I agree this behaviour seems more consistent, I just hope this change does not break any setups.
The only effect would be some performance drop on cases, where admin turned off the offloads and they stay like that regardless of what user part does.
Best Regards,
Michał Mirosław
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 21 lines · Michał Mirosław
Quoted 4 lines · Michael S. Tsirkin
Quoted 3 lines · Michał Mirosław
Quoted 2 lines · Michael S. Tsirkin
Quoted 1 line · Michał Mirosław
tun->set_featuresare meant to limit not force the features.[...]
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
One thing that this will do though: previously, if ethtool disables offloads, then an application enables them, the application will have the last say. With this patch, the most conservative approach wins. Right?
Exactly.
Quoted 7 lines
Quoted 7 lines
On device creation,
wanted_featuresdefault to all offloads enabled, so unless an admin changes the flags, the application controls what is enabled. This matters only when using persistent tun/tap and admin and user are two different people. If the admin is using queues and doesn't want to handle e.g. TSO packets (I'm not sure if they are properly accounted in all queuing disciplines), then the feature should not be enabled by user.
Quoted 8 lines
[...]
Quoted 3 lines
Yes, with virtualization admin and the app are two different people usually. The device doesn't have to be persistent though I think - what limits this to persistent devices?
Hmm. Nothing really. I just forgot about the virtualization case. You usually will change the offloads just after device creation unless you're testing or debugging something.
That's true. kvm invokes a user script after creating device but just before configuring it, if there might be a problem it's likely only because of something such a script might do (which used to be harmless). My gut feeling is this is unlikely.
Quoted 8 lines
Quoted 2 lines
I agree this behaviour seems more consistent, I just hope this change does not break any setups.
The only effect would be some performance drop on cases, where admin turned off the offloads and they stay like that regardless of what user part does.
Best Regards,
Michał Mirosław
The performance drop is actually quite drastic :), but yes it will keep going, which is a good thing.
Signature · 1 line
MST
Re: [PATCH] net: tuntap: Fix tun_net_fix_features()
In reply to Michał MirosławQuoted 3 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>
Not sure when I'll find the time to test this but
just looking at the code, the patch makes sense
to me. So:
We probably want this in -stable too?
Quoted 19 lines
--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +1
-1@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }static const struct net_device_ops tun_netdev_ops = {Signature · 1 line
1.7.2.5
PATCH 1/1 net: tuntap: Fix tun_net_fix_features()
In reply to Michał Mirosławtun->set_features are meant to limit not force the features.
--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/net/tun.c +1 -1
@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES ); }static const struct net_device_ops tun_netdev_ops = {
Signature · 1 line
1.7.2.5
Re: [RESENT PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 14 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +0
-0
drivers/net/tun.c static u32 tun_net_fix_features(struct net_device *dev, u32 features) · line 461
Quoted 5 lines
drivers/net/tun.c +1
-1@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev); -return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES );
(A & B) | (A & C) <=> A & (B | C)
This is a cosmetic change, not a fix. Maybe you think it is clearer but then you should say that in the commit message.
Ben.
Quoted 3 lines
drivers/net/tun.c +0
-0}static const struct net_device_ops tun_netdev_ops = {
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [RESENT PATCH] net: tuntap: Fix tun_net_fix_features()
Quoted 18 lines · Ben Hutchings
Quoted 17 lines · Michał Mirosław
tun->set_featuresare meant to limit not force the features.Signed-off-by Michał Mirosław <[email protected]>--- drivers/net/tun.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)drivers/net/tun.c +0
-0@@ -458,7 +458,7 @@ static u32 tun_net_fix_features(struct net_device *dev, u32 features) {struct tun_struct *tun =netdev_priv (dev);
Quoted 7 lines
drivers/net/tun.c static u32 tun_net_fix_features(struct net_device *dev, u32 features) · line 461
Quoted 2 lines
drivers/net/tun.c +1
-1-return (features & tun->set_features) | (features & ~TUN_USER_FEATURES ); +return features & (tun->set_features | ~TUN_USER_FEATURES );(A & B) | (A & C) <=> A & (B | C)
This is a cosmetic change, not a fix. Maybe you think it is clearer but then you should say that in the commit message.
Hmm. Noise through my braincells, I suppose. Let's just drop this patch.
Best Regards,
Michał Mirosław