Lift UDP_SEGMENT restriction for egress via device w/o csum offload
Subsystems (4)
v2 cover Lift UDP_SEGMENT restriction for egress via device w/o csum offload
This is a follow-up to an earlier question [1] if we can make UDP GSO work with any egress device, even those with no checksum offload capability. That's the default setup for TUN/TAP.
Because there is a change in behavior - sendmsg() does no longer return EIO error - I'm submitting through net-next tree, rather than net, as per Willem's advice.
[1] https://lore.kernel.org/netdev/[email protected]/
Changes in v2:
- Fixup ip link arguments order (Jakub)
- Describe performance impact compared to regular sendmsg (Willem)
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Jakub Sitnicki (2):
udp: Allow GSO transmit from devices with no checksum offload selftests/net: Add test coverage for UDP GSO software fallback
net/ipv4/udp.c | 3 +-- net/ipv4/udp_offload.c | 8 +++++++ net/ipv6/udp.c | 3 +-- tools/testing/selftests/net/udpgso.c | 15 +++++++++--- tools/testing/selftests/net/udpgso.sh | 43 +++++++++++++++++++++++++++++++++++ 5 files changed, 65 insertions(+), 7 deletions(-)
PATCH v2 1/2 udp: Allow GSO transmit from devices with no checksum offload
Today sending a UDP GSO packet from a TUN device results in an EIO error:
import fcntl, os, struct
from socket import *
TUNSETIFF = 0x400454CA
IFF_TUN = 0x0001
IFF_NO_PI = 0x1000
UDP_SEGMENT = 103
tun_fd = os.open("/dev/net/tun", os.O_RDWR) ifr = struct.pack("16sH", b"tun0", IFF_TUN | IFF_NO_PI) fcntl.ioctl(tun_fd, TUNSETIFF, ifr)
os.system("ip addr add 192.0.2.1/24 dev tun0")
os.system("ip link set dev tun0 up")
s = socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_UDP, UDP_SEGMENT, 1200)
s.sendto(b"x" * 3000, ("192.0.2.2", 9)) # EIO
This is due to a check in the udp stack if the egress device offers checksum offload. While TUN/TAP devices, by default, don't advertise this capability because it requires support from the TUN/TAP reader.
However, the GSO stack has a software fallback for checksum calculation, which we can use. This way we don't force UDP_SEGMENT users to handle the EIO error and implement a segmentation fallback.
Lift the restriction so that UDP_SEGMENT can be used with any egress device. We also need to adjust the UDP GSO code to match the GSO stack expectation about ip_summed field, as set in commit 8d63bee643f1 ("net:
avoid skb_warn_bad_offload false positives on UFO"). Otherwise we will hit the bad offload check.
Users should, however, expect a potential performance impact when batch-sending packets with UDP_SEGMENT without checksum offload on the egress device. In such case the packet payload is read twice: first during the sendmsg syscall when copying data from user memory, and then in the GSO stack for checksum computation. This double memory read can be less efficient than a regular sendmsg where the checksum is calculated during the initial data copy from user memory.
--- net/ipv4/udp.c | 3 +-- net/ipv4/udp_offload.c | 8 ++++++++ net/ipv6/udp.c | 3 +-- 3 files changed, 10 insertions(+), 4 deletions(-)
net/ipv4/udp.c +1 -2
@@ -938,8 +938,7 @@ static int udp_send_skb(struct sk_buff *skb, struct flowi4 *fl4,kfree_skb (skb);return -EINVAL ; } -if (skb->ip_summed !=CHECKSUM_PARTIAL || is_udplite || -dst_xfrm (skb_dst (skb))) { +if (is_udplite ||dst_xfrm (skb_dst (skb))) {kfree_skb (skb);return -EIO ; }
net/ipv4/udp_offload.c +8 -0
@@ -357,6 +357,14 @@ struct sk_buff *__udp_gso_segment(struct sk_buff *gso_skb,else uh->check =gso_make_checksum (seg, ~check) ? :CSUM_MANGLED_0 ; +/* On the TX path, CHECKSUM_NONE and CHECKSUM_UNNECESSARY have the same +* meaning. However, check for bad offloads in the GSO stack expects the +* latter, if the checksum was calculated in software. To vouch for the +* segment skbs we actually need to set it on the gso_skb. +*/ +if (gso_skb->ip_summed ==CHECKSUM_NONE ) + gso_skb->ip_summed =CHECKSUM_UNNECESSARY ; +/* update refcount for the packet */ if (copy_dtor) {int delta = sum_truesize - gso_skb->truesize;
net/ipv6/udp.c +1 -2
@@ -1257,8 +1257,7 @@ static int udp_v6_send_skb(struct sk_buff *skb, struct flowi6 *fl6,kfree_skb (skb);return -EINVAL ; } -if (skb->ip_summed !=CHECKSUM_PARTIAL || is_udplite || -dst_xfrm (skb_dst (skb))) { +if (is_udplite ||dst_xfrm (skb_dst (skb))) {kfree_skb (skb);return -EIO ; }
Signature · 1 line
2.40.1
Re: [PATCH net-next v2 1/2] udp: Allow GSO transmit from devices with no checksum offload
Quoted 43 lines · Jakub Sitnicki
Today sending a UDP GSO packet from a TUN device results in an EIO error:
import fcntl, os, struct
from socket import *TUNSETIFF =
0x400454CA
IFF_TUN=0x0001
IFF_NO_PI=0x1000
UDP_SEGMENT= 103
tun_fd=os.open("/dev/net/tun", os.O_RDWR)ifr =struct.pack("16sH", b"tun0", IFF_TUN | IFF_NO_PI)fcntl.ioctl(tun_fd, TUNSETIFF, ifr)
os.system("ip addr add 192.0.2.1/24 dev tun0")
os.system("ip link set dev tun0 up")s =
socket(AF_INET, SOCK_DGRAM)
s.setsockopt(SOL_UDP, UDP_SEGMENT, 1200)
s.sendto(b"x" * 3000, ("192.0.2.2", 9))# EIOThis is due to a check in the udp stack if the egress device offers checksum offload. While TUN/TAP devices, by default, don't advertise this capability because it requires support from the TUN/TAP reader.
However, the GSO stack has a software fallback for checksum calculation, which we can use. This way we don't force
UDP_SEGMENTusers to handle the EIO error and implement a segmentation fallback.Lift the restriction so that
UDP_SEGMENTcan be used with any egress device. We also need to adjust the UDP GSO code to match the GSO stack expectation aboutip_summedfield, as set in commit8d63bee643f1("net:
avoidskb_warn_bad_offloadfalse positives on UFO"). Otherwise we will hit the bad offload check.Users should, however, expect a potential performance impact when batch-sending packets with
UDP_SEGMENTwithout checksum offload on the egress device. In such case the packet payload is read twice: first during the sendmsg syscall when copying data from user memory, and then in the GSO stack for checksum computation. This double memory read can be less efficient than a regular sendmsg where the checksum is calculated during the initial data copy from user memory.
Quoted 1 line
Signed-off-by Jakub Sitnicki <[email protected]>
PATCH v2 2/2 selftests/net: Add test coverage for UDP GSO software fallback
In reply to Jakub SitnickiExtend the existing test to exercise UDP GSO egress through devices with various offload capabilities, including lack of checksum offload, which is the default case for TUN/TAP devices.
Test against a dummy device because it is simpler to set up then TUN/TAP.
--- tools/testing/selftests/net/udpgso.c | 15 +++++++++--- tools/testing/selftests/net/udpgso.sh | 43 +++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 3 deletions(-)
tools/testing/selftests/net/udpgso.c +12 -3
@@ -53,6 +53,7 @@ static bool cfg_do_ipv6;static bool cfg_do_connected;static bool cfg_do_connectionless;static bool cfg_do_msgmore; +static bool cfg_do_recv = true;static bool cfg_do_setsockopt;static int cfg_specific_test_id =-1 ;
@@ -414,6 +415,9 @@ static void run_one(struct testcase *test, int fdt, int fdr,if (!sent)return ; +if (!cfg_do_recv) +return ; +if (test->gso_len) mss = test->gso_len;else
@@ -464,8 +468,10 @@ static void run_test(struct sockaddr *addr, socklen_t alen)if (fdr ==-1 )error (1 , errno,"socket r" ); -if (bind (fdr, addr, alen)) -error (1 , errno,"bind" ); +if (cfg_do_recv) { +if (bind (fdr, addr, alen)) +error (1 , errno,"bind" ); + }/* Have tests fail quickly instead of hang */ if (setsockopt (fdr,SOL_SOCKET ,SO_RCVTIMEO , &tv,sizeof (tv)))
@@ -524,7 +530,7 @@ static void parse_opts(int argc, char **argv) {int c; -while ((c =getopt (argc, argv,"46cCmst:" )) !=-1 ) { +while ((c =getopt (argc, argv,"46cCmRst:" )) !=-1 ) {switch (c) {case '4' : cfg_do_ipv4 = true;
@@ -541,6 +547,9 @@ static void parse_opts(int argc, char **argv)case 'm' : cfg_do_msgmore = true;break ; +case 'R' : + cfg_do_recv = false; +break ;case 's' : cfg_do_setsockopt = true;break ;
tools/testing/selftests/net/udpgso.sh +43 -0
52 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH net-next v2 2/2] selftests/net: Add test coverage for UDP GSO software fallback
Quoted 7 lines · Jakub Sitnicki
Extend the existing test to exercise UDP GSO egress through devices with various offload capabilities, including lack of checksum offload, which is the default case for TUN/TAP devices.
Test against a dummy device because it is simpler to set up then TUN/TAP.
Signed-off-by Jakub Sitnicki <[email protected]>
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 9 lines · On Wed, 26 Jun 2024 19:51:25 +0200 you wrote:
This is a follow-up to an earlier question [1] if we can make UDP GSO work with any egress device, even those with no checksum offload capability. That's the default setup for TUN/TAP.
Because there is a change in behavior -
sendmsg()does no longer return EIO error - I'm submitting through net-next tree, rather than net, as per Willem's advice.[...]
Here is the summary with links:
- [net-next,v2,1/2] udp: Allow GSO transmit from devices with no checksum offload
https://git.kernel.org/netdev/net-next/c/10154dbded6d - [net-next,v2,2/2] selftests/net: Add test coverage for UDP GSO software fallback https://git.kernel.org/netdev/net-next/c/3e400219c04d
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html