ixgbe and UDP with zero checksum
Hi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.
In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.
I also verified it using testpmd (on 20.11 and using the latest sources):
# lshw -businfo -c network Bus info Device Class Description ======================================================= pci@0000:01:00.0 em1 network 82599ES 10-Gigabit SFI/SFP+ Network Connection pci@0000:01:00.1 em2 network 82599ES 10-Gigabit SFI/SFP+ Network Connection
testpmd -l 2,4 -w 0000:01:00.0 -- -i --port-topology=chained --enable-rx-cksum testpmd> show device info all
********************* Infos for device 0000:01:00.0 ********************* Bus name: pci
Driver name: net_ixgbe
Devargs:
Connect to socket: 0
Port id: 0
MAC address: EC:F4:BB:DB:FC:18
Device name: 0000:01:00.0
Device speed capability: 1 Gbps 10 Gbps
testpmd> set fwd rxonly
testpmd> set verbose 1
testpmd> start
and sending packets from a tester machine using scapy (w/ non zero and zero checksum):
sendp(Ether(src="ec:f4:bb:dc:09:d0",dst="ec:f4:bb:db:fc:18")/IP(src="192.168.30.200", dst="192.168.30.100")/UDP()/Raw("a"*100), iface="em1")
sendp(Ether(src="ec:f4:bb:dc:09:d0",dst="ec:f4:bb:db:fc:18")/IP(src="192.168.30.200", dst="192.168.30.100")/UDP(chksum=0)/Raw("a"*100), iface="em1")
the results are respectively:
port 0/queue 0: received 1 packets
src=EC:F4:BB:DC:09:D0 - dst=EC:F4:BB:DB:FC:18 - type=0x0800 - length=142 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4 L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x0
ol_flags: PKT_RX_L4_CKSUM_GOOD PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
port 0/queue 0: received 1 packets
src=EC:F4:BB:DC:09:D0 - dst=EC:F4:BB:DB:FC:18 - type=0x0800 - length=142 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4 L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x0
ol_flags: PKT_RX_L4_CKSUM_BAD PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
Considering that the checksum for UDP over IPv4 can be optionally disabled by the sender (although usually computed and transmitted), is this behavior expected?
For completeness sake, I found an old patch [1] in the ixgbe linux driver that seems to be related to what I'm seeing. The code changed a little, but it's still there.
[1] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/
Regards,
Paolo
Hi Paolo,
Quoted 14 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.
Quoted 8 lines
In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
+ /*
+ * 82599 errata, UDP frames with a 0 checksum can be marked as
+ * checksum errors.
+ */
+ if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) &&
+ (adapter->hw.mac.type == ixgbe_mac_82599EB))
+ return;
Quoted 3 lines
Regards,
Paolo
Quoted 21 lines · Haiyue
Hi Paolo,
Quoted 19 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.
Quoted 7 lines
Quoted 3 lines
[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Just wondering that it means here - there's a user of DPDK who has an ixgbe card exhibiting this behavior. What do DPDK engineers tell them to do? Change the hardware? Upgrade the firmware?
Quoted 11 lines
(quoted diff) +7
-0+ /* + * 82599 errata, UDP frames with a 0 checksum can be marked as + * checksum errors. + */ + if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) && + (adapter->hw.mac.type == ixgbe_mac_82599EB)) + return;Quoted 4 lines
Regards,
Paolo
Quoted 21 lines · Haiyue
Hi Paolo,
Quoted 19 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.
Quoted 8 lines
Quoted 3 lines
[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.
I would refer to Aaron's questions because they bring up the point.
Quoted 11 lines
(quoted diff) +7
-0+ /* + * 82599 errata, UDP frames with a 0 checksum can be marked as + * checksum errors. + */ + if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) && + (adapter->hw.mac.type == ixgbe_mac_82599EB)) + return;Quoted 4 lines
Regards,
Paolo
Quoted 35 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 23 lines · Haiyue
Hi Paolo,
Quoted 21 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
Quoted 2 lines
Quoted 1 line
n/
Quoted 9 lines
Quoted 4 lines
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.
What's your PCI device ID ? My worked ixgbe:
86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
I'm wondering if people will complain that the patch will mark the real bad checksum UDP as GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...
Hi Paolo,
Quoted 48 lines
-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 7 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 26 lines
Quoted 23 lines · Haiyue
Hi Paolo,
Quoted 21 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
n/
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Quoted 7 lines
Quoted 4 lines
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.What's your PCI device ID ? My worked ixgbe:
Sorry, I missed the PKT_RX_L4_CKSUM_BAD information, yes, my NIC have the issue.
Quoted 4 lines
86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
I'm wondering if people will complain that the patch will mark the real bad checksum UDP as
Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.
Thanks for pointing it out.
Quoted 4 lines
GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...
Quoted 60 lines · Haiyue
Hi Paolo,
Quoted 48 lines
-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 7 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 26 lines
Quoted 23 lines · Haiyue
Hi Paolo,
Quoted 21 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
n/
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Quoted 7 lines
Quoted 4 lines
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.What's your PCI device ID ? My worked ixgbe:
Sorry, I missed the
PKT_RX_L4_CKSUM_BADinformation, yes, my NIC have the issue.
Quoted 9 lines
Quoted 4 lines
86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
I'm wondering if people will complain that the patch will mark the real bad checksum UDP as
Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.
Thanks for pointing it out.
ack, I'm going to file it.
Thanks,
Paolo
Quoted 4 lines
Quoted 4 lines
GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...
+ ARM experts, Feifei and Ruifeng.
Need your further support for ARM NEON path.
https://patchwork.dpdk.org/patch/87617/
Quoted 90 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Friday, January 29, 2021 22:20
To: Wang, Haiyue <[email protected]>; Yang, Qiming <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 43 lines · Haiyue
Hi Paolo,
Quoted 41 lines
-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 33 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 23 lines · Haiyue
Hi Paolo,
Quoted 21 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set toPKT_RX_L4_CKSUM_BAD.In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not tracked because of that.[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Quoted 1 line
n/
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.Quoted 6 lines
Quoted 2 lines
What's your PCI device ID ? My worked ixgbe:
Sorry, I missed the
PKT_RX_L4_CKSUM_BADinformation, yes, my NIC have the issue.Quoted 1 line
86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network
Connection
Quoted 8 lines
Quoted 3 lines
[8086:10fb] (rev 01)
I'm wondering if people will complain that the patch will mark the real bad checksum UDP as
Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.
Thanks for pointing it out.
ack, I'm going to file it.
Thanks,
PaoloQuoted 4 lines
Quoted 4 lines
GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...
Hi, Haiyue
We will release the patch to support zero checksum in arm neon path later.
Best Regards
Feifei
Quoted 119 lines
-----邮件原件-----
发件人: Wang, Haiyue <[email protected]>
发送时间: 2021年2月2日 15:42
收件人: Feifei Wang <[email protected]>; Ruifeng Wang <[email protected]>
抄送: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]; Yang, Qiming <[email protected]>; Paolo Valerio <[email protected]>
主题: RE: ixgbe and UDP with zero checksum+ ARM experts, Feifei and Ruifeng.
Need your further support for ARM NEON path.
https://patchwork.dpdk.org/patch/87617/
Quoted 9 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Friday, January 29, 2021 22:20
To: Wang, Haiyue <[email protected]>; Yang, Qiming <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 37 lines
Quoted 35 lines · Haiyue
Hi Paolo,
Quoted 33 lines
-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole
<[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 25 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole
<[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksumQuoted 15 lines · Haiyue
Hi Paolo,
Quoted 13 lines
-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksumHi,
performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding
ol_flagfor the l4 checksum is set to
PKT_RX_L4_CKSUM_BAD.Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
In particular, this apparently has an impact on OvS using
ct()action where UDP packets with zero checksum are not trackedbecause of that.
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
[1]
https://patchwork.ozlabs.org/project/netdev/patch/20090724040031.30202
Quoted 17 lines
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
n/
About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(
Quoted 7 lines
Quoted 4 lines
Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.What's your PCI device ID ? My worked ixgbe:
Sorry, I missed the
PKT_RX_L4_CKSUM_BADinformation, yes, my NIChave the issue.
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network
Connection
Quoted 6 lines
Quoted 4 lines
[8086:10fb] (rev 01)
I'm wondering if people will complain that the patch will mark the real bad checksum UDP as
Zero checksum is more popular case, please file a bug on
https://bugs.dpdk.org/ to trace the fix.
Quoted 11 lines
Quoted 3 lines
Thanks for pointing it out.
ack, I'm going to file it.
Thanks,
PaoloQuoted 2 lines
Quoted 2 lines
GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but
this makes driver do the network stack things ...
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines