Skip to contentSkip to conversation

ixgbe and UDP with zero checksum

dpdk-dev Paolo Valerio 4 participants 9 messages Jan 27, 2021 – Feb 2, 2021

lore
Share & tools
lore
#0 lore raw

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

I also verified it using testpmd (on 20.11 and using the latest sources):

# lshw -businfo -c network
Bus info          Device     Class          Description
=======================================================
pci@0000:01:00.0  em1        network        82599ES 10-Gigabit SFI/SFP+ Network Connection
pci@0000:01:00.1  em2        network        82599ES 10-Gigabit SFI/SFP+ Network Connection

testpmd -l 2,4 -w 0000:01:00.0 -- -i --port-topology=chained --enable-rx-cksum testpmd> show device info all

********************* Infos for device 0000:01:00.0 ********************* Bus name: pci
Driver name: net_ixgbe
Devargs:
Connect to socket: 0

Port id: 0
MAC address: EC:F4:BB:DB:FC:18
Device name: 0000:01:00.0
Device speed capability: 1 Gbps 10 Gbps

testpmd> set fwd rxonly
testpmd> set verbose 1
testpmd> start

and sending packets from a tester machine using scapy (w/ non zero and zero checksum):

sendp(Ether(src="ec:f4:bb:dc:09:d0",dst="ec:f4:bb:db:fc:18")/IP(src="192.168.30.200", dst="192.168.30.100")/UDP()/Raw("a"*100), iface="em1")

sendp(Ether(src="ec:f4:bb:dc:09:d0",dst="ec:f4:bb:db:fc:18")/IP(src="192.168.30.200", dst="192.168.30.100")/UDP(chksum=0)/Raw("a"*100), iface="em1")

the results are respectively:

port 0/queue 0: received 1 packets
src=EC:F4:BB:DC:09:D0 - dst=EC:F4:BB:DB:FC:18 - type=0x0800 - length=142 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4 L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x0
ol_flags: PKT_RX_L4_CKSUM_GOOD PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN

port 0/queue 0: received 1 packets
src=EC:F4:BB:DC:09:D0 - dst=EC:F4:BB:DB:FC:18 - type=0x0800 - length=142 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV4 L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Receive queue=0x0
ol_flags: PKT_RX_L4_CKSUM_BAD PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN

Considering that the checksum for UDP over IPv4 can be optionally disabled by the sender (although usually computed and transmitted), is this behavior expected?

For completeness sake, I found an old patch [1] in the ixgbe linux driver that seems to be related to what I'm seeing. The code changed a little, but it's still there.

[1] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/

Regards,
Paolo

#1 lore raw

Hi Paolo,

Quoted 14 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

Quoted 8 lines

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

+ /*
+ * 82599 errata, UDP frames with a 0 checksum can be marked as
+ * checksum errors.
+ */
+ if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) &&
+ (adapter->hw.mac.type == ixgbe_mac_82599EB))
+ return;

Quoted 3 lines

Regards,
Paolo

#2 lore raw
Quoted 21 lines · Haiyue

Hi Paolo,

Quoted 19 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

Quoted 7 lines
Quoted 3 lines

[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Just wondering that it means here - there's a user of DPDK who has an ixgbe card exhibiting this behavior. What do DPDK engineers tell them to do? Change the hardware? Upgrade the firmware?

Quoted 11 lines
(quoted diff) +7 -0
+		/*
+		 * 82599 errata, UDP frames with a 0 checksum can be marked as
+		 * checksum errors.
+		 */
+		if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) &&
+		    (adapter->hw.mac.type == ixgbe_mac_82599EB))
+			return;
Quoted 4 lines

Regards,
Paolo

#3 lore raw
In reply to Wang, Haiyue
Quoted 21 lines · Haiyue

Hi Paolo,

Quoted 19 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

Quoted 8 lines
Quoted 3 lines

[1]
https://patchwork.ozlabs.org/project/netdev/patch/[email protected]
n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

I would refer to Aaron's questions because they bring up the point.

Quoted 11 lines
(quoted diff) +7 -0
+		/*
+		 * 82599 errata, UDP frames with a 0 checksum can be marked as
+		 * checksum errors.
+		 */
+		if ((pkt_info & IXGBE_RXDADV_PKTTYPE_UDP) &&
+		    (adapter->hw.mac.type == ixgbe_mac_82599EB))
+			return;
Quoted 4 lines

Regards,
Paolo

#4 lore raw
Quoted 35 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 23 lines · Haiyue

Hi Paolo,

Quoted 21 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

[1]

https://patchwork.ozlabs.org/project/netdev/patch/[email protected]

Quoted 2 lines
Quoted 1 line

n/

Quoted 9 lines
Quoted 4 lines

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

What's your PCI device ID ? My worked ixgbe:

86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

I'm wondering if people will complain that the patch will mark the real bad checksum UDP as GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...

#5 lore raw

Hi Paolo,

Quoted 48 lines

-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 7 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 26 lines
Quoted 23 lines · Haiyue

Hi Paolo,

Quoted 21 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

[1]

https://patchwork.ozlabs.org/project/netdev/patch/[email protected]

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Quoted 7 lines
Quoted 4 lines

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

What's your PCI device ID ? My worked ixgbe:

Sorry, I missed the PKT_RX_L4_CKSUM_BAD information, yes, my NIC have the issue.

Quoted 4 lines

86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

I'm wondering if people will complain that the patch will mark the real bad checksum UDP as

Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.

Thanks for pointing it out.

Quoted 4 lines

GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...

#6 lore raw
Quoted 60 lines · Haiyue

Hi Paolo,

Quoted 48 lines

-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 7 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 26 lines
Quoted 23 lines · Haiyue

Hi Paolo,

Quoted 21 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

[1]

https://patchwork.ozlabs.org/project/netdev/patch/[email protected]

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Quoted 7 lines
Quoted 4 lines

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

What's your PCI device ID ? My worked ixgbe:

Sorry, I missed the PKT_RX_L4_CKSUM_BAD information, yes, my NIC have the issue.

Quoted 9 lines
Quoted 4 lines

86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

I'm wondering if people will complain that the patch will mark the real bad checksum UDP as

Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.

Thanks for pointing it out.

ack, I'm going to file it.

Thanks,
Paolo

Quoted 4 lines
Quoted 4 lines

GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...

#7 lore raw

+ ARM experts, Feifei and Ruifeng.

Need your further support for ARM NEON path.

https://patchwork.dpdk.org/patch/87617/

Quoted 90 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Friday, January 29, 2021 22:20
To: Wang, Haiyue <[email protected]>; Yang, Qiming <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 43 lines · Haiyue

Hi Paolo,

Quoted 41 lines

-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 33 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 23 lines · Haiyue

Hi Paolo,

Quoted 21 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to PKT_RX_L4_CKSUM_BAD.

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked because of that.

[1]

https://patchwork.ozlabs.org/project/netdev/patch/[email protected]

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Quoted 1 line

n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

Quoted 6 lines
Quoted 2 lines

What's your PCI device ID ? My worked ixgbe:

Sorry, I missed the PKT_RX_L4_CKSUM_BAD information, yes, my NIC have the issue.

Quoted 1 line

86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network

Connection

Quoted 8 lines
Quoted 3 lines

[8086:10fb] (rev 01)

I'm wondering if people will complain that the patch will mark the real bad checksum UDP as

Zero checksum is more popular case, please file a bug on https://bugs.dpdk.org/ to trace the fix.

Thanks for pointing it out.

ack, I'm going to file it.

Thanks,
Paolo

Quoted 4 lines
Quoted 4 lines

GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but this makes driver do the network stack things ...

#8 lore raw

Hi, Haiyue

We will release the patch to support zero checksum in arm neon path later.

Best Regards
Feifei

Quoted 119 lines

-----邮件原件-----
发件人: Wang, Haiyue <[email protected]>
发送时间: 2021年2月2日 15:42
收件人: Feifei Wang <[email protected]>; Ruifeng Wang <[email protected]>
抄送: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]; Yang, Qiming <[email protected]>; Paolo Valerio <[email protected]>
主题: RE: ixgbe and UDP with zero checksum

+ ARM experts, Feifei and Ruifeng.

Need your further support for ARM NEON path.

https://patchwork.dpdk.org/patch/87617/

Quoted 9 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Friday, January 29, 2021 22:20
To: Wang, Haiyue <[email protected]>; Yang, Qiming <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole <[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 37 lines
Quoted 35 lines · Haiyue

Hi Paolo,

Quoted 33 lines

-----Original Message-----
From: Wang, Haiyue
Sent: Friday, January 29, 2021 10:02
To: Paolo Valerio <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole
<[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 25 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Thursday, January 28, 2021 05:35
To: Wang, Haiyue <[email protected]>
Cc: Guo, Jia <[email protected]>; Aaron Conole
<[email protected]>; [email protected]
Subject: RE: ixgbe and UDP with zero checksum

Quoted 15 lines · Haiyue

Hi Paolo,

Quoted 13 lines

-----Original Message-----
From: Paolo Valerio <[email protected]>
Sent: Wednesday, January 27, 2021 21:50
To: [email protected]
Cc: Guo, Jia <[email protected]>; Wang, Haiyue <[email protected]>; Aaron Conole <[email protected]>
Subject: ixgbe and UDP with zero checksum

Hi,

performing some tests, I noticed that on ixgbe when receiving UDP packets with zero checksum (no checksum) over IPv4, the corresponding ol_flag for the l4 checksum is set to

PKT_RX_L4_CKSUM_BAD.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

In particular, this apparently has an impact on OvS using ct() action where UDP packets with zero checksum are not tracked

because of that.

Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

[1]

https://patchwork.ozlabs.org/project/netdev/patch/20090724040031.30202

Quoted 17 lines

[email protected]

Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 1 line

n/

About 12 years old patch, it is hardware errata. For fixing this, have to always disable vector Rx path for 82599, it seems not a good idea to bring in this workaround. :(

Quoted 7 lines
Quoted 4 lines

Thanks for the answer.
Yes, as I mentioned, the patch is old although still meaningful. I linked it mostly because it mentions the hw errata.

What's your PCI device ID ? My worked ixgbe:

Sorry, I missed the PKT_RX_L4_CKSUM_BAD information, yes, my NIC

have the issue.

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

86:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network

Connection

Quoted 6 lines
Quoted 4 lines

[8086:10fb] (rev 01)

I'm wondering if people will complain that the patch will mark the real bad checksum UDP as

Zero checksum is more popular case, please file a bug on

https://bugs.dpdk.org/ to trace the fix.

Quoted 11 lines
Quoted 3 lines

Thanks for pointing it out.

ack, I'm going to file it.

Thanks,
Paolo

Quoted 2 lines
Quoted 2 lines

GOOD. For handling this correctly, looks like driver needs to check the UDP's checksum value, if zero, then skip the error information, but

this makes driver do the network stack things ...

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines