myri10ge: LRO to GRO conversion

Hi,

The following patchset is a resubmission of '[PATCH resend net-next 0/3]
myri10ge: LRO to GRO conversion', and converts myri10ge from using the old inet_lro interface to GRO, and to do vlan tag decap in the driver so as to not suffer a performance penalty for vlan tagged traffic due to the conversion.

Changes this time are:

- Clean up some messy indenting & parens in
"2/3 myri10ge-Add-vlan-rx-for-better-GRO-perf"
and store the vlan hdr csum in a variable, so now we are not calling csum_partial() nested so deeply it spreads across 3 lines

- Folded 3/3 myri10ge-Use-skb_fill_page_desc into 1/3 myri10ge-Convert-from-LRO-to-GRO.patch since it is really just small part of the LRO removal cleanup. It was originally a separate patch because I noticed the cleanup at the last second, and was too lazy to fold it into the first patch where it belonged.

Note that a naive LRO->GRO conversion of myri10ge will result in a performance regression for vlan tagged frames. This is because myri10ge does not offer hardware vlan tag offload, and because GRO requires hardware vlan tag offload to aggregate vlan tagged frames.

To address this performance regression, I have implemented vlan tag popping in the myri10ge driver, as it seems to be the lesser of two evils. As [email protected] commented when I asked about this on
netdev: "Given GRO assumes NIC does hardware vlan offloading, I guess I would chose to do that. It seems unfortunate to add vlan decap in GRO path, already very complex."

Andrew Gallatin (2):
myri10ge: Convert from LRO to GRO
myri10ge: Add vlan rx for better GRO perf.

 drivers/net/ethernet/myricom/Kconfig             |    1 -
 drivers/net/ethernet/myricom/myri10ge/myri10ge.c |  275 
++++++----------------
 2 files changed, 74 insertions(+), 202 deletions(-)

From: Andrew Gallatin <[email protected]>
Date: Thu, 29 Nov 2012 16:07:15 -0500

Your email client is chopping up long lines, corrupting your patches.

Hi,

I'm resending this after fixing email client issues.

The following patchset is a resubmission of '[PATCH resend net-next 0/3]
myri10ge: LRO to GRO conversion', and converts myri10ge from using the old inet_lro interface to GRO, and to do vlan tag decap in the driver so as to not suffer a performance penalty for vlan tagged traffic due to the conversion.

Changes this time are:

- Clean up some messy indenting & parens in
"2/3 myri10ge-Add-vlan-rx-for-better-GRO-perf"
and store the vlan hdr csum in a variable, so now we are not calling csum_partial() nested so deeply it spreads across 3 lines

- Folded 3/3 myri10ge-Use-skb_fill_page_desc into 1/3 myri10ge-Convert-from-LRO-to-GRO.patch since it is really just small part of the LRO removal cleanup. It was originally a separate patch because I noticed the cleanup at the last second, and was too lazy to fold it into the first patch where it belonged.

Note that a naive LRO->GRO conversion of myri10ge will result in a performance regression for vlan tagged frames. This is because myri10ge does not offer hardware vlan tag offload, and because GRO requires hardware vlan tag offload to aggregate vlan tagged frames.

To address this performance regression, I have implemented vlan tag popping in the myri10ge driver, as it seems to be the lesser of two evils. As [email protected] commented when I asked about this on
netdev: "Given GRO assumes NIC does hardware vlan offloading, I guess I would chose to do that. It seems unfortunate to add vlan decap in GRO path, already very complex."

Andrew Gallatin (2):
myri10ge: Convert from LRO to GRO
myri10ge: Add vlan rx for better GRO perf.

 drivers/net/ethernet/myricom/Kconfig             |    1 -
 drivers/net/ethernet/myricom/myri10ge/myri10ge.c |  275 ++++++----------------
 2 files changed, 74 insertions(+), 202 deletions(-)

1.7.9.5

Convert myri10ge from LRO to GRO, and simplify the driver by removing various LRO-related code which is no longer needed including ndo_fix_features op, custom skb building from frags, and LRO header parsing.

---
 drivers/net/ethernet/myricom/Kconfig             |    1 -
 drivers/net/ethernet/myricom/myri10ge/myri10ge.c |  236 ++++------------------
 2 files changed, 34 insertions(+), 203 deletions(-)

index 540f0c6..3932d08 100644
@@ -23,7 +23,6 @@ config MYRI10GE
 	depends on PCI && INET
 	select FW_LOADER
 	select CRC32
-	select INET_LRO
 	---help---
 	  This driver supports Myricom Myri-10G Dual Protocol interface in
 	  Ethernet mode. If the eeprom on your board is not recent enough,

1.7.9.5

Unlike LRO, GRO requires that vlan tags be removed before aggregation can occur. Since the myri10ge NIC does not support hardware vlan tag offload, we must remove the tag in the driver to achieve performance comparable to LRO for vlan tagged frames.

Thanks to Eric Duzamet for his help simplifying the original patch.

---
 drivers/net/ethernet/myricom/myri10ge/myri10ge.c |   41 ++++++++++++++++++++++
 1 file changed, 41 insertions(+)

index 84207c0..2fc984a 100644
@@ -1264,6 +1264,42 @@ myri10ge_unmap_rx_page(struct pci_dev *pdev,
 	}
 }
 
+/*
+ * GRO does not support acceleration of tagged vlan frames, and
+ * this NIC does not support vlan tag offload, so we must pop
+ * the tag ourselves to be able to achieve GRO performance that
+ * is comparable to LRO.
+ */
+
+static inline void
+myri10ge_vlan_rx(struct net_device *dev, void *addr, struct sk_buff *skb)
+{
+	u8 *va;
+	struct vlan_ethhdr *veh;
+	struct skb_frag_struct *frag;
+	__wsum vsum;
+
+	va = addr;
+	va += MXGEFW_PAD;
+	veh = (struct vlan_ethhdr *)va;
+	if ((dev->features & NETIF_F_HW_VLAN_RX) == NETIF_F_HW_VLAN_RX &&
+	    veh->h_vlan_proto == ntohs(ETH_P_8021Q)) {
+		/* fixup csum if needed */
+		if (skb->ip_summed == CHECKSUM_COMPLETE) {
+			vsum = csum_partial(va + ETH_HLEN, VLAN_HLEN, 0);
+			skb->csum = csum_sub(skb->csum, vsum);
+		}
+		/* pop tag */
+		__vlan_hwaccel_put_tag(skb, ntohs(veh->h_vlan_TCI));
+		memmove(va + VLAN_HLEN, va, 2 * ETH_ALEN);
+		skb->len -= VLAN_HLEN;
+		skb->data_len -= VLAN_HLEN;
+		frag = skb_shinfo(skb)->frags;
+		frag->page_offset += VLAN_HLEN;
+		skb_frag_size_set(frag, skb_frag_size(frag) - VLAN_HLEN);
+	}
+}
+
 static inline int
 myri10ge_rx_done(struct myri10ge_slice_state *ss, int len, __wsum csum)
 {

@@ -1326,6 +1362,7 @@ myri10ge_rx_done(struct myri10ge_slice_state *ss, int len, __wsum csum)
 		skb->ip_summed = CHECKSUM_COMPLETE;
 		skb->csum = csum;
 	}
+	myri10ge_vlan_rx(mgp->dev, va, skb);
 	skb_record_rx_queue(skb, ss - &mgp->ss[0]);
 
 	napi_gro_frags(&ss->napi);

@@ -3851,6 +3888,10 @@ static int myri10ge_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 	netdev->netdev_ops = &myri10ge_netdev_ops;
 	netdev->mtu = myri10ge_initial_mtu;
 	netdev->hw_features = mgp->features | NETIF_F_RXCSUM;
+
+	/* fake NETIF_F_HW_VLAN_RX for good GRO performance */
+	netdev->hw_features |= NETIF_F_HW_VLAN_RX;
+
 	netdev->features = netdev->hw_features;
 
 	if (dac_enabled)

1.7.9.5

From: Andrew Gallatin <[email protected]>
Date: Fri, 30 Nov 2012 13:31:57 -0500

Applied, thanks.

Thank you.

I also want to apologize for making so many beginner mistakes, and to thank you for your patience with me. There has usually been somebody more "plugged in" to the linux community that has submitted patches for us in the past, so I'm just not used to doing it yet.

Thanks again,

Drew