Skip to contentSkip to conversation

xfrm: Don't clobber inner headers when already set

PATCH In review Cosmin Ratiu 2 participants 5 messages Apr 22 – Apr 29

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover xfrm: Don't clobber inner headers when already set

Commit [1] exposed an existing issue in xfrm which broke tunneling protocols over IPsec (e.g. VXLAN over IPsec).

This series fixes that and adds a test which exercises VXLAN over IPsec. For details please see the individual patches.

[1] commit 7fb4c1967011 ("net: pull headers in qdisc_pkt_len_segs_init()")

Cosmin Ratiu (3):
tools/selftests: Use a sensible timeout value for iperf3 client tools/selftests: Add a VXLAN+IPsec traffic test
xfrm: Don't clobber inner headers when already set

 net/xfrm/xfrm_output.c                        |  20 +-
 .../testing/selftests/drivers/net/hw/Makefile |   1 +
 tools/testing/selftests/drivers/net/hw/config |   5 +
 .../selftests/drivers/net/hw/ipsec_vxlan.py   | 204 ++++++++++++++++++
 .../selftests/drivers/net/lib/py/load.py      |   5 +-
 5 files changed, 228 insertions(+), 7 deletions(-)
 create mode 100755 tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/3 tools/selftests: Use a sensible timeout value for iperf3 client

The default timeout of cmd() is 5 seconds and Iperf3Runner requests the iperf3 client to run for 10 seconds, which clearly doesn't work since commit [1] enforced the timeout parameter.

Use a value derived from duration as timeout (+5 seconds for startup/teardown/various other overhead).

[1] commit f0bd19316663 ("selftests: net: fix timeout passed as positional argument to communicate()")

---
 tools/testing/selftests/drivers/net/lib/py/load.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/drivers/net/lib/py/load.py +4 -1
index f181fa2d38fc..e24660e5c27f 100644
@@ -48,7 +48,10 @@ class Iperf3Runner:
         Starts the iperf3 client with the configured options.
         """
         cmdline = self._build_client(streams, duration, reverse)
-        return cmd(cmdline, background=background, host=self.env.remote)
+        kwargs = {"background": background, "host": self.env.remote}
+        if not background:
+            kwargs["timeout"] = duration + 5
+        return cmd(cmdline, **kwargs)
 
     def measure_bandwidth(self, reverse=False):
         """
Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/3 tools/selftests: Add a VXLAN+IPsec traffic test

In reply to Cosmin Ratiu

There are VXLAN tests and IPsec tests, but there is no test that combines the two protocols and exercises the tunnel-over-ipsec code paths. Fix that by adding a traffic test with VXLAN and IPsec using crypto offload. This is runnable on HW which supports ESP offload (so no nsim unfortunately).

Traffic is done with iperf3 and the test validates that there are no packet drops and iperf3 can get to at least 100 Mbps (a very conservative value on today's crypto offload HW, as it can typically reach multi-Gbps rates).

Ran right now, the test fails due to a recently exposed bug in xfrm, which will be fixed in the next patch:

 # ./tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py
 TAP version 13
 1..4
 # Check| At ./tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py,
 # line 161, in test_vxlan_ipsec_crypto_offload:
 # Check|     ksft_eq(drops_after - drops_before, 0,
 # Check failed 189 != 0 TX drops during VXLAN+IPsec
 # Check| At ./tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py,
 # line 163, in test_vxlan_ipsec_crypto_offload:
 # Check|     ksft_ge(bw_gbps, 0.1,
 # Check failed 0.0015058278404812596 < 0.1 Minimum 100Mbps over
 # VXLAN+IPsec
 not ok 1 ipsec_vxlan.test_vxlan_ipsec_crypto_offload.outer_v4_inner_v4
 ...
---
 .../testing/selftests/drivers/net/hw/Makefile |   1 +
 tools/testing/selftests/drivers/net/hw/config |   5 +
 .../selftests/drivers/net/hw/ipsec_vxlan.py   | 204 ++++++++++++++++++
 3 files changed, 210 insertions(+)
 create mode 100755 tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py
tools/testing/selftests/drivers/net/hw/Makefile +1 -0
index 85ca4d1ecf9e..3b6ff4708005 100644
@@ -30,6 +30,7 @@ TEST_PROGS = \
 	gro_hw.py \
 	hw_stats_l3.sh \
 	hw_stats_l3_gre.sh \
+	ipsec_vxlan.py \
 	iou-zcrx.py \
 	irq.py \
 	loopback.sh \
tools/testing/selftests/drivers/net/hw/config +5 -0
index dd50cb8a7911..ae0168c2bbe6 100644
@@ -12,5 +12,10 @@ CONFIG_NET_IPGRE=y
 CONFIG_NET_IPGRE_DEMUX=y
 CONFIG_NETKIT=y
 CONFIG_NET_SCH_INGRESS=y
+CONFIG_INET6_ESP=y
+CONFIG_INET6_ESP_OFFLOAD=y
+CONFIG_INET_ESP=y
+CONFIG_INET_ESP_OFFLOAD=y
 CONFIG_UDMABUF=y
 CONFIG_VXLAN=y
+CONFIG_XFRM_USER=y
tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py new file +204 -0

204 lines not on this page · raw message

Signature · 1 line
2.53.0
#3 lore raw

PATCH 3/3 xfrm: Don't clobber inner headers when already set

In reply to Cosmin Ratiu

On VXLAN over IPsec egress, xfrm{4,6}_transport_output() blindly overwrite inner_transport_header (== the inner TCP header saved in VXLAN iptunnel_handle_offloads() -> skb_reset_inner_headers()) with the current transport_header (== the VXLAN outer UDP header set by udp_tunnel_xmit_skb()).

This was a latent bug, harmless until commit [1] added a doff validation check in qdisc_pkt_len_segs_init() for encapsulated GSO packets. With the wrong inner_transport_header set by xfrm, qdisc_pkt_len_segs_init() interprets inner_transport_header as a TCP header, reads doff=0 from the upper byte of the VNI and drops the packet with DROP_REASON_SKB_BAD_GSO.

Besides the use in GSO to determine the header size of segmented packets, inner_transport_header might be used by drivers to set up inner checksum offloading by pointing the HW to the inner transport header. A quick browse through available drivers shows that mlx5 uses skb->csum_start specifically for this scenario, while others either don't support VXLAN over IPsec crypto offload (ixgbe) or the HW is capable of parsing the packets itself (nfp, Chelsio).

But in all cases, it is more correct to let the inner_transport_header point to the innermost header instead of overwriting it in xfrm.

So fix this by guarding all four inner header save sites in xfrm_output.c (xfrm{4,6}_transport_output, xfrm{4,6}_tunnel_encap_add) with a check for skb->inner_protocol. When inner_protocol is set, a tunnel layer (VXLAN, Geneve, GRE, etc.) has already saved the correct inner header offsets and they must not be overwritten. When inner_protocol is zero, no prior tunnel encapsulation exists and xfrm must save the inner headers itself. The tunnel mode checks are only added for completion, since they aren't strictly required, as xfrm_output() forces software GSO in tunnel mode before encap.

This makes the previously added test pass:

 # ./tools/testing/selftests/drivers/net/hw/ipsec_vxlan.py
 TAP version 13
 1..4
 ok 1 ipsec_vxlan.test_vxlan_ipsec_crypto_offload.outer_v4_inner_v4
 ok 2 ipsec_vxlan.test_vxlan_ipsec_crypto_offload.outer_v4_inner_v6
 ok 3 ipsec_vxlan.test_vxlan_ipsec_crypto_offload.outer_v6_inner_v4
 ok 4 ipsec_vxlan.test_vxlan_ipsec_crypto_offload.outer_v6_inner_v6
 # Totals: pass:4 fail:0 xfail:0 xpass:0 skip:0 error:0

[1] commit 7fb4c1967011 ("net: pull headers in qdisc_pkt_len_segs_init()")

Fixes f1bd7d659ef0 ("xfrm: Add encapsulation header offsets while SKB is not encrypted")
---
 net/xfrm/xfrm_output.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
net/xfrm/xfrm_output.c +14 -6
index a9652b422f51..cc35c2fcbbe0 100644
@@ -66,7 +66,9 @@ static int xfrm4_transport_output(struct xfrm_state *x, struct sk_buff *skb)
 	struct iphdr *iph = ip_hdr(skb);
 	int ihl = iph->ihl * 4;
 
-	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	if (!skb->inner_protocol)
+		skb_set_inner_transport_header(skb,
+					       skb_transport_offset(skb));
 
 	skb_set_network_header(skb, -x->props.header_len);
 	skb->mac_header = skb->network_header +
@@ -167,7 +169,9 @@ static int xfrm6_transport_output(struct xfrm_state *x, struct sk_buff *skb)
 	int hdr_len;
 
 	iph = ipv6_hdr(skb);
-	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	if (!skb->inner_protocol)
+		skb_set_inner_transport_header(skb,
+					       skb_transport_offset(skb));
 
 	hdr_len = xfrm6_hdr_offset(x, skb, &prevhdr);
 	if (hdr_len < 0)
@@ -276,8 +280,10 @@ static int xfrm4_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
 	struct iphdr *top_iph;
 	int flags;
 
-	skb_set_inner_network_header(skb, skb_network_offset(skb));
-	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	if (!skb->inner_protocol) {
+		skb_set_inner_network_header(skb, skb_network_offset(skb));
+		skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	}
 
 	skb_set_network_header(skb, -x->props.header_len);
 	skb->mac_header = skb->network_header +
@@ -321,8 +327,10 @@ static int xfrm6_tunnel_encap_add(struct xfrm_state *x, struct sk_buff *skb)
 	struct ipv6hdr *top_iph;
 	int dsfield;
 
-	skb_set_inner_network_header(skb, skb_network_offset(skb));
-	skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	if (!skb->inner_protocol) {
+		skb_set_inner_network_header(skb, skb_network_offset(skb));
+		skb_set_inner_transport_header(skb, skb_transport_offset(skb));
+	}
 
 	skb_set_network_header(skb, -x->props.header_len);
 	skb->mac_header = skb->network_header +
Signature · 1 line
2.53.0
maintainer
#4 lore raw
In reply to Cosmin Ratiu
Quoted 6 lines · Cosmin Ratiu

Commit [1] exposed an existing issue in xfrm which broke tunneling protocols over IPsec (e.g. VXLAN over IPsec).

This series fixes that and adds a test which exercises VXLAN over IPsec. For details please see the individual patches.

Quoted 6 lines

[1] commit 7fb4c1967011 ("net: pull headers in qdisc_pkt_len_segs_init()")

Cosmin Ratiu (3):
tools/selftests: Use a sensible timeout value for iperf3 client tools/selftests: Add a VXLAN+IPsec traffic test
xfrm: Don't clobber inner headers when already set

Series applied, thanks a lot Cosmin!