Skip to contentSkip to conversation

Support IPsec crypto offload for IPv6 ESP and IPv4 UDP-encapsulated ESP data paths

PATCH In review Mike Yu 2 participants 6 messages Jul 12, 2024 – Jul 13, 2024

Subsystems (3)
lore
Share & tools
lore

Mike Yu

author
#0 lore raw

v4 cover Support IPsec crypto offload for IPv6 ESP and IPv4 UDP-encapsulated ESP data paths

Currently, IPsec crypto offload is enabled for GRO code path. However, there are other code paths where the XFRM stack is involved; for example, IPv6 ESP packets handled by xfrm6_esp_rcv() in ESP layer, and IPv4 UDP-encapsulated ESP packets handled by udp_rcv() in UDP layer.

This patchset extends the crypto offload support to cover these two cases. This is useful for devices with traffic accounting (e.g., Android), where GRO can lead to inaccurate accounting on the underlying network. For example, VPN traffic might not be counted on the wifi network interface wlan0 if the packets are handled in GRO code path before entering the network stack for accounting.

Below is the RX data path scenario the crypto offload can be applied to.

  +-----------+   +-------+
  | HW Driver |-->| wlan0 |--------+
  +-----------+   +-------+        |
                                   v
                             +---------------+   +------+
                     +------>| Network Stack |-->| Apps |
                     |       +---------------+   +------+
                     |             |
                     |             v
                 +--------+   +------------+
                 | ipsec1 |<--| XFRM Stack |
                 +--------+   +------------+

v3 -> v4:
- Change the target tree to ipsec-next.
v2 -> v3:
- Correct ESP seq in esp_xmit().
v1 -> v2:
- Fix comment style.

Mike Yu (4):
xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO
path
xfrm: Allow UDP encapsulation in crypto offload control path
xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP
packet
xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP
packet

 net/ipv4/esp4.c         |  8 +++++++-
 net/ipv4/esp4_offload.c | 17 ++++++++++++++++-
 net/xfrm/xfrm_device.c  |  6 +++---
 net/xfrm/xfrm_input.c   |  3 ++-
 net/xfrm/xfrm_policy.c  |  5 ++++-
 5 files changed, 32 insertions(+), 7 deletions(-)
Signature · 1 line
2.45.2.993.g49e7a77208-goog
author
#1 lore raw

PATCH v4 1/4 xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO path

IPsec crypt offload supports outbound IPv6 ESP packets, but it doesn't support inbound IPv6 ESP packets.

This change enables the crypto offload for inbound IPv6 ESP packets that are not handled through GRO code path. If HW drivers add the offload information to the skb, the packet will be handled in the crypto offload rx code path.

Apart from the change in crypto offload rx code path, the change in xfrm_policy_check is also needed.

Exampe of RX data path:

  +-----------+   +-------+
  | HW Driver |-->| wlan0 |--------+
  +-----------+   +-------+        |
                                   v
                             +---------------+   +------+
                     +------>| Network Stack |-->| Apps |
                     |       +---------------+   +------+
                     |             |
                     |             v
                 +--------+   +------------+
                 | ipsec1 |<--| XFRM Stack |
                 +--------+   +------------+

Test: Enabled both in/out IPsec crypto offload, and verified IPv6
ESP packets on Android device on both wifi/cellular network

---
 net/xfrm/xfrm_input.c  | 2 +-
 net/xfrm/xfrm_policy.c | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)
net/xfrm/xfrm_input.c +1 -1
index d2ea18dcb0cb..ba8deb0235ba 100644
@@ -471,7 +471,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 	struct xfrm_offload *xo = xfrm_offload(skb);
 	struct sec_path *sp;
 
-	if (encap_type < 0 || (xo && xo->flags & XFRM_GRO)) {
+	if (encap_type < 0 || (xo && (xo->flags & XFRM_GRO || encap_type == 0))) {
 		x = xfrm_input_state(skb);
 
 		if (unlikely(x->dir && x->dir != XFRM_SA_DIR_IN)) {
net/xfrm/xfrm_policy.c +4 -1
index 6603d3bd171f..2a9a31f2a9c1 100644
@@ -3718,12 +3718,15 @@ int __xfrm_policy_check(struct sock *sk, int dir, struct sk_buff *skb,
 		pol = xfrm_in_fwd_icmp(skb, &fl, family, if_id);
 
 	if (!pol) {
+		const bool is_crypto_offload = sp &&
+			(xfrm_input_state(skb)->xso.type == XFRM_DEV_OFFLOAD_CRYPTO);
+
 		if (net->xfrm.policy_default[dir] == XFRM_USERPOLICY_BLOCK) {
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMINNOPOLS);
 			return 0;
 		}
 
-		if (sp && secpath_has_nontransport(sp, 0, &xerr_idx)) {
+		if (sp && secpath_has_nontransport(sp, 0, &xerr_idx) && !is_crypto_offload) {
 			xfrm_secpath_reject(xerr_idx, skb, &fl);
 			XFRM_INC_STATS(net, LINUX_MIB_XFRMINNOPOLS);
 			return 0;
Signature · 1 line
2.45.2.993.g49e7a77208-goog
author
#2 lore raw

PATCH v4 2/4 xfrm: Allow UDP encapsulation in crypto offload control path

In reply to Mike Yu

Unblock this limitation so that SAs with encapsulation specified can be passed to HW drivers. HW drivers can still reject the SA in their implementation of xdo_dev_state_add if the encapsulation is not supported.

Test: Verified on Android device

---
 net/xfrm/xfrm_device.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/xfrm/xfrm_device.c +3 -3
index 2455a76a1cff..9a44d363ba62 100644
@@ -261,9 +261,9 @@ int xfrm_dev_state_add(struct net *net, struct xfrm_state *x,
 
 	is_packet_offload = xuo->flags & XFRM_OFFLOAD_PACKET;
 
-	/* We don't yet support UDP encapsulation and TFC padding. */
-	if ((!is_packet_offload && x->encap) || x->tfcpad) {
-		NL_SET_ERR_MSG(extack, "Encapsulation and TFC padding can't be offloaded");
+	/* We don't yet support TFC padding. */
+	if (x->tfcpad) {
+		NL_SET_ERR_MSG(extack, "TFC padding can't be offloaded");
 		return -EINVAL;
 	}
 
Signature · 1 line
2.45.2.993.g49e7a77208-goog
author
#3 lore raw

PATCH v4 3/4 xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP packet

In reply to Mike Yu

If xfrm_input() is called with UDP_ENCAP_ESPINUDP, the packet is already processed in UDP layer that removes the UDP header. Therefore, there should be no much difference to treat it as an ESP packet in the XFRM stack.

Test: Enabled dir=in IPsec crypto offload, and verified IPv4
UDP-encapsulated ESP packets on both wifi/cellular network

---
 net/xfrm/xfrm_input.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
net/xfrm/xfrm_input.c +2 -1
index ba8deb0235ba..7cee9c0a2cdc 100644
@@ -471,7 +471,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 	struct xfrm_offload *xo = xfrm_offload(skb);
 	struct sec_path *sp;
 
-	if (encap_type < 0 || (xo && (xo->flags & XFRM_GRO || encap_type == 0))) {
+	if (encap_type < 0 || (xo && (xo->flags & XFRM_GRO || encap_type == 0 ||
+				      encap_type == UDP_ENCAP_ESPINUDP))) {
 		x = xfrm_input_state(skb);
 
 		if (unlikely(x->dir && x->dir != XFRM_SA_DIR_IN)) {
Signature · 1 line
2.45.2.993.g49e7a77208-goog
author
#4 lore raw

PATCH v4 4/4 xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP packet

In reply to Mike Yu

esp_xmit() is already able to handle UDP encapsulation through the call to esp_output_head(). However, the ESP header and the outer IP header are not correct and need to be corrected.

Test: Enabled both dir=in/out IPsec crypto offload, and verified IPv4
UDP-encapsulated ESP packets on both wifi/cellular network

---
v2->v3: https://lore.kernel.org/all/[email protected]
- Correct ESP seq in esp_xmit().
v1->v2: https://lore.kernel.org/all/[email protected]
- Fix comment style.

---
 net/ipv4/esp4.c         |  8 +++++++-
 net/ipv4/esp4_offload.c | 17 ++++++++++++++++-
 2 files changed, 23 insertions(+), 2 deletions(-)
net/ipv4/esp4.c +7 -1
index 3968d3f98e08..73981595f062 100644
@@ -349,6 +349,7 @@ static struct ip_esp_hdr *esp_output_udp_encap(struct sk_buff *skb,
 {
 	struct udphdr *uh;
 	unsigned int len;
+	struct xfrm_offload *xo = xfrm_offload(skb);
 
 	len = skb->len + esp->tailen - skb_transport_offset(skb);
 	if (len + sizeof(struct iphdr) > IP_MAX_MTU)
@@ -360,7 +361,12 @@ static struct ip_esp_hdr *esp_output_udp_encap(struct sk_buff *skb,
 	uh->len = htons(len);
 	uh->check = 0;
 
-	*skb_mac_header(skb) = IPPROTO_UDP;
+	/* For IPv4 ESP with UDP encapsulation, if xo is not null, the skb is in the crypto offload
+	 * data path, which means that esp_output_udp_encap is called outside of the XFRM stack.
+	 * In this case, the mac header doesn't point to the IPv4 protocol field, so don't set it.
+	 */
+	if (!xo || encap_type != UDP_ENCAP_ESPINUDP)
+		*skb_mac_header(skb) = IPPROTO_UDP;
 
 	return (struct ip_esp_hdr *)(uh + 1);
 }
net/ipv4/esp4_offload.c +16 -1
index b3271957ad9a..a37d18858c72 100644
@@ -264,6 +264,7 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb,  netdev_features_
 	struct esp_info esp;
 	bool hw_offload = true;
 	__u32 seq;
+	int encap_type = 0;
 
 	esp.inplace = true;
 
@@ -296,8 +297,10 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb,  netdev_features_
 
 	esp.esph = ip_esp_hdr(skb);
 
+	if (x->encap)
+		encap_type = x->encap->encap_type;
 
-	if (!hw_offload || !skb_is_gso(skb)) {
+	if (!hw_offload || !skb_is_gso(skb) || (hw_offload && encap_type == UDP_ENCAP_ESPINUDP)) {
 		esp.nfrags = esp_output_head(x, skb, &esp);
 		if (esp.nfrags < 0)
 			return esp.nfrags;
@@ -324,6 +327,18 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb,  netdev_features_
 
 	esp.seqno = cpu_to_be64(seq + ((u64)xo->seq.hi << 32));
 
+	if (hw_offload && encap_type == UDP_ENCAP_ESPINUDP) {
+		/* In the XFRM stack, the encapsulation protocol is set to iphdr->protocol by
+		 * setting *skb_mac_header(skb) (see esp_output_udp_encap()) where skb->mac_header
+		 * points to iphdr->protocol (see xfrm4_tunnel_encap_add()).
+		 * However, in esp_xmit(), skb->mac_header doesn't point to iphdr->protocol.
+		 * Therefore, the protocol field needs to be corrected.
+		 */
+		ip_hdr(skb)->protocol = IPPROTO_UDP;
+
+		esph->seq_no = htonl(seq);
+	}
+
 	ip_hdr(skb)->tot_len = htons(skb->len);
 	ip_send_check(ip_hdr(skb));
 
Signature · 1 line
2.45.2.993.g49e7a77208-goog
maintainer
#5 lore raw
In reply to Mike Yu
Quoted 33 lines · Mike Yu

Currently, IPsec crypto offload is enabled for GRO code path. However, there are other code paths where the XFRM stack is involved; for example, IPv6 ESP packets handled by xfrm6_esp_rcv() in ESP layer, and IPv4 UDP-encapsulated ESP packets handled by udp_rcv() in UDP layer.

This patchset extends the crypto offload support to cover these two cases. This is useful for devices with traffic accounting (e.g., Android), where GRO can lead to inaccurate accounting on the underlying network. For example, VPN traffic might not be counted on the wifi network interface wlan0 if the packets are handled in GRO code path before entering the network stack for accounting.

Below is the RX data path scenario the crypto offload can be applied to.

  +-----------+   +-------+
  | HW Driver |-->| wlan0 |--------+
  +-----------+   +-------+        |
                                   v
                             +---------------+   +------+
                     +------>| Network Stack |-->| Apps |
                     |       +---------------+   +------+
                     |             |
                     |             v
                 +--------+   +------------+
                 | ipsec1 |<--| XFRM Stack |
                 +--------+   +------------+

v3 -> v4:
- Change the target tree to ipsec-next.
v2 -> v3:
- Correct ESP seq in esp_xmit().
v1 -> v2:
- Fix comment style.

Quoted 8 lines

Mike Yu (4):
xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO
path
xfrm: Allow UDP encapsulation in crypto offload control path
xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP
packet
xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP
packet

Series applied, thanks a lot Mike!