Support IPsec crypto offload for IPv6 ESP and IPv4 UDP-encapsulated ESP data paths
Subsystems (3)
v4 cover Support IPsec crypto offload for IPv6 ESP and IPv4 UDP-encapsulated ESP data paths
Currently, IPsec crypto offload is enabled for GRO code path. However, there are other code paths where the XFRM stack is involved; for example, IPv6 ESP packets handled by xfrm6_esp_rcv() in ESP layer, and IPv4 UDP-encapsulated ESP packets handled by udp_rcv() in UDP layer.
This patchset extends the crypto offload support to cover these two cases. This is useful for devices with traffic accounting (e.g., Android), where GRO can lead to inaccurate accounting on the underlying network. For example, VPN traffic might not be counted on the wifi network interface wlan0 if the packets are handled in GRO code path before entering the network stack for accounting.
Below is the RX data path scenario the crypto offload can be applied to.
+-----------+ +-------+
| HW Driver |-->| wlan0 |--------+
+-----------+ +-------+ |
v
+---------------+ +------+
+------>| Network Stack |-->| Apps |
| +---------------+ +------+
| |
| v
+--------+ +------------+
| ipsec1 |<--| XFRM Stack |
+--------+ +------------+v3 -> v4:
- Change the target tree to ipsec-next.
v2 -> v3:
- Correct ESP seq in esp_xmit().
v1 -> v2:
- Fix comment style.
Mike Yu (4):
xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO
path
xfrm: Allow UDP encapsulation in crypto offload control path
xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP
packet
xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP
packet
net/ipv4/esp4.c | 8 +++++++- net/ipv4/esp4_offload.c | 17 ++++++++++++++++- net/xfrm/xfrm_device.c | 6 +++--- net/xfrm/xfrm_input.c | 3 ++- net/xfrm/xfrm_policy.c | 5 ++++- 5 files changed, 32 insertions(+), 7 deletions(-)
Signature · 1 line
2.45.2.993.g49e7a77208-goog
PATCH v4 1/4 xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO path
IPsec crypt offload supports outbound IPv6 ESP packets, but it doesn't support inbound IPv6 ESP packets.
This change enables the crypto offload for inbound IPv6 ESP packets that are not handled through GRO code path. If HW drivers add the offload information to the skb, the packet will be handled in the crypto offload rx code path.
Apart from the change in crypto offload rx code path, the change in xfrm_policy_check is also needed.
Exampe of RX data path:
+-----------+ +-------+
| HW Driver |-->| wlan0 |--------+
+-----------+ +-------+ |
v
+---------------+ +------+
+------>| Network Stack |-->| Apps |
| +---------------+ +------+
| |
| v
+--------+ +------------+
| ipsec1 |<--| XFRM Stack |
+--------+ +------------+Test: Enabled both in/out IPsec crypto offload, and verified IPv6
ESP packets on Android device on both wifi/cellular network
--- net/xfrm/xfrm_input.c | 2 +- net/xfrm/xfrm_policy.c | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-)
net/xfrm/xfrm_input.c +1 -1
@@ -471,7 +471,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)struct xfrm_offload *xo =xfrm_offload (skb);struct sec_path *sp; -if (encap_type <0 || (xo && xo->flags &XFRM_GRO )) { +if (encap_type <0 || (xo && (xo->flags &XFRM_GRO || encap_type ==0 ))) { x =xfrm_input_state (skb);if (unlikely (x->dir && x->dir !=XFRM_SA_DIR_IN )) {
net/xfrm/xfrm_policy.c +4 -1
@@ -3718,12 +3718,15 @@ int __xfrm_policy_check(struct sock *sk, int dir, struct sk_buff *skb, pol =xfrm_in_fwd_icmp (skb, &fl, family, if_id);if (!pol) { +const bool is_crypto_offload = sp && + (xfrm_input_state (skb)->xso.type ==XFRM_DEV_OFFLOAD_CRYPTO ); +if (net->xfrm.policy_default[dir] ==XFRM_USERPOLICY_BLOCK ) {XFRM_INC_STATS (net,LINUX_MIB_XFRMINNOPOLS );return 0 ; } -if (sp &&secpath_has_nontransport (sp,0 , &xerr_idx)) { +if (sp &&secpath_has_nontransport (sp,0 , &xerr_idx) && !is_crypto_offload) {xfrm_secpath_reject (xerr_idx, skb, &fl);XFRM_INC_STATS (net,LINUX_MIB_XFRMINNOPOLS );return 0 ;
Signature · 1 line
2.45.2.993.g49e7a77208-goog
PATCH v4 2/4 xfrm: Allow UDP encapsulation in crypto offload control path
In reply to Mike YuUnblock this limitation so that SAs with encapsulation specified can be passed to HW drivers. HW drivers can still reject the SA in their implementation of xdo_dev_state_add if the encapsulation is not supported.
Test: Verified on Android device
--- net/xfrm/xfrm_device.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
net/xfrm/xfrm_device.c +3 -3
@@ -261,9 +261,9 @@ int xfrm_dev_state_add(struct net *net, struct xfrm_state *x, is_packet_offload = xuo->flags &XFRM_OFFLOAD_PACKET ; -/* We don't yet support UDP encapsulation and TFC padding. */ -if ((!is_packet_offload && x->encap) || x->tfcpad) { -NL_SET_ERR_MSG (extack,"Encapsulation and TFC padding can't be offloaded" ); +/* We don't yet support TFC padding. */ +if (x->tfcpad) { +NL_SET_ERR_MSG (extack,"TFC padding can't be offloaded" );return -EINVAL ; }
Signature · 1 line
2.45.2.993.g49e7a77208-goog
PATCH v4 3/4 xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP packet
In reply to Mike YuIf xfrm_input() is called with UDP_ENCAP_ESPINUDP, the packet is already processed in UDP layer that removes the UDP header. Therefore, there should be no much difference to treat it as an ESP packet in the XFRM stack.
Test: Enabled dir=in IPsec crypto offload, and verified IPv4
UDP-encapsulated ESP packets on both wifi/cellular network
--- net/xfrm/xfrm_input.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
net/xfrm/xfrm_input.c +2 -1
@@ -471,7 +471,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)struct xfrm_offload *xo =xfrm_offload (skb);struct sec_path *sp; -if (encap_type <0 || (xo && (xo->flags &XFRM_GRO || encap_type ==0 ))) { +if (encap_type <0 || (xo && (xo->flags &XFRM_GRO || encap_type ==0 || + encap_type ==UDP_ENCAP_ESPINUDP ))) { x =xfrm_input_state (skb);if (unlikely (x->dir && x->dir !=XFRM_SA_DIR_IN )) {
Signature · 1 line
2.45.2.993.g49e7a77208-goog
PATCH v4 4/4 xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP packet
In reply to Mike Yuesp_xmit() is already able to handle UDP encapsulation through the call to esp_output_head(). However, the ESP header and the outer IP header are not correct and need to be corrected.
Test: Enabled both dir=in/out IPsec crypto offload, and verified IPv4
UDP-encapsulated ESP packets on both wifi/cellular network
---v2->v3: https://lore.kernel.org/all/[email protected]
- Correct ESP seq in esp_xmit().v1->v2: https://lore.kernel.org/all/[email protected]
- Fix comment style.
--- net/ipv4/esp4.c | 8 +++++++- net/ipv4/esp4_offload.c | 17 ++++++++++++++++- 2 files changed, 23 insertions(+), 2 deletions(-)
net/ipv4/esp4.c +7 -1
@@ -349,6 +349,7 @@ static struct ip_esp_hdr *esp_output_udp_encap(struct sk_buff *skb, {struct udphdr *uh;unsigned int len; +struct xfrm_offload *xo =xfrm_offload (skb); len = skb->len + esp->tailen -skb_transport_offset (skb);if (len +sizeof (struct iphdr ) >IP_MAX_MTU )
@@ -360,7 +361,12 @@ static struct ip_esp_hdr *esp_output_udp_encap(struct sk_buff *skb, uh->len =htons (len); uh->check =0 ; - *skb_mac_header (skb) =IPPROTO_UDP ; +/* For IPv4 ESP with UDP encapsulation, if xo is not null, the skb is in the crypto offload +* data path, which means that esp_output_udp_encap is called outside of the XFRM stack. +* In this case, the mac header doesn't point to the IPv4 protocol field, so don't set it. +*/ +if (!xo || encap_type !=UDP_ENCAP_ESPINUDP ) + *skb_mac_header (skb) =IPPROTO_UDP ;return (struct ip_esp_hdr *)(uh +1 ); }
net/ipv4/esp4_offload.c +16 -1
@@ -264,6 +264,7 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb, netdev_features_struct esp_info esp;bool hw_offload = true;__u32 seq; +int encap_type =0 ; esp.inplace = true;
@@ -296,8 +297,10 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb, netdev_features_ esp.esph =ip_esp_hdr (skb); +if (x->encap) + encap_type = x->encap->encap_type; -if (!hw_offload || !skb_is_gso (skb)) { +if (!hw_offload || !skb_is_gso (skb) || (hw_offload && encap_type ==UDP_ENCAP_ESPINUDP )) { esp.nfrags =esp_output_head (x, skb, &esp);if (esp.nfrags <0 )return esp.nfrags;
@@ -324,6 +327,18 @@ static int esp_xmit(struct xfrm_state *x, struct sk_buff *skb, netdev_features_ esp.seqno =cpu_to_be64 (seq + ((u64 )xo->seq.hi <<32 )); +if (hw_offload && encap_type ==UDP_ENCAP_ESPINUDP ) { +/* In the XFRM stack, the encapsulation protocol is set to iphdr->protocol by +* setting *skb_mac_header(skb) (see esp_output_udp_encap()) where skb->mac_header +* points to iphdr->protocol (see xfrm4_tunnel_encap_add()). +* However, in esp_xmit(), skb->mac_header doesn't point to iphdr->protocol. +* Therefore, the protocol field needs to be corrected. +*/ +ip_hdr (skb)->protocol =IPPROTO_UDP ; + + esph->seq_no =htonl (seq); + } +ip_hdr (skb)->tot_len =htons (skb->len);ip_send_check (ip_hdr (skb));
Signature · 1 line
2.45.2.993.g49e7a77208-goog
Steffen Klassert
maintainerQuoted 33 lines · Mike Yu
Currently, IPsec crypto offload is enabled for GRO code path. However, there are other code paths where the XFRM stack is involved; for example, IPv6 ESP packets handled by
xfrm6_esp_rcv()in ESP layer, and IPv4 UDP-encapsulated ESP packets handled byudp_rcv()in UDP layer.This patchset extends the crypto offload support to cover these two cases. This is useful for devices with traffic accounting (e.g., Android), where GRO can lead to inaccurate accounting on the underlying network. For example, VPN traffic might not be counted on the wifi network interface wlan0 if the packets are handled in GRO code path before entering the network stack for accounting.
Below is the RX data path scenario the crypto offload can be applied to.
+-----------+ +-------+ | HW Driver |-->| wlan0 |--------+ +-----------+ +-------+ | v +---------------+ +------+ +------>| Network Stack |-->| Apps | | +---------------+ +------+ | | | v +--------+ +------------+ | ipsec1 |<--| XFRM Stack | +--------+ +------------+v3 -> v4:
- Change the target tree to ipsec-next.
v2 -> v3:
- Correct ESP seq inesp_xmit().
v1 -> v2:
- Fix comment style.
Quoted 8 lines
Mike Yu (4):
xfrm: Support crypto offload for inbound IPv6 ESP packets not in GRO
path
xfrm: Allow UDP encapsulation in crypto offload control path
xfrm: Support crypto offload for inbound IPv4 UDP-encapsulated ESP
packet
xfrm: Support crypto offload for outbound IPv4 UDP-encapsulated ESP
packet
Series applied, thanks a lot Mike!