Skip to contentSkip to conversation

xfrm: hold dev ref until after transport_finish NF_HOOK

PATCH In review Simon Liebold 2 participants 4 messages Jun 12 – Jun 13

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover xfrm: hold dev ref until after transport_finish NF_HOOK

Thanks for the detailed analysis on v2, Sasha. Here's v3.

v3: Backport b05d42eefac7 ("xfrm: hold device only for the asynchronous decryption") as a prerequisite, making the tree structurally match mainline so the fix applies without the lifetime gap Sasha identified in v2, where the dev_put at resume: dropped the ref before the re-hold could cover it.

v2: Restore unconditional dev_put at resume: and instead take a fresh dev_hold immediately before transport_finish (when async && !xfrm_gro), avoiding the reference leak on nested transport-mode that v1's suppressed resume: dev_put caused. Prerequisite b05d42eefac7 ("xfrm: hold device only for the asynchronous decryption") was not backported as it restructures the lock ordering and resume:
label semantics of the decryption loop, requiring non-trivial adaptation beyond what a minimal stable fix warrants.

Jianbo Liu (1):
xfrm: hold device only for the asynchronous decryption

Qi Tang (1):
xfrm: hold dev ref until after transport_finish NF_HOOK

 net/ipv4/xfrm4_input.c |  5 ++++-
 net/ipv6/xfrm6_input.c |  5 ++++-
 net/xfrm/xfrm_input.c  | 25 +++++++++++++++++--------
 3 files changed, 25 insertions(+), 10 deletions(-)

base-commit: 1d3a00d3bacff25652c96e1527610c69e91f7c38

Signature · 12 lines
2.50.1




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
#1 lore raw

PATCH v3 1/2 xfrm: hold device only for the asynchronous decryption

From: Jianbo Liu <[email protected]>

[ Upstream commit b05d42eefac737ce3cd80114d3579111023941b8 ]

The dev_hold() on skb->dev during packet reception was originally added to prevent the device from being released prematurely during asynchronous decryption operations.

As current hardware can offload decryption, this asynchronous path is not always utilized. This often results in a pattern of dev_hold() immediately followed by dev_put() for each packet, creating unnecessary reference counting overhead detrimental to performance.

This patch optimizes this by skipping the dev_hold() and subsequent dev_put() when asynchronous decryption is not being performed.

Stable-dep-of: 1c428b038400 ("xfrm: hold dev ref until after transport_finish NF_HOOK")

---
 net/xfrm/xfrm_input.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
net/xfrm/xfrm_input.c +9 -8
index 8edcb32735e59..90a79558dca25 100644
@@ -492,6 +492,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 		/* An encap_type of -1 indicates async resumption. */
 		if (encap_type == -1) {
 			async = 1;
+			dev_put(skb->dev);
 			seq = XFRM_SKB_CB(skb)->seq.input.low;
 			goto resume;
 		}
@@ -638,18 +639,18 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 		XFRM_SKB_CB(skb)->seq.input.low = seq;
 		XFRM_SKB_CB(skb)->seq.input.hi = seq_hi;
 
-		dev_hold(skb->dev);
-
-		if (crypto_done)
+		if (crypto_done) {
 			nexthdr = x->type_offload->input_tail(x, skb);
-		else
+		} else {
+			dev_hold(skb->dev);
+
 			nexthdr = x->type->input(x, skb);
+			if (nexthdr == -EINPROGRESS)
+				return 0;
 
-		if (nexthdr == -EINPROGRESS)
-			return 0;
+			dev_put(skb->dev);
+		}
 resume:
-		dev_put(skb->dev);
-
 		spin_lock(&x->lock);
 		if (nexthdr < 0) {
 			if (nexthdr == -EBADMSG) {
Signature · 12 lines
2.50.1




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
#2 lore raw

PATCH v3 2/2 xfrm: hold dev ref until after transport_finish NF_HOOK

In reply to Simon Liebold

From: Qi Tang <[email protected]>

[ Upstream commit 1c428b03840094410c5fb6a5db30640486bbbfcb ]

After async crypto completes, xfrm_input_resume() calls dev_put() immediately on re-entry before the skb reaches transport_finish. The skb->dev pointer is then used inside NF_HOOK and its okfn, which can race with device teardown.

Remove the dev_put from the async resumption entry and instead drop the reference after the NF_HOOK call in transport_finish, using a saved device pointer since NF_HOOK may consume the skb. This covers NF_DROP, NF_QUEUE and NF_STOLEN paths that skip the okfn.

For non-transport exits (decaps, gro, drop) and secondary async return points, release the reference inline when async is set.

Fixes acf568ee859f ("xfrm: Reinject transport-mode packets through tasklet")

[ xfrm_inner_mode_input() always completes synchronously in this kernel version and cannot return -EINPROGRESS. That requires 7ac64f4598b4 ("xfrm: add mode_cbs module functionality"), which is not present, so the async dev_put path is unreachable and the hunk was omitted ]

---
 net/ipv4/xfrm4_input.c |  5 ++++-
 net/ipv6/xfrm6_input.c |  5 ++++-
 net/xfrm/xfrm_input.c  | 12 ++++++++++--
 3 files changed, 18 insertions(+), 4 deletions(-)
net/ipv4/xfrm4_input.c +4 -1
index 12a1a0f421956..adf21d6b6076c 100644
@@ -50,6 +50,7 @@ int xfrm4_transport_finish(struct sk_buff *skb, int async)
 {
 	struct xfrm_offload *xo = xfrm_offload(skb);
 	struct iphdr *iph = ip_hdr(skb);
+	struct net_device *dev = skb->dev;
 
 	iph->protocol = XFRM_MODE_SKB_CB(skb)->protocol;
 
@@ -73,8 +74,10 @@ int xfrm4_transport_finish(struct sk_buff *skb, int async)
 	}
 
 	NF_HOOK(NFPROTO_IPV4, NF_INET_PRE_ROUTING,
-		dev_net(skb->dev), NULL, skb, skb->dev, NULL,
+		dev_net(dev), NULL, skb, dev, NULL,
 		xfrm4_rcv_encap_finish);
+	if (async)
+		dev_put(dev);
 	return 0;
 }
 
net/ipv6/xfrm6_input.c +4 -1
index 9005fc156a20e..699a001ac1662 100644
@@ -43,6 +43,7 @@ static int xfrm6_transport_finish2(struct net *net, struct sock *sk,
 int xfrm6_transport_finish(struct sk_buff *skb, int async)
 {
 	struct xfrm_offload *xo = xfrm_offload(skb);
+	struct net_device *dev = skb->dev;
 	int nhlen = -skb_network_offset(skb);
 
 	skb_network_header(skb)[IP6CB(skb)->nhoff] =
@@ -68,8 +69,10 @@ int xfrm6_transport_finish(struct sk_buff *skb, int async)
 	}
 
 	NF_HOOK(NFPROTO_IPV6, NF_INET_PRE_ROUTING,
-		dev_net(skb->dev), NULL, skb, skb->dev, NULL,
+		dev_net(dev), NULL, skb, dev, NULL,
 		xfrm6_transport_finish2);
+	if (async)
+		dev_put(dev);
 	return 0;
 }
 
net/xfrm/xfrm_input.c +10 -2
index 90a79558dca25..5d3633ce6ba32 100644
@@ -492,7 +492,6 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 		/* An encap_type of -1 indicates async resumption. */
 		if (encap_type == -1) {
 			async = 1;
-			dev_put(skb->dev);
 			seq = XFRM_SKB_CB(skb)->seq.input.low;
 			goto resume;
 		}
@@ -645,8 +644,11 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 			dev_hold(skb->dev);
 
 			nexthdr = x->type->input(x, skb);
-			if (nexthdr == -EINPROGRESS)
+			if (nexthdr == -EINPROGRESS) {
+				if (async)
+					dev_put(skb->dev);
 				return 0;
+			}
 
 			dev_put(skb->dev);
 		}
@@ -717,6 +719,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 			sp->olen = 0;
 		if (skb_valid_dst(skb))
 			skb_dst_drop(skb);
+		if (async)
+			dev_put(skb->dev);
 		gro_cells_receive(&gro_cells, skb);
 		return 0;
 	} else {
@@ -736,6 +740,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 				sp->olen = 0;
 			if (skb_valid_dst(skb))
 				skb_dst_drop(skb);
+			if (async)
+				dev_put(skb->dev);
 			gro_cells_receive(&gro_cells, skb);
 			return err;
 		}
@@ -746,6 +752,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
 drop_unlock:
 	spin_unlock(&x->lock);
 drop:
+	if (async)
+		dev_put(skb->dev);
 	xfrm_rcv_cb(skb, family, x && x->type ? x->type->proto : nexthdr, -1);
 	kfree_skb(skb);
 	return 0;
Signature · 12 lines
2.50.1




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
maintainer
#3 lore raw
In reply to Simon Liebold
Quoted 6 lines · Simon Liebold

Thanks for the detailed analysis on v2, Sasha. Here's v3.

v3: Backport b05d42eefac7 ("xfrm: hold device only for the asynchronous decryption") as a prerequisite, making the tree structurally match mainline so the fix applies without the lifetime gap Sasha identified in v2, where the dev_put at resume: dropped the ref before the re-hold could cover it.

Whole series queued for 6.12.y, thanks.

--
Thanks,
Sasha