Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDD1F559C9D for ; Tue, 22 Sep 2026 15:35:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091358; cv=none; b=SCC+wn52qxVWFueb+SvQUlrcU7arfXUj9zOE89BzkCv/GK766IV9kk9vo3FeIBimfMdFs29pw57dTkFMjlMGvRxFnJ7+aygSN7eH8P+NW0uOxA96gbF+gkibLoCGp0yN2v5BtMaVvBY0GpWRidZMeEr/i+CeXq0RaaZaW5Szi8U= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790091358; c=relaxed/simple; bh=QfwsydRxIhDzJmym0LsrxbqSTBdd6CSpFxV/E4WMzKg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=D8q1HXXjmQk71fJvgknfBnTg12P69DksF0PSYKQkfcvTwD0MkRYJB6G8LuZojul/sqmV1DJorod0am2wyZdsiTPGFhC+7WYSlYcR2ZZNOc69vCS/+zhO/rJ4CSED+MsYVmtAfGCqrs7eMEZv2NPdbG+Sc+pnz5I75gTe6erc6R0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=SJBgVEPo; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="SJBgVEPo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 031121F000FF; Tue, 22 Sep 2026 15:35:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790091356; bh=ATECcJqOaCrnaGsX/XwJKJZPHw3ofx/9YFQfmJAaBr8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=SJBgVEPon9ur0fIS+3lpaiKPtoV0Ug11dDW6QknIbCJg+KdE/7O882MkehPo5wcMS Ct/bonx16QGXmcjjx5bAQx355KNQ/mdhHkPVKwkbCikarDa8/qzxMf82BvwDD2E2sR j8j/SR3PHVWI+heojGBjxg+SmTkvQl0GmKBZI3wDo3H6nw21xqtv403WH5NEvvwVKJ B+NVKG0hI5IpINenWFyi0aKHofdCFPWrlYAKOtyw6yB6DPXlTWyrdm/fPb0oNSXhEi JIujYIkaHDDPqWafsuF6RmFRoFKqxcLhcWhkrFGxcmYlfQL3GRG2KIG9OKwcAQHLrn yjo13Gb6UC9Eg== From: Sasha Levin To: stable@vger.kernel.org Cc: Jianbo Liu , Cosmin Ratiu , Steffen Klassert , Sasha Levin Subject: [PATCH 6.18.y 1/3] xfrm: Refactor xfrm_input lock to reduce contention with RSS Date: Tue, 22 Sep 2026 11:35:52 -0400 Message-ID: <20260922153554.2829723-1-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <2026092225-extortion-shelving-5087@gregkh> References: <2026092225-extortion-shelving-5087@gregkh> Precedence: bulk X-Mailing-List: stable@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jianbo Liu [ Upstream commit 10a11861943902fda74f37f456b45183b2bca270 ] With newer NICs like mlx5 supporting RSS for IPsec crypto offload, packets for a single Security Association (SA) are scattered across multiple CPU cores for parallel processing. The xfrm_state spinlock (x->lock) is held for each packet during xfrm processing. When multiple connections or flows share the same SA, this parallelism causes high lock contention on x->lock, creating a performance bottleneck and limiting scalability. The original xfrm_input() function exacerbated this issue by releasing and immediately re-acquiring x->lock. For hardware crypto offload paths, this unlock/relock sequence is unnecessary and introduces significant overhead. This patch refactors the function to relocate the type_offload->input_tail call for the offload path, performing all necessary work while continuously holding the lock. This reordering is safe, since packets which don't pass the checks below will still fail them with the new code. Performance testing with iperf using multiple parallel streams over a single IPsec SA shows significant improvement in throughput as the number of queues (and thus CPU cores) increases: +-----------+---------------+--------------+-----------------+ | RX queues | Before (Gbps) | After (Gbps) | Improvement (%) | +-----------+---------------+--------------+-----------------+ | 2 | 32.3 | 34.4 | 6.5 | | 4 | 34.4 | 40.0 | 16.3 | | 6 | 24.5 | 38.3 | 56.3 | | 8 | 23.1 | 38.3 | 65.8 | | 12 | 18.1 | 29.9 | 65.2 | | 16 | 16.0 | 25.2 | 57.5 | +-----------+---------------+--------------+-----------------+ Signed-off-by: Jianbo Liu Reviewed-by: Cosmin Ratiu Signed-off-by: Steffen Klassert Stable-dep-of: 3cf5cdecd99c ("xfrm: save input state data before secpath resets") Signed-off-by: Sasha Levin --- net/xfrm/xfrm_input.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c index cfee63a6c87d2..8b2d2bb72d362 100644 --- a/net/xfrm/xfrm_input.c +++ b/net/xfrm/xfrm_input.c @@ -504,6 +504,7 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type) if (encap_type == -1) { async = 1; seq = XFRM_SKB_CB(skb)->seq.input.low; + spin_lock(&x->lock); goto resume; } /* GRO call */ @@ -540,6 +541,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type) XFRM_INC_STATS(net, LINUX_MIB_XFRMINHDRERROR); goto drop; } + + nexthdr = x->type_offload->input_tail(x, skb); } goto lock; @@ -637,11 +640,9 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type) goto drop_unlock; } - spin_unlock(&x->lock); - if (xfrm_tunnel_check(skb, x, family)) { XFRM_INC_STATS(net, LINUX_MIB_XFRMINSTATEMODEERROR); - goto drop; + goto drop_unlock; } seq_hi = htonl(xfrm_replay_seqhi(x, seq)); @@ -649,9 +650,8 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type) XFRM_SKB_CB(skb)->seq.input.low = seq; XFRM_SKB_CB(skb)->seq.input.hi = seq_hi; - if (crypto_done) { - nexthdr = x->type_offload->input_tail(x, skb); - } else { + if (!crypto_done) { + spin_unlock(&x->lock); dev_hold(skb->dev); nexthdr = x->type->input(x, skb); @@ -662,9 +662,9 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type) } dev_put(skb->dev); + spin_lock(&x->lock); } resume: - spin_lock(&x->lock); if (nexthdr < 0) { if (nexthdr == -EBADMSG) { xfrm_audit_state_icvfail(x, skb, -- 2.53.0