Skip to contentSkip to conversation

net: datagram: Bypass usercopy checks for kernel iterators

PATCH In review Chuck Lever 2 participants 2 messages Feb 25 – Feb 28

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net: datagram: Bypass usercopy checks for kernel iterators

From: Chuck Lever <[email protected]>

Profiling NFSD under an iozone workload showed that hardened usercopy checks consume roughly 1.3% of CPU in the TCP receive path. These checks validate memory regions during copies, but provide no security benefit when both source (skb data) and destination (kernel pages in BVEC/KVEC iterators) reside in kernel address space.

Modify simple_copy_to_iter() and crc32c_and_copy_to_iter() to call _copy_to_iter() directly when the destination is a kernel-only iterator, bypassing the usercopy hardening validation. User-backed iterators (ITER_UBUF, ITER_IOVEC) continue to use copy_to_iter() with full validation.

This benefits kernel consumers of TCP receive such as the NFS client and server and NVMe-TCP, which use ITER_BVEC for their receive buffers.

---
 net/core/datagram.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
net/core/datagram.c +12 -2
index c285c6465923..e83cf0125008 100644
@@ -490,7 +490,10 @@ static size_t crc32c_and_copy_to_iter(const void *addr, size_t bytes,
 	u32 *crcp = _crcp;
 	size_t copied;
 
-	copied = copy_to_iter(addr, bytes, i);
+	if (user_backed_iter(i))
+		copied = copy_to_iter(addr, bytes, i);
+	else
+		copied = _copy_to_iter(addr, bytes, i);
 	*crcp = crc32c(*crcp, addr, copied);
 	return copied;
 }
@@ -515,10 +518,17 @@ int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offset,
 EXPORT_SYMBOL(skb_copy_and_crc32c_datagram_iter);
 #endif /* CONFIG_NET_CRC32C */
 
+/*
+ * Bypass usercopy hardening for kernel-only iterators: no data
+ * crosses the user/kernel boundary, so the slab whitelist check
+ * on the source buffer is unnecessary overhead.
+ */
 static size_t simple_copy_to_iter(const void *addr, size_t bytes,
 		void *data __always_unused, struct iov_iter *i)
 {
-	return copy_to_iter(addr, bytes, i);
+	if (user_backed_iter(i))
+		return copy_to_iter(addr, bytes, i);
+	return _copy_to_iter(addr, bytes, i);
 }
 
 /**
Signature · 1 line
2.52.0
maintainer
#1 lore raw
Quoted 14 lines · On Wed, 25 Feb 2026 11:25:32 -0500 Chuck Lever wrote:

From: Chuck Lever <[email protected]>

Profiling NFSD under an iozone workload showed that hardened usercopy checks consume roughly 1.3% of CPU in the TCP receive path. These checks validate memory regions during copies, but provide no security benefit when both source (skb data) and destination (kernel pages in BVEC/KVEC iterators) reside in kernel address space.

Modify simple_copy_to_iter() and crc32c_and_copy_to_iter() to call _copy_to_iter() directly when the destination is a kernel-only iterator, bypassing the usercopy hardening validation. User-backed iterators (ITER_UBUF, ITER_IOVEC) continue to use copy_to_iter() with full validation.

Quoted 3 lines

This benefits kernel consumers of TCP receive such as the NFS client and server and NVMe-TCP, which use ITER_BVEC for their receive buffers.

If it makes such a difference why not make copy_to_iter() check the iter type? Why force callers to check it?

Quoted 35 lines
net/core/datagram.c +12 -2
index c285c6465923..e83cf0125008 100644
@@ -490,7 +490,10 @@ static size_t crc32c_and_copy_to_iter(const void *addr, size_t bytes,
 	u32 *crcp = _crcp;
 	size_t copied;
 
-	copied = copy_to_iter(addr, bytes, i);
+	if (user_backed_iter(i))
+		copied = copy_to_iter(addr, bytes, i);
+	else
+		copied = _copy_to_iter(addr, bytes, i);
 	*crcp = crc32c(*crcp, addr, copied);
 	return copied;
 }
@@ -515,10 +518,17 @@ int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offset,
 EXPORT_SYMBOL(skb_copy_and_crc32c_datagram_iter);
 #endif /* CONFIG_NET_CRC32C */
 
+/*
+ * Bypass usercopy hardening for kernel-only iterators: no data
+ * crosses the user/kernel boundary, so the slab whitelist check
+ * on the source buffer is unnecessary overhead.
+ */
 static size_t simple_copy_to_iter(const void *addr, size_t bytes,
 		void *data __always_unused, struct iov_iter *i)
 {
-	return copy_to_iter(addr, bytes, i);
+	if (user_backed_iter(i))
+		return copy_to_iter(addr, bytes, i);
+	return _copy_to_iter(addr, bytes, i);
 }
 
 /**