Skip to contentSkip to conversation

iomap: avoid compaction for costly folio order allocation

PATCH In review Salvatore Dipietro 2 participants 2 messages Apr 3 – Apr 4 1 message not in the archive

Subsystems (2)
Stable branch Filesystems iomap
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Salvatore Dipietro

author
#1 lore raw

PATCH 1/1 iomap: avoid compaction for costly folio order allocation

Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace") introduced high-order folio allocations in the buffered write path. When memory is fragmented, each failed allocation triggers compaction and drain_all_pages() via __alloc_pages_slowpath(), causing a 0.75x throughput drop on pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system.

Strip __GFP_DIRECT_RECLAIM from folio allocations in iomap_get_folio() when the order exceeds PAGE_ALLOC_COSTLY_ORDER, making them purely opportunistic.

Fixes 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
Cc [email protected]
Signed-off-by Salvatore Dipietro <[email protected]>
---
 fs/iomap/buffered-io.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
fs/iomap/buffered-io.c +14 -1
index 92a831cf4bf1..cb843d54b4d9 100644
@@ -715,6 +715,7 @@ EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate);
 struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 {
 	fgf_t fgp = FGP_WRITEBEGIN | FGP_NOFS;
+	gfp_t gfp;
 
 	if (iter->flags & IOMAP_NOWAIT)
 		fgp |= FGP_NOWAIT;

@@ -722,8 +723,20 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 		fgp |= FGP_DONTCACHE;
 	fgp |= fgf_set_order(len);
 
+	gfp = mapping_gfp_mask(iter->inode->i_mapping);
+
+	/*
+	 * If the folio order hint exceeds PAGE_ALLOC_COSTLY_ORDER,
+	 * strip __GFP_DIRECT_RECLAIM to make the allocation purely
+	 * opportunistic.  This avoids compaction + drain_all_pages()
+	 * in __alloc_pages_slowpath() that devastate throughput
+	 * on large systems during buffered writes.
+	 */
+	if (FGF_GET_ORDER(fgp) > PAGE_ALLOC_COSTLY_ORDER)
+		gfp &= ~__GFP_DIRECT_RECLAIM;
+
 	return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
-			fgp, mapping_gfp_mask(iter->inode->i_mapping));
+			fgp, gfp);
 }
 EXPORT_SYMBOL_GPL(iomap_get_folio);
Signature · 6 lines
2.50.1 (Apple Git-155)




AMAZON DEVELOPMENT CENTER ITALY SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2504859, Capitale Sociale: 10.000 EUR i.v., Cod. Fisc. e P.IVA 10100050961, Societa con Socio Unico
#2 lore raw
Quoted 11 lines · Salvatore Dipietro

Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace") introduced high-order folio allocations in the buffered write path. When memory is fragmented, each failed allocation triggers compaction and drain_all_pages() via __alloc_pages_slowpath(), causing a 0.75x throughput drop on pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system.

Strip __GFP_DIRECT_RECLAIM from folio allocations in iomap_get_folio() when the order exceeds PAGE_ALLOC_COSTLY_ORDER, making them purely opportunistic.

Quoted 7 lines
Fixes 5d8edfb900d5 ("iomap: Copy larger chunks from userspace")
Cc [email protected]
Signed-off-by Salvatore Dipietro <[email protected]>
---
 fs/iomap/buffered-io.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

<formletter>

This is not the correct way to submit patches for inclusion in the stable kernel tree. Please read:
https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>