Skip to contentSkip to conversation

f2fs: enable buffered RWF_DONTCACHE

PATCH Superseded Wenjie Qi 4 participants 12 messages Aug 20 – Aug 25

Subsystems (2)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Wenjie Qi

author
#0 lore raw

v2 cover f2fs: enable buffered RWF_DONTCACHE

This series enables buffered RWF_DONTCACHE on F2FS for sustained one-pass streaming writes, where retaining the written data can displace more useful cache.

Patch 1 keeps normal and dropbehind folios in separate write bios and defers dropbehind completion unless it runs in preemptible task context. Its eligibility check matches the proposed bio_in_atomic() helper instead of checking only in_task().

Patch 2 passes FGP_DONTCACHE to the F2FS buffered write folio lookup and advertises FOP_DONTCACHE.

Tests were run on a Xiaomi phone with 10.7 GiB of kernel-visible memory, running Android 16 and Linux 6.12.69 with 4 KiB pages. /data used F2FS.

The performance test wrote exactly 64 GiB per run at 4 KiB,
8 KiB, 16 KiB, 32 KiB, 64 KiB, 128 KiB, 256 KiB, 512 KiB, and 1 MiB. Two counterbalanced rounds ran ascending normal-first and descending dontcache-first. Values below are equal-weight means of both runs; N=2. The pwritev2() writer models the streaming workload; it does not show that an unchanged Android application already issues RWF_DONTCACHE.

Android remained active with displays off. Each run started after a cache reset and at least 120 seconds of cooldown. Throughput and one-second kswapd0/global-memory samples cover the write loop.

Write-loop throughput was:

normal MiB/s dontcache MiB/s

 I/O      r1       r2      mean      r1       r2      mean     change
 4K      946.28   935.74   941.01   291.36   309.31   300.33   -68.08%
 8K     1077.05  1105.84  1091.45   477.79   479.79   478.79   -56.13%
 16K    1126.84  1118.49  1122.67   643.40   652.06   647.73   -42.30%
 32K    1150.62  1036.33  1093.48   762.24   751.45   756.84   -30.79%
 64K    1144.80  1163.82  1154.31   852.11   851.19   851.65   -26.22%
 128K   1166.29  1162.84  1164.57   867.47   865.05   866.26   -25.61%
 256K   1153.61  1172.78  1163.19   895.53   885.33   890.43   -23.45%
 512K   1173.61  1197.34  1185.48   903.09   903.01   903.05   -23.82%
 1M     1126.22  1154.59  1140.41   850.74   894.60   872.67   -23.48%

Average kswapd0 CPU and average global Cached were:

I/O kswapd0 CPU, normal/DC Cached MiB, normal/DC

 4K             18.45% / 0%                 4830.15 / 686.56
 8K             21.60% / 0%                 4862.35 / 591.48
 16K            22.75% / 0%                 4905.47 / 625.07
 32K            22.05% / 0%                 4945.82 / 561.59
 64K            23.46% / 0%                 4920.77 / 639.78
 128K           23.16% / 0%                 4971.37 / 693.26
 256K           23.68% / 0%                 4956.93 / 668.60
 512K           24.25% / 0%                 4972.22 / 663.92
 1M             22.01% / 0%                 5001.09 / 705.30

Other global memory means were:

MemAvailable MiB Dirty MiB Writeback MiB
I/O normal / DC normal / DC normal / DC

 4K      6513.08 / 6382.31     640.48 / 33.93       37.94 / 0.09
 8K      6560.77 / 6529.33     690.89 / 43.57       40.84 / 0.54
 16K     6587.46 / 6519.94     789.16 / 62.67       61.58 / 4.64
 32K     6571.95 / 6566.74     850.48 / 69.14       67.69 / 9.24
 64K     6600.64 / 6559.89    856.97 / 134.28      59.60 / 16.19
 128K    6627.03 / 6465.10    873.68 / 137.57      60.57 / 41.50
 256K    6615.58 / 6541.09    885.98 / 158.15      61.25 / 29.57
 512K    6625.52 / 6534.41    900.65 / 139.07      63.65 / 30.35
 1M      6677.09 / 6539.89    909.70 / 187.56      51.58 / 33.46

Active(file) MiB Inactive(file) MiB
I/O normal / DC normal / DC

 4K       279.57 / 264.13         4426.38 / 183.12
 8K       262.32 / 260.97         4472.62 / 182.19
 16K      392.09 / 252.95         4392.78 / 195.82
 32K      254.30 / 248.44         4554.18 / 187.34
 64K      252.73 / 244.94         4545.40 / 267.28
 128K     322.02 / 243.95         4530.63 / 298.91
 256K     245.47 / 240.02         4586.64 / 303.92
 512K     254.75 / 232.05         4591.17 / 288.27
 1M       238.88 / 233.64         4638.81 / 341.00

Dontcache left zero target-file pages resident at every size. Normal retained about 1.19--1.24 million pages. Normal runs incurred roughly 15.6 million kswapd page scans and steals per run, while dontcache recorded zero. Direct scan and allocation-stall deltas were zero in both modes.

Changes since the RFC:

- match the stricter proposed bio_in_atomic() eligibility rule;
- add phone correctness and two-round 64-GiB performance results.

Wenjie Qi (2):
f2fs: complete dropbehind write bios in safe task context
f2fs: enable buffered RWF_DONTCACHE

 fs/f2fs/data.c | 59 ++++++++++++++++++++++++++++++++++++++++----------
 fs/f2fs/file.c |  2 +-
 2 files changed, 49 insertions(+), 12 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v2 1/2 f2fs: complete dropbehind write bios in safe task context

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer a dropbehind bio to sbi->wq unless completion runs in preemptible task context.

Use the same eligibility conditions as the proposed bio_in_atomic() helper. Unlike !in_task(), this also covers disabled preemption and preemptible RCU read-side critical sections.

Keep the existing large-ATC deferral unchanged.

---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +42 -8
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
 	struct work_struct work;
+	bool dropbehind;
 	struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+	return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION
+	if (rcu_preempt_depth())
+		return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+	return true;
+#else
+	return !preemptible();
+#endif
+}
+
 #define	F2FS_BIO_POOL_SIZE	NR_CURSEG_TYPE
 
 int __init f2fs_init_bioset(void)
@@ -426,12 +447,13 @@ static void f2fs_write_end_io(struct bio *bio)
 
 	sbi = bio->bi_private;
 
-	if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
-		struct work_struct *w;
+	if ((to_f2fs_bio(bio)->dropbehind && f2fs_bio_in_atomic()) ||
+	    (in_atomic() &&
+	     bio->bi_iter.bi_size > sbi->max_atc_write_bio_size)) {
+		struct work_struct *work = &to_f2fs_bio(bio)->work;
 
-		w = &container_of(bio, struct f2fs_bio, bio)->work;
-		INIT_WORK(w, f2fs_write_end_io_work);
-		queue_work(sbi->wq, w);
+		INIT_WORK(work, f2fs_write_end_io_work);
+		queue_work(sbi->wq, work);
 	} else {
 		f2fs_write_end_bio(bio);
 	}
@@ -530,6 +552,8 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
 	bio = bio_alloc_bioset(bdev, npages,
 				fio->op | fio->op_flags | f2fs_io_flags(fio),
 				GFP_NOIO, &f2fs_bioset);
+	to_f2fs_bio(bio)->dropbehind =
+			!is_read_io(fio->op) && folio_test_dropbehind(fio->folio);
 	bio->bi_iter.bi_sector = sector;
 	if (is_read_io(fio->op)) {
 		bio->bi_end_io = f2fs_read_end_io;
@@ -825,6 +849,13 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
 	return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
 }
 
+static bool f2fs_bio_dropbehind_mergeable(struct bio *bio,
+					  struct f2fs_io_info *fio)
+{
+	return to_f2fs_bio(bio)->dropbehind ==
+		folio_test_dropbehind(fio->folio);
+}
+
 static bool io_type_is_mergeable(struct f2fs_bio_info *io,
 						struct f2fs_io_info *fio)
 {
@@ -1017,8 +1048,10 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)
 
 	trace_f2fs_submit_folio_bio(data_folio, fio);
 
-	if (bio && !page_is_mergeable(fio->sbi, bio, *fio->last_block,
-						fio->new_blkaddr))
+	if (bio &&
+	    (!page_is_mergeable(fio->sbi, bio, *fio->last_block,
+				fio->new_blkaddr) ||
+	     !f2fs_bio_dropbehind_mergeable(bio, fio)))
 		f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL);
 alloc_new:
 	if (!bio) {
@@ -1118,7 +1151,8 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
 	    (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,
 			      fio->new_blkaddr) ||
 	     !f2fs_crypt_mergeable_bio(io->bio, fio_inode(fio),
-				bio_folio->index, fio)))
+				bio_folio->index, fio) ||
+	     !f2fs_bio_dropbehind_mergeable(io->bio, fio)))
 		__submit_merged_bio(io);
 alloc_new:
 	if (io->bio == NULL) {
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 1/2] f2fs: complete dropbehind write bios in safe task context

Quoted 47 lines · Wenjie Qi

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer a dropbehind bio to sbi->wq unless completion runs in preemptible task context.

Use the same eligibility conditions as the proposed bio_in_atomic() helper. Unlike !in_task(), this also covers disabled preemption and preemptible RCU read-side critical sections.

Keep the existing large-ATC deferral unchanged.

---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +11 -0
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
        struct work_struct work;
+       bool dropbehind;
        struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+       return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION

fs/f2fs/data.c struct f2fs_folio_state { · line 69

Quoted 10 lines
fs/f2fs/data.c +10 -0
+       if (rcu_preempt_depth())
+               return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+       return true;
+#else
+       return !preemptible();
+#endif
+}
+

Hi Wenjie,

I see that this [1] has already been included in the pull request [2], so perhaps you can use bio_in_atomic() directly once it lands.

Also, I feel that something is quite odd with mm/filemap.c. It shouldn't require everyone to reinvent their own workqueues, as I mentioned here [3]. I also noticed that Alexandre is adding a workqueue in filemap.c[4], which might be extended to file page cache handling as well in the future.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/CAGsJ_4wUvLoKGafR3U-ji1kUUaz6K_QdN6QxMT-apSaO5tXp0A@mail.gmail.com/
[4] https://lore.kernel.org/all/[email protected]/

Thanks
Barry

author
#3 lore raw

Re: [PATCH v2 1/2] f2fs: complete dropbehind write bios in safe task context

Hi Barry,

bio_in_atomic() is not in the current f2fs dev-test base yet, so v2 carries the same predicate locally. I will replace it with bio_in_atomic() once the block pull is merged into the base.

I also checked Alexandre's v3 series. Its deferred path is currently specific to swap-cache folios. folio_end_writeback() detects a dropbehind swap-cache folio and calls swap_writeback_dropbehind_folio(), while the per-CPU lists and swap_dropbehind_wq live in mm/swap_state.c. The non-swap path still calls folio_end_dropbehind() directly, so the series does not yet provide a common deferred path for file-backed folios.

This F2FS patch reuses the existing sbi->wq rather than allocating a new workqueue, although the completion deferral remains F2FS-specific.

Regards,
Wenjie

Quoted 78 lines · Barry Song
Quoted 47 lines · Wenjie Qi

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer a dropbehind bio to sbi->wq unless completion runs in preemptible task context.

Use the same eligibility conditions as the proposed bio_in_atomic() helper. Unlike !in_task(), this also covers disabled preemption and preemptible RCU read-side critical sections.

Keep the existing large-ATC deferral unchanged.

---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +11 -0
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
        struct work_struct work;
+       bool dropbehind;
        struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+       return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION

fs/f2fs/data.c struct f2fs_folio_state { · line 69

Quoted 10 lines
fs/f2fs/data.c +10 -0
+       if (rcu_preempt_depth())
+               return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+       return true;
+#else
+       return !preemptible();
+#endif
+}
+

Hi Wenjie,

I see that this [1] has already been included in the pull request [2], so perhaps you can use bio_in_atomic() directly once it lands.

Also, I feel that something is quite odd with mm/filemap.c. It shouldn't require everyone to reinvent their own workqueues, as I mentioned here [3]. I also noticed that Alexandre is adding a workqueue in filemap.c[4], which might be extended to file page cache handling as well in the future.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/CAGsJ_4wUvLoKGafR3U-ji1kUUaz6K_QdN6QxMT-apSaO5tXp0A@mail.gmail.com/
[4] https://lore.kernel.org/all/[email protected]/

Thanks
Barry

author
#4 lore raw

PATCH v2 2/2 f2fs: enable buffered RWF_DONTCACHE

In reply to Wenjie Qi

Pass FGP_DONTCACHE to f2fs_filemap_get_folio() for IOCB_DONTCACHE writes and advertise FOP_DONTCACHE.

Keep the F2FS-specific lookup flags because write_begin_get_folio() adds FGP_STABLE, which can deadlock here.

---
 fs/f2fs/data.c | 9 ++++++---
 fs/f2fs/file.c | 2 +-
 2 files changed, 7 insertions(+), 4 deletions(-)
fs/f2fs/data.c +6 -3
index 774a3e2d8e3..ebd9dc582f9 100644
@@ -3981,11 +3981,15 @@ static int f2fs_write_begin(const struct kiocb *iocb,
 	struct inode *inode = mapping->host;
 	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
 	struct folio *folio;
+	fgf_t fgp_flags = FGP_LOCK | FGP_WRITE | FGP_CREAT;
 	pgoff_t index = pos >> PAGE_SHIFT;
 	bool need_balance = false;
 	block_t blkaddr = NULL_ADDR;
 	int err = 0;
 
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		fgp_flags |= FGP_DONTCACHE;
+
 	trace_f2fs_write_begin(inode, pos, len);
 
 	if (!f2fs_is_checkpoint_ready(sbi)) {
@@ -4031,9 +4035,8 @@ static int f2fs_write_begin(const struct kiocb *iocb,
 	 * Do not use FGP_STABLE to avoid deadlock.
 	 * Will wait that below with our IO control.
 	 */
-	folio = f2fs_filemap_get_folio(mapping, index,
-				FGP_LOCK | FGP_WRITE | FGP_CREAT,
-				mapping_gfp_mask(mapping));
+	folio = f2fs_filemap_get_folio(mapping, index, fgp_flags,
+				       mapping_gfp_mask(mapping));
 	if (IS_ERR(folio)) {
 		err = PTR_ERR(folio);
 		goto fail;
fs/f2fs/file.c +1 -1
index a54b3ab52f1..08285c07c2d 100644
@@ -5911,6 +5911,6 @@ const struct file_operations f2fs_file_operations = {
 	.splice_read	= f2fs_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 	.fadvise	= f2fs_file_fadvise,
-	.fop_flags	= FOP_BUFFER_RASYNC,
+	.fop_flags	= FOP_BUFFER_RASYNC | FOP_DONTCACHE,
 	.setlease	= generic_setlease,
 };
Signature · 1 line
2.43.0
maintainer
#5 lore raw
In reply to Wenjie Qi
Quoted 22 lines · Wenjie Qi

This series enables buffered RWF_DONTCACHE on F2FS for sustained one-pass streaming writes, where retaining the written data can displace more useful cache.

Patch 1 keeps normal and dropbehind folios in separate write bios and defers dropbehind completion unless it runs in preemptible task context. Its eligibility check matches the proposed bio_in_atomic() helper instead of checking only in_task().

Patch 2 passes FGP_DONTCACHE to the F2FS buffered write folio lookup and advertises FOP_DONTCACHE.

Tests were run on a Xiaomi phone with 10.7 GiB of kernel-visible memory, running Android 16 and Linux 6.12.69 with 4 KiB pages. /data used F2FS.

The performance test wrote exactly 64 GiB per run at 4 KiB,
8 KiB, 16 KiB, 32 KiB, 64 KiB, 128 KiB, 256 KiB, 512 KiB, and 1 MiB. Two counterbalanced rounds ran ascending normal-first and descending dontcache-first. Values below are equal-weight means of both runs; N=2. The pwritev2() writer models the streaming workload; it does not show that an unchanged Android application already issues RWF_DONTCACHE.

Quoted 4 lines

Android remained active with displays off. Each run started after a cache reset and at least 120 seconds of cooldown. Throughput and one-second kswapd0/global-memory samples cover the write loop.

Thanks very much for the update.

Quoted 1 line

Write-loop throughput was:

What about read?

Quoted 12 lines

normal MiB/s dontcache MiB/s

  I/O      r1       r2      mean      r1       r2      mean     change
  4K      946.28   935.74   941.01   291.36   309.31   300.33   -68.08%
  8K     1077.05  1105.84  1091.45   477.79   479.79   478.79   -56.13%
  16K    1126.84  1118.49  1122.67   643.40   652.06   647.73   -42.30%
  32K    1150.62  1036.33  1093.48   762.24   751.45   756.84   -30.79%
  64K    1144.80  1163.82  1154.31   852.11   851.19   851.65   -26.22%
  128K   1166.29  1162.84  1164.57   867.47   865.05   866.26   -25.61%
  256K   1153.61  1172.78  1163.19   895.53   885.33   890.43   -23.45%
  512K   1173.61  1197.34  1185.48   903.09   903.01   903.05   -23.82%
  1M     1126.22  1154.59  1140.41   850.74   894.60   872.67   -23.48%

Seems side-effect, how will this affect userspace applications?

Quoted 30 lines

Average kswapd0 CPU and average global Cached were:

I/O kswapd0 CPU, normal/DC Cached MiB, normal/DC

  4K             18.45% / 0%                 4830.15 / 686.56
  8K             21.60% / 0%                 4862.35 / 591.48
  16K            22.75% / 0%                 4905.47 / 625.07
  32K            22.05% / 0%                 4945.82 / 561.59
  64K            23.46% / 0%                 4920.77 / 639.78
  128K           23.16% / 0%                 4971.37 / 693.26
  256K           23.68% / 0%                 4956.93 / 668.60
  512K           24.25% / 0%                 4972.22 / 663.92
  1M             22.01% / 0%                 5001.09 / 705.30

Other global memory means were:

MemAvailable MiB Dirty MiB Writeback MiB
I/O normal / DC normal / DC normal / DC

  4K      6513.08 / 6382.31     640.48 / 33.93       37.94 / 0.09
  8K      6560.77 / 6529.33     690.89 / 43.57       40.84 / 0.54
  16K     6587.46 / 6519.94     789.16 / 62.67       61.58 / 4.64
  32K     6571.95 / 6566.74     850.48 / 69.14       67.69 / 9.24
  64K     6600.64 / 6559.89    856.97 / 134.28      59.60 / 16.19
  128K    6627.03 / 6465.10    873.68 / 137.57      60.57 / 41.50
  256K    6615.58 / 6541.09    885.98 / 158.15      61.25 / 29.57
  512K    6625.52 / 6534.41    900.65 / 139.07      63.65 / 30.35
  1M      6677.09 / 6539.89    909.70 / 187.56      51.58 / 33.46

Active(file) MiB Inactive(file) MiB
I/O normal / DC normal / DC

Quoted 9 lines
  4K       279.57 / 264.13         4426.38 / 183.12
  8K       262.32 / 260.97         4472.62 / 182.19
  16K      392.09 / 252.95         4392.78 / 195.82
  32K      254.30 / 248.44         4554.18 / 187.34
  64K      252.73 / 244.94         4545.40 / 267.28
  128K     322.02 / 243.95         4530.63 / 298.91
  256K     245.47 / 240.02         4586.64 / 303.92
  512K     254.75 / 232.05         4591.17 / 288.27
  1M       238.88 / 233.64         4638.81 / 341.00

Can we compare the data before/after the patch?

Thanks,

Quoted 19 lines

Dontcache left zero target-file pages resident at every size. Normal retained about 1.19--1.24 million pages. Normal runs incurred roughly 15.6 million kswapd page scans and steals per run, while dontcache recorded zero. Direct scan and allocation-stall deltas were zero in both modes.

Changes since the RFC:

- match the stricter proposed bio_in_atomic() eligibility rule;
- add phone correctness and two-round 64-GiB performance results.

Wenjie Qi (2):
f2fs: complete dropbehind write bios in safe task context
f2fs: enable buffered RWF_DONTCACHE

  fs/f2fs/data.c | 59 ++++++++++++++++++++++++++++++++++++++++----------
  fs/f2fs/file.c |  2 +-
  2 files changed, 49 insertions(+), 12 deletions(-)
author
#6 lore raw

I focused on writes in v2. FOP_DONTCACHE also enables buffered reads, and F2FS uses the generic filemap_read() path for them. I will add read cache-residency, and throughput results.

Only applications that explicitly pass RWF_DONTCACHE pay this cost; normal buffered I/O does not opt in. The current numbers show the cost at maximum write throughput. A rate-limited workload may behave differently, which this test does not measure.

Before the patch F2FS returns EOPNOTSUPP for RWF_DONTCACHE, so there is no direct dontcache comparison. I will compare normal buffered read/write before and after the patch to check common-path overhead.

Thanks,

Quoted 114 lines · Chao Yu
Quoted 22 lines · Wenjie Qi

This series enables buffered RWF_DONTCACHE on F2FS for sustained one-pass streaming writes, where retaining the written data can displace more useful cache.

Patch 1 keeps normal and dropbehind folios in separate write bios and defers dropbehind completion unless it runs in preemptible task context. Its eligibility check matches the proposed bio_in_atomic() helper instead of checking only in_task().

Patch 2 passes FGP_DONTCACHE to the F2FS buffered write folio lookup and advertises FOP_DONTCACHE.

Tests were run on a Xiaomi phone with 10.7 GiB of kernel-visible memory, running Android 16 and Linux 6.12.69 with 4 KiB pages. /data used F2FS.

The performance test wrote exactly 64 GiB per run at 4 KiB,
8 KiB, 16 KiB, 32 KiB, 64 KiB, 128 KiB, 256 KiB, 512 KiB, and 1 MiB. Two counterbalanced rounds ran ascending normal-first and descending dontcache-first. Values below are equal-weight means of both runs; N=2. The pwritev2() writer models the streaming workload; it does not show that an unchanged Android application already issues RWF_DONTCACHE.

Quoted 4 lines

Android remained active with displays off. Each run started after a cache reset and at least 120 seconds of cooldown. Throughput and one-second kswapd0/global-memory samples cover the write loop.

Thanks very much for the update.

Quoted 1 line

Write-loop throughput was:

What about read?

Quoted 12 lines

normal MiB/s dontcache MiB/s

  I/O      r1       r2      mean      r1       r2      mean     change
  4K      946.28   935.74   941.01   291.36   309.31   300.33   -68.08%
  8K     1077.05  1105.84  1091.45   477.79   479.79   478.79   -56.13%
  16K    1126.84  1118.49  1122.67   643.40   652.06   647.73   -42.30%
  32K    1150.62  1036.33  1093.48   762.24   751.45   756.84   -30.79%
  64K    1144.80  1163.82  1154.31   852.11   851.19   851.65   -26.22%
  128K   1166.29  1162.84  1164.57   867.47   865.05   866.26   -25.61%
  256K   1153.61  1172.78  1163.19   895.53   885.33   890.43   -23.45%
  512K   1173.61  1197.34  1185.48   903.09   903.01   903.05   -23.82%
  1M     1126.22  1154.59  1140.41   850.74   894.60   872.67   -23.48%

Seems side-effect, how will this affect userspace applications?

Quoted 30 lines

Average kswapd0 CPU and average global Cached were:

I/O kswapd0 CPU, normal/DC Cached MiB, normal/DC

  4K             18.45% / 0%                 4830.15 / 686.56
  8K             21.60% / 0%                 4862.35 / 591.48
  16K            22.75% / 0%                 4905.47 / 625.07
  32K            22.05% / 0%                 4945.82 / 561.59
  64K            23.46% / 0%                 4920.77 / 639.78
  128K           23.16% / 0%                 4971.37 / 693.26
  256K           23.68% / 0%                 4956.93 / 668.60
  512K           24.25% / 0%                 4972.22 / 663.92
  1M             22.01% / 0%                 5001.09 / 705.30

Other global memory means were:

MemAvailable MiB Dirty MiB Writeback MiB
I/O normal / DC normal / DC normal / DC

  4K      6513.08 / 6382.31     640.48 / 33.93       37.94 / 0.09
  8K      6560.77 / 6529.33     690.89 / 43.57       40.84 / 0.54
  16K     6587.46 / 6519.94     789.16 / 62.67       61.58 / 4.64
  32K     6571.95 / 6566.74     850.48 / 69.14       67.69 / 9.24
  64K     6600.64 / 6559.89    856.97 / 134.28      59.60 / 16.19
  128K    6627.03 / 6465.10    873.68 / 137.57      60.57 / 41.50
  256K    6615.58 / 6541.09    885.98 / 158.15      61.25 / 29.57
  512K    6625.52 / 6534.41    900.65 / 139.07      63.65 / 30.35
  1M      6677.09 / 6539.89    909.70 / 187.56      51.58 / 33.46

Active(file) MiB Inactive(file) MiB
I/O normal / DC normal / DC

Quoted 9 lines
  4K       279.57 / 264.13         4426.38 / 183.12
  8K       262.32 / 260.97         4472.62 / 182.19
  16K      392.09 / 252.95         4392.78 / 195.82
  32K      254.30 / 248.44         4554.18 / 187.34
  64K      252.73 / 244.94         4545.40 / 267.28
  128K     322.02 / 243.95         4530.63 / 298.91
  256K     245.47 / 240.02         4586.64 / 303.92
  512K     254.75 / 232.05         4591.17 / 288.27
  1M       238.88 / 233.64         4638.81 / 341.00

Can we compare the data before/after the patch?

Thanks,

Quoted 19 lines

Dontcache left zero target-file pages resident at every size. Normal retained about 1.19--1.24 million pages. Normal runs incurred roughly 15.6 million kswapd page scans and steals per run, while dontcache recorded zero. Direct scan and allocation-stall deltas were zero in both modes.

Changes since the RFC:

- match the stricter proposed bio_in_atomic() eligibility rule;
- add phone correctness and two-round 64-GiB performance results.

Wenjie Qi (2):
f2fs: complete dropbehind write bios in safe task context
f2fs: enable buffered RWF_DONTCACHE

  fs/f2fs/data.c | 59 ++++++++++++++++++++++++++++++++++++++++----------
  fs/f2fs/file.c |  2 +-
  2 files changed, 49 insertions(+), 12 deletions(-)
author
#7 lore raw

v3 cover f2fs: enable buffered RWF_DONTCACHE

In reply to Wenjie Qi

This series enables buffered RWF_DONTCACHE on F2FS for sustained one-pass streaming writes, where retaining the written data can displace more useful cache.

Patch 1 records whether an F2FS write bio contains dropbehind folios, keeps normal and dropbehind folios in separate bios in the IPU and OPU paths, and defers unsafe dropbehind completion through the existing sbi->wq.

Patch 2 passes FGP_DONTCACHE to the F2FS buffered write folio lookup and advertises FOP_DONTCACHE.

Tests were run on a Xiaomi phone with 10.7 GiB of kernel-visible memory, running Android 16 and Linux 6.12.69 with 4 KiB pages. /data used F2FS.

The performance test wrote exactly 64 GiB per run at 4 KiB,
8 KiB, 16 KiB, 32 KiB, 64 KiB, 128 KiB, 256 KiB, 512 KiB, and 1 MiB. Two counterbalanced rounds ran ascending normal-first and descending dontcache-first. Values below are equal-weight means of both runs; N=2. The pwritev2() writer models the streaming workload; it does not show that an unchanged Android application already issues RWF_DONTCACHE.

Android remained active with displays off. Each run started after a cache reset and at least 120 seconds of cooldown. Throughput and one-second kswapd0/global-memory samples cover the write loop.

Write-loop throughput was:

normal MiB/s dontcache MiB/s

 I/O      r1       r2      mean      r1       r2      mean     change
 4K      946.28   935.74   941.01   291.36   309.31   300.33   -68.08%
 8K     1077.05  1105.84  1091.45   477.79   479.79   478.79   -56.13%
 16K    1126.84  1118.49  1122.67   643.40   652.06   647.73   -42.30%
 32K    1150.62  1036.33  1093.48   762.24   751.45   756.84   -30.79%
 64K    1144.80  1163.82  1154.31   852.11   851.19   851.65   -26.22%
 128K   1166.29  1162.84  1164.57   867.47   865.05   866.26   -25.61%
 256K   1153.61  1172.78  1163.19   895.53   885.33   890.43   -23.45%
 512K   1173.61  1197.34  1185.48   903.09   903.01   903.05   -23.82%
 1M     1126.22  1154.59  1140.41   850.74   894.60   872.67   -23.48%

Average kswapd0 CPU and average global Cached were:

I/O kswapd0 CPU, normal/DC Cached MiB, normal/DC

 4K             18.45% / 0%                 4830.15 / 686.56
 8K             21.60% / 0%                 4862.35 / 591.48
 16K            22.75% / 0%                 4905.47 / 625.07
 32K            22.05% / 0%                 4945.82 / 561.59
 64K            23.46% / 0%                 4920.77 / 639.78
 128K           23.16% / 0%                 4971.37 / 693.26
 256K           23.68% / 0%                 4956.93 / 668.60
 512K           24.25% / 0%                 4972.22 / 663.92
 1M             22.01% / 0%                 5001.09 / 705.30

Other global memory means were:

MemAvailable MiB Dirty MiB Writeback MiB
I/O normal / DC normal / DC normal / DC

 4K      6513.08 / 6382.31     640.48 / 33.93       37.94 / 0.09
 8K      6560.77 / 6529.33     690.89 / 43.57       40.84 / 0.54
 16K     6587.46 / 6519.94     789.16 / 62.67       61.58 / 4.64
 32K     6571.95 / 6566.74     850.48 / 69.14       67.69 / 9.24
 64K     6600.64 / 6559.89    856.97 / 134.28      59.60 / 16.19
 128K    6627.03 / 6465.10    873.68 / 137.57      60.57 / 41.50
 256K    6615.58 / 6541.09    885.98 / 158.15      61.25 / 29.57
 512K    6625.52 / 6534.41    900.65 / 139.07      63.65 / 30.35
 1M      6677.09 / 6539.89    909.70 / 187.56      51.58 / 33.46

Active(file) MiB Inactive(file) MiB
I/O normal / DC normal / DC

 4K       279.57 / 264.13         4426.38 / 183.12
 8K       262.32 / 260.97         4472.62 / 182.19
 16K      392.09 / 252.95         4392.78 / 195.82
 32K      254.30 / 248.44         4554.18 / 187.34
 64K      252.73 / 244.94         4545.40 / 267.28
 128K     322.02 / 243.95         4530.63 / 298.91
 256K     245.47 / 240.02         4586.64 / 303.92
 512K     254.75 / 232.05         4591.17 / 288.27
 1M       238.88 / 233.64         4638.81 / 341.00

Dontcache left zero target-file pages resident at every size. Normal retained about 1.19--1.24 million pages. Normal runs incurred roughly 15.6 million kswapd page scans and steals per run, while dontcache recorded zero. Direct scan and allocation-stall deltas were zero in both modes.

A controlled explicit-dontcache model issued 64 KiB writes for 120 seconds at 64, 128, and 256 MiB/s. Both modes sustained all three rates in both rounds with no final schedule overrun. The late-write ratio was 0.02%--0.41%, and maximum schedule lag was 3.3--6.0 ms. Dontcache left zero target pages resident. This was a controlled model, not an unchanged Xiaomi application.

Read tests were unpaced. Each run started with zero source pages resident and read the same 64 GiB file sequentially at full speed with preadv2(). Results from two counterbalanced rounds were:

I/O normal MiB/s dontcache MiB/s change

 4K          1825.50          1695.10        -7.14%
 8K          1901.46          1847.96        -2.81%
 16K         1941.60          1872.55        -3.56%
 32K         1960.95          1905.92        -2.81%
 64K         1951.35          1886.54        -3.32%
 128K        1960.03          1915.82        -2.26%
 256K        1976.01          1900.41        -3.83%
 512K        1973.31          1911.24        -3.15%
 1M          2230.91          2007.82       -10.00%

Dontcache left zero source pages resident in all measured read runs.

The normal-I/O control showed read-throughput differences of +0.67%, -0.90%, and -2.58%, and write-throughput differences of -1.75%, +0.33%, and +2.33%, at 4 KiB, 64 KiB, and 1 MiB respectively.

Changes since v2:

- keep the implementation unchanged;
- add paced-write, unpaced-read, and normal-I/O control results.

Wenjie Qi (2):
f2fs: complete dropbehind write bios in safe task context
f2fs: enable buffered RWF_DONTCACHE

 fs/f2fs/data.c | 59 ++++++++++++++++++++++++++++++++++++++++----------
 fs/f2fs/file.c |  2 +-
 2 files changed, 49 insertions(+), 12 deletions(-)
Signature · 1 line
2.43.0
author
#8 lore raw

PATCH v3 1/2 f2fs: complete dropbehind write bios in safe task context

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer dropbehind bios to sbi->wq unless completion runs in preemptible task context.

Use an F2FS-local context check for this decision. Task context alone is not sufficient: preemption may still be disabled, or completion may run in a preemptible RCU read-side critical section.

Keep the existing large-ATC deferral unchanged.

---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +42 -8
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
 	struct work_struct work;
+	bool dropbehind;
 	struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+	return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION
+	if (rcu_preempt_depth())
+		return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+	return true;
+#else
+	return !preemptible();
+#endif
+}
+
 #define	F2FS_BIO_POOL_SIZE	NR_CURSEG_TYPE
 
 int __init f2fs_init_bioset(void)
@@ -426,12 +447,13 @@ static void f2fs_write_end_io(struct bio *bio)
 
 	sbi = bio->bi_private;
 
-	if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
-		struct work_struct *w;
+	if ((to_f2fs_bio(bio)->dropbehind && f2fs_bio_in_atomic()) ||
+	    (in_atomic() &&
+	     bio->bi_iter.bi_size > sbi->max_atc_write_bio_size)) {
+		struct work_struct *work = &to_f2fs_bio(bio)->work;
 
-		w = &container_of(bio, struct f2fs_bio, bio)->work;
-		INIT_WORK(w, f2fs_write_end_io_work);
-		queue_work(sbi->wq, w);
+		INIT_WORK(work, f2fs_write_end_io_work);
+		queue_work(sbi->wq, work);
 	} else {
 		f2fs_write_end_bio(bio);
 	}
@@ -530,6 +552,8 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
 	bio = bio_alloc_bioset(bdev, npages,
 				fio->op | fio->op_flags | f2fs_io_flags(fio),
 				GFP_NOIO, &f2fs_bioset);
+	to_f2fs_bio(bio)->dropbehind =
+			!is_read_io(fio->op) && folio_test_dropbehind(fio->folio);
 	bio->bi_iter.bi_sector = sector;
 	if (is_read_io(fio->op)) {
 		bio->bi_end_io = f2fs_read_end_io;
@@ -825,6 +849,13 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
 	return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
 }
 
+static bool f2fs_bio_dropbehind_mergeable(struct bio *bio,
+					  struct f2fs_io_info *fio)
+{
+	return to_f2fs_bio(bio)->dropbehind ==
+		folio_test_dropbehind(fio->folio);
+}
+
 static bool io_type_is_mergeable(struct f2fs_bio_info *io,
 						struct f2fs_io_info *fio)
 {
@@ -1017,8 +1048,10 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)
 
 	trace_f2fs_submit_folio_bio(data_folio, fio);
 
-	if (bio && !page_is_mergeable(fio->sbi, bio, *fio->last_block,
-						fio->new_blkaddr))
+	if (bio &&
+	    (!page_is_mergeable(fio->sbi, bio, *fio->last_block,
+				fio->new_blkaddr) ||
+	     !f2fs_bio_dropbehind_mergeable(bio, fio)))
 		f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL);
 alloc_new:
 	if (!bio) {
@@ -1118,7 +1151,8 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
 	    (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,
 			      fio->new_blkaddr) ||
 	     !f2fs_crypt_mergeable_bio(io->bio, fio_inode(fio),
-				bio_folio->index, fio)))
+				bio_folio->index, fio) ||
+	     !f2fs_bio_dropbehind_mergeable(io->bio, fio)))
 		__submit_merged_bio(io);
 alloc_new:
 	if (io->bio == NULL) {
Signature · 1 line
2.43.0
#9 lore raw

Re: [PATCH v3 1/2] f2fs: complete dropbehind write bios in safe task context

Quoted 10 lines · Wenjie Qi

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer dropbehind bios to sbi->wq unless completion runs in preemptible task context.

Use an F2FS-local context check for this decision. Task context alone is not sufficient: preemption may still be disabled, or completion may run in a preemptible RCU read-side critical section.

Keep the existing large-ATC deferral unchanged.

Please reuse all the helpers added in common code in 7.3 for deferring bios and tsting if that that is neeeded instead of badly reinventing the logic. It also is really helpful to Cc people involved with the code and the relevant mailing lists.

Quoted 116 lines
---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +42 -8
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
 	struct work_struct work;
+	bool dropbehind;
 	struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+	return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION
+	if (rcu_preempt_depth())
+		return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+	return true;
+#else
+	return !preemptible();
+#endif
+}
+
 #define	F2FS_BIO_POOL_SIZE	NR_CURSEG_TYPE
 
 int __init f2fs_init_bioset(void)
@@ -426,12 +447,13 @@ static void f2fs_write_end_io(struct bio *bio)
 
 	sbi = bio->bi_private;
 
-	if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
-		struct work_struct *w;
+	if ((to_f2fs_bio(bio)->dropbehind && f2fs_bio_in_atomic()) ||
+	    (in_atomic() &&
+	     bio->bi_iter.bi_size > sbi->max_atc_write_bio_size)) {
+		struct work_struct *work = &to_f2fs_bio(bio)->work;
 
-		w = &container_of(bio, struct f2fs_bio, bio)->work;
-		INIT_WORK(w, f2fs_write_end_io_work);
-		queue_work(sbi->wq, w);
+		INIT_WORK(work, f2fs_write_end_io_work);
+		queue_work(sbi->wq, work);
 	} else {
 		f2fs_write_end_bio(bio);
 	}
@@ -530,6 +552,8 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
 	bio = bio_alloc_bioset(bdev, npages,
 				fio->op | fio->op_flags | f2fs_io_flags(fio),
 				GFP_NOIO, &f2fs_bioset);
+	to_f2fs_bio(bio)->dropbehind =
+			!is_read_io(fio->op) && folio_test_dropbehind(fio->folio);
 	bio->bi_iter.bi_sector = sector;
 	if (is_read_io(fio->op)) {
 		bio->bi_end_io = f2fs_read_end_io;
@@ -825,6 +849,13 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
 	return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
 }
 
+static bool f2fs_bio_dropbehind_mergeable(struct bio *bio,
+					  struct f2fs_io_info *fio)
+{
+	return to_f2fs_bio(bio)->dropbehind ==
+		folio_test_dropbehind(fio->folio);
+}
+
 static bool io_type_is_mergeable(struct f2fs_bio_info *io,
 						struct f2fs_io_info *fio)
 {
@@ -1017,8 +1048,10 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)
 
 	trace_f2fs_submit_folio_bio(data_folio, fio);
 
-	if (bio && !page_is_mergeable(fio->sbi, bio, *fio->last_block,
-						fio->new_blkaddr))
+	if (bio &&
+	    (!page_is_mergeable(fio->sbi, bio, *fio->last_block,
+				fio->new_blkaddr) ||
+	     !f2fs_bio_dropbehind_mergeable(bio, fio)))
 		f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL);
 alloc_new:
 	if (!bio) {
@@ -1118,7 +1151,8 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
 	    (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,
 			      fio->new_blkaddr) ||
 	     !f2fs_crypt_mergeable_bio(io->bio, fio_inode(fio),
-				bio_folio->index, fio)))
+				bio_folio->index, fio) ||
+	     !f2fs_bio_dropbehind_mergeable(io->bio, fio)))
 		__submit_merged_bio(io);
 alloc_new:
 	if (io->bio == NULL) {
Signature · 1 line
2.43.0

---end quoted text---

author
#10 lore raw

Re: [PATCH v3 1/2] f2fs: complete dropbehind write bios in safe task context

Thanks, I missed that these helpers had landed for 7.3.

I'll rebase the next revision and use BIO_COMPLETE_IN_TASK for dropbehind write bios, removing the F2FS-local context check and dropbehind-specific workqueue deferral. I'll also Cc the people and lists involved with the common code.

Quoted 135 lines · Christoph Hellwig
Quoted 10 lines · Wenjie Qi

Buffered RWF_DONTCACHE writes invalidate dropbehind folios from writeback completion. Keep normal and dropbehind folios in separate write bios, and defer dropbehind bios to sbi->wq unless completion runs in preemptible task context.

Use an F2FS-local context check for this decision. Task context alone is not sufficient: preemption may still be disabled, or completion may run in a preemptible RCU read-side critical section.

Keep the existing large-ATC deferral unchanged.

Please reuse all the helpers added in common code in 7.3 for deferring bios and tsting if that that is neeeded instead of badly reinventing the logic. It also is really helpful to Cc people involved with the code and the relevant mailing lists.

Quoted 114 lines
---
 fs/f2fs/data.c | 50 ++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 8 deletions(-)
fs/f2fs/data.c +42 -9
index 6ae0eb37d20..774a3e2d8e3 100644
@@ -21,6 +21,7 @@
 #include <linux/fiemap.h>
 #include <linux/iomap.h>
 #include <linux/fserror.h>
+#include <linux/rcupdate.h>
 
 #include "f2fs.h"
 #include "node.h"
@@ -43,9 +44,29 @@ struct f2fs_folio_state {
 
 struct f2fs_bio {
      struct work_struct work;
+     bool dropbehind;
      struct bio bio;
 };
 
+static struct f2fs_bio *to_f2fs_bio(struct bio *bio)
+{
+     return container_of(bio, struct f2fs_bio, bio);
+}
+
+/* Keep in sync with the proposed block-layer bio_in_atomic(). */
+static bool f2fs_bio_in_atomic(void)
+{
+#ifdef CONFIG_PREEMPTION
+     if (rcu_preempt_depth())
+             return true;
+#endif
+#ifndef CONFIG_PREEMPT_COUNT
+     return true;
+#else
+     return !preemptible();
+#endif
+}
+
 #define      F2FS_BIO_POOL_SIZE      NR_CURSEG_TYPE
 
 int __init f2fs_init_bioset(void)
@@ -426,12 +447,13 @@ static void f2fs_write_end_io(struct bio *bio)
 
      sbi = bio->bi_private;
 
-     if (in_atomic() && bio->bi_iter.bi_size > sbi->max_atc_write_bio_size) {
-             struct work_struct *w;
+     if ((to_f2fs_bio(bio)->dropbehind && f2fs_bio_in_atomic()) ||
+         (in_atomic() &&
+          bio->bi_iter.bi_size > sbi->max_atc_write_bio_size)) {
+             struct work_struct *work = &to_f2fs_bio(bio)->work;
 
-             w = &container_of(bio, struct f2fs_bio, bio)->work;
-             INIT_WORK(w, f2fs_write_end_io_work);
-             queue_work(sbi->wq, w);
+             INIT_WORK(work, f2fs_write_end_io_work);
+             queue_work(sbi->wq, work);
      } else {
              f2fs_write_end_bio(bio);
      }
@@ -530,6 +552,8 @@ static struct bio *__bio_alloc(struct f2fs_io_info *fio, int npages)
      bio = bio_alloc_bioset(bdev, npages,
                              fio->op | fio->op_flags | f2fs_io_flags(fio),
                              GFP_NOIO, &f2fs_bioset);
+     to_f2fs_bio(bio)->dropbehind =
+                     !is_read_io(fio->op) && folio_test_dropbehind(fio->folio);
      bio->bi_iter.bi_sector = sector;
      if (is_read_io(fio->op)) {
              bio->bi_end_io = f2fs_read_end_io;
@@ -825,6 +849,13 @@ static bool page_is_mergeable(struct f2fs_sb_info *sbi, struct bio *bio,
      return bio->bi_bdev == f2fs_target_device(sbi, cur_blkaddr, NULL);
 }
 
+static bool f2fs_bio_dropbehind_mergeable(struct bio *bio,
+                                       struct f2fs_io_info *fio)
+{
+     return to_f2fs_bio(bio)->dropbehind ==
+             folio_test_dropbehind(fio->folio);
+}
+
 static bool io_type_is_mergeable(struct f2fs_bio_info *io,
                                              struct f2fs_io_info *fio)
 {
@@ -1017,8 +1048,10 @@ int f2fs_merge_page_bio(struct f2fs_io_info *fio)
 
      trace_f2fs_submit_folio_bio(data_folio, fio);
 
-     if (bio && !page_is_mergeable(fio->sbi, bio, *fio->last_block,
-                                             fio->new_blkaddr))
+     if (bio &&
+         (!page_is_mergeable(fio->sbi, bio, *fio->last_block,
+                             fio->new_blkaddr) ||
+          !f2fs_bio_dropbehind_mergeable(bio, fio)))
              f2fs_submit_merged_ipu_write(fio->sbi, &bio, NULL);
 alloc_new:
      if (!bio) {
@@ -1118,7 +1151,8 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
          (!io_is_mergeable(sbi, io->bio, io, fio, io->last_block_in_bio,
                            fio->new_blkaddr) ||
           !f2fs_crypt_mergeable_bio(io->bio, fio_inode(fio),
-                             bio_folio->index, fio)))
+                             bio_folio->index, fio) ||
+          !f2fs_bio_dropbehind_mergeable(io->bio, fio)))
              __submit_merged_bio(io);
 alloc_new:
      if (io->bio == NULL) {
--
Quoted 2 lines

2.43.0

---end quoted text---

author
#11 lore raw

PATCH v3 2/2 f2fs: enable buffered RWF_DONTCACHE

In reply to Wenjie Qi

Pass FGP_DONTCACHE to f2fs_filemap_get_folio() for IOCB_DONTCACHE writes and advertise FOP_DONTCACHE.

Keep the F2FS-specific lookup flags because write_begin_get_folio() adds FGP_STABLE, which can deadlock here.

---
 fs/f2fs/data.c | 9 ++++++---
 fs/f2fs/file.c | 2 +-
 2 files changed, 7 insertions(+), 4 deletions(-)
fs/f2fs/data.c +6 -3
index 774a3e2d8e3..ebd9dc582f9 100644
@@ -3981,11 +3981,15 @@ static int f2fs_write_begin(const struct kiocb *iocb,
 	struct inode *inode = mapping->host;
 	struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
 	struct folio *folio;
+	fgf_t fgp_flags = FGP_LOCK | FGP_WRITE | FGP_CREAT;
 	pgoff_t index = pos >> PAGE_SHIFT;
 	bool need_balance = false;
 	block_t blkaddr = NULL_ADDR;
 	int err = 0;
 
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		fgp_flags |= FGP_DONTCACHE;
+
 	trace_f2fs_write_begin(inode, pos, len);
 
 	if (!f2fs_is_checkpoint_ready(sbi)) {
@@ -4031,9 +4035,8 @@ static int f2fs_write_begin(const struct kiocb *iocb,
 	 * Do not use FGP_STABLE to avoid deadlock.
 	 * Will wait that below with our IO control.
 	 */
-	folio = f2fs_filemap_get_folio(mapping, index,
-				FGP_LOCK | FGP_WRITE | FGP_CREAT,
-				mapping_gfp_mask(mapping));
+	folio = f2fs_filemap_get_folio(mapping, index, fgp_flags,
+				       mapping_gfp_mask(mapping));
 	if (IS_ERR(folio)) {
 		err = PTR_ERR(folio);
 		goto fail;
fs/f2fs/file.c +1 -1
index d82be8c1502..8e9d86dd42c 100644
@@ -5912,6 +5912,6 @@ const struct file_operations f2fs_file_operations = {
 	.splice_read	= f2fs_file_splice_read,
 	.splice_write	= iter_file_splice_write,
 	.fadvise	= f2fs_file_fadvise,
-	.fop_flags	= FOP_BUFFER_RASYNC,
+	.fop_flags	= FOP_BUFFER_RASYNC | FOP_DONTCACHE,
 	.setlease	= generic_setlease,
 };
Signature · 1 line
2.43.0