Skip to contentSkip to conversation

iomap: add helper to keep uptodate bitmap in sync

PATCH Superseded Joanne Koong 4 participants 10 messages Jun 24 – Jun 25

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover iomap: add helper to keep uptodate bitmap in sync

This series is on top of the main linux tree [1] because it depends on some fuse changes in Miklos's tree that were merged into 7.2 but not yet pulled into Christian's vfs tree.

The first patch in this series adds a helper to iomap that also updates the uptodate bitmap whenever a folio needs to be marked uptodate.

This is so that external callers who need to mark a folio as uptodate from a non-iomap path can do so and keep the bitmap in sync.

The second patch adds the fuse caller side of it. fuse uses this helper in the notify store path for storing server-pushed data into the page cache pages and in the writethrough path when a full folio is copied to the page cache. This series is needed before fuse can start using large folios [2].

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/
[2] https://lore.kernel.org/fuse-devel/[email protected]/

Changelog
---------
v1: https://lore.kernel.org/linux-fsdevel/[email protected]/
v1 -> v2:
* drop helper for clearing uptodate state, as per Matthew and Christoph's comments

Joanne Koong (2):
iomap: add helper to mark folio uptodate
fuse: use iomap helper to mark folio uptodate

 fs/fuse/file.c         | 2 +-
 fs/fuse/notify.c       | 4 +++-
 fs/iomap/buffered-io.c | 6 ++++++
 include/linux/iomap.h  | 1 +
 4 files changed, 11 insertions(+), 2 deletions(-)
Signature · 1 line
2.52.0
#1 lore raw

PATCH v2 1/2 iomap: add helper to mark folio uptodate

Add an exported helper iomap_folio_mark_uptodate() to mark a folio as uptodate and update its uptodate bitmap if the folio has iomap state data attached.

This is needed because there are some filesystems (eg fuse) that have paths outside of conventional iomap calls that need to mark a folio as uptodate (eg writing server-pushed data directly into the page cache) and need the iomap-internal uptodate bitmap to be in sync with the uptodate state of the folio.

---
 fs/iomap/buffered-io.c | 6 ++++++
 include/linux/iomap.h  | 1 +
 2 files changed, 7 insertions(+)
fs/iomap/buffered-io.c +6 -0
index 8d4806dc46d4..bc33c34800e8 100644
@@ -105,6 +105,12 @@ static void iomap_set_range_uptodate(struct folio *folio, size_t off,
 		folio_mark_uptodate(folio);
 }
 
+void iomap_folio_mark_uptodate(struct folio *folio)
+{
+	iomap_set_range_uptodate(folio, 0, folio_size(folio));
+}
+EXPORT_SYMBOL_GPL(iomap_folio_mark_uptodate);
+
 /*
  * Find the next dirty block in the folio. end_blk is inclusive.
  * If no dirty block is found, this will return end_blk + 1.
include/linux/iomap.h +1 -0
index 3582ed1fe236..21e73cb9c51e 100644
@@ -365,6 +365,7 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len);
 bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags);
 void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len);
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);
+void iomap_folio_mark_uptodate(struct folio *folio);
 int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,
 		const struct iomap_ops *ops,
 		const struct iomap_write_ops *write_ops);
Signature · 1 line
2.52.0
#4 lore raw

PATCH v2 2/2 fuse: use iomap helper to mark folio uptodate

In reply to Joanne Koong

When fuse enables large folios, a large folio will be backed by iomap_folio_state that keeps track of uptodate and dirty state in an internal bitmap.

Fuse writethrough and notify store paths currently set folio uptodate state with folio_mark_uptodate(), which touches only the folio-level flag, but on an iomap-backed folio, that leaves the uptodate bitmap out of sync.

Use the iomap_folio_mark_uptodate() helper to update both the folio uptodate state and the iomap uptodate bitmap.

---
 fs/fuse/file.c   | 2 +-
 fs/fuse/notify.c | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
fs/fuse/file.c +1 -1
index e052a0d44dee..26f27e113e9c 100644
@@ -1335,7 +1335,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 
 		/* If we copied full folio, mark it uptodate */
 		if (tmp == folio_size(folio))
-			folio_mark_uptodate(folio);
+			iomap_folio_mark_uptodate(folio);
 
 		if (folio_test_uptodate(folio)) {
 			folio_unlock(folio);
fs/fuse/notify.c +3 -1
index 29578104ae6c..1ba763705d91 100644
@@ -2,6 +2,8 @@
 
 #include "dev.h"
 #include "fuse_i.h"
+
+#include <linux/iomap.h>
 #include <linux/pagemap.h>
 
 static int fuse_notify_poll(struct fuse_conn *fc, unsigned int size,
@@ -192,7 +194,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 		if (!folio_test_uptodate(folio) && !err && folio_offset == 0 &&
 		    (nr_bytes == folio_size(folio) || file_size == end)) {
 			folio_zero_segment(folio, nr_bytes, folio_size(folio));
-			folio_mark_uptodate(folio);
+			iomap_folio_mark_uptodate(folio);
 		}
 		folio_unlock(folio);
 		folio_put(folio);
Signature · 1 line
2.52.0
#5 lore raw

Re: [PATCH v2 2/2] fuse: use iomap helper to mark folio uptodate

Quoted 45 lines · Joanne Koong

When fuse enables large folios, a large folio will be backed by iomap_folio_state that keeps track of uptodate and dirty state in an internal bitmap.

Fuse writethrough and notify store paths currently set folio uptodate state with folio_mark_uptodate(), which touches only the folio-level flag, but on an iomap-backed folio, that leaves the uptodate bitmap out of sync.

Use the iomap_folio_mark_uptodate() helper to update both the folio uptodate state and the iomap uptodate bitmap.

---
 fs/fuse/file.c   | 2 +-
 fs/fuse/notify.c | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)
fs/fuse/file.c +1 -1
index e052a0d44dee..26f27e113e9c 100644
@@ -1335,7 +1335,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia,
 
 		/* If we copied full folio, mark it uptodate */
 		if (tmp == folio_size(folio))
-			folio_mark_uptodate(folio);
+			iomap_folio_mark_uptodate(folio);
 
 		if (folio_test_uptodate(folio)) {
 			folio_unlock(folio);
fs/fuse/notify.c +2 -0
index 29578104ae6c..1ba763705d91 100644
@@ -2,6 +2,8 @@
 
 #include "dev.h"
 #include "fuse_i.h"
+
+#include <linux/iomap.h>
 #include <linux/pagemap.h>
 
 static int fuse_notify_poll(struct fuse_conn *fc, unsigned int size,

fs/fuse/notify.c static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, · line 197

Quoted 5 lines
fs/fuse/notify.c +1 -1
@@ -192,7 +194,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
 		if (!folio_test_uptodate(folio) && !err && folio_offset == 0 &&
 		    (nr_bytes == folio_size(folio) || file_size == end)) {
 			folio_zero_segment(folio, nr_bytes, folio_size(folio));
-			folio_mark_uptodate(folio);
+			iomap_folio_mark_uptodate(folio);

I wonder, if a fuse server stores to a range of dirty pagecache and completely over-stores the dirty range, can that folio still get scheduled for writeback? I guess it's no big deal if that writeback happens since the fuse server already knew about that content.

--D

Quoted 6 lines
fs/fuse/notify.c +0 -0
 		}
 		folio_unlock(folio);
 		folio_put(folio);
Signature · 1 line
2.52.0
#6 lore raw

Re: [PATCH v2 2/2] fuse: use iomap helper to mark folio uptodate

Quoted 15 lines · Darrick J. Wong

fs/fuse/notify.c static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, · line 197

Quoted 9 lines
fs/fuse/notify.c +1 -1
@@ -192,7 +194,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
              if (!folio_test_uptodate(folio) && !err && folio_offset == 0 &&
                  (nr_bytes == folio_size(folio) || file_size == end)) {
                      folio_zero_segment(folio, nr_bytes, folio_size(folio));
-                     folio_mark_uptodate(folio);
+                     iomap_folio_mark_uptodate(folio);

I wonder, if a fuse server stores to a range of dirty pagecache and completely over-stores the dirty range, can that folio still get scheduled for writeback? I guess it's no big deal if that writeback happens since the fuse server already knew about that content.

Yes I agree, I believe it still gets scheduled for writeback since it still has the dirty bit set.

Thanks,
Joanne

Reviewed-by
#7 lore raw

Re: [PATCH v2 2/2] fuse: use iomap helper to mark folio uptodate

Quoted 16 lines · Joanne Koong
Quoted 15 lines · Darrick J. Wong

fs/fuse/notify.c static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, · line 197

Quoted 9 lines
fs/fuse/notify.c +1 -1
@@ -192,7 +194,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size,
              if (!folio_test_uptodate(folio) && !err && folio_offset == 0 &&
                  (nr_bytes == folio_size(folio) || file_size == end)) {
                      folio_zero_segment(folio, nr_bytes, folio_size(folio));
-                     folio_mark_uptodate(folio);
+                     iomap_folio_mark_uptodate(folio);

I wonder, if a fuse server stores to a range of dirty pagecache and completely over-stores the dirty range, can that folio still get scheduled for writeback? I guess it's no big deal if that writeback happens since the fuse server already knew about that content.

Quoted 2 lines

Yes I agree, I believe it still gets scheduled for writeback since it still has the dirty bit set.

Ok, seems fine to me then. I hope there isn't a fuse server somewhere depending on the "sorta immediate writeback of what I just stored via this backchannel" behavior?

I mean, it would be crazy to implement an RNG based on jitter between dirty->store->writeback, right??

--D

#8 lore raw
In reply to Joanne Koong
Quoted 3 lines

This series is on top of the main linux tree [1] because it depends on some fuse changes in Miklos's tree that were merged into 7.2 but not yet pulled into Christian's vfs tree.

So I pushed out vfs-7.3.iomap based on current mainline. It'll be rebased once -rc1 is out ofc but so you've got something to base things one.

Signature · 1 line
Christian Brauner <[email protected]>
#9 lore raw
Quoted 8 lines · Christian Brauner
Quoted 4 lines

This series is on top of the main linux tree [1] because it depends on some fuse changes in Miklos's tree that were merged into 7.2 but not yet pulled into Christian's vfs tree.

So I pushed out vfs-7.3.iomap based on current mainline. It'll be rebased once -rc1 is out ofc but so you've got something to base things one.

Thanks, Christian! I'll base v3 on vfs-7.3.iomap