Skip to contentSkip to conversation

netfs, afs, ceph: Use folios

RFC PATCH In review David Howells 4 participants 9 messages Aug 11, 2021 – Aug 13, 2021

Subsystems (6)
lore
Share & tools
lore

David Howells

author maintainer
#0 lore raw

RFC PATCH 1/1 netfs, afs, ceph: Use folios

Convert the netfs helper library and the afs filesystem to use folios.

NOTE: This patch will also need to alter the ceph filesystem, but as that's not been done that yet, ceph will fail to build.

The patch makes two alterations to the mm headers:

(1) Fix a bug in readahead_folio() where a NULL return from __readahead_folio() will cause folio_put() to oops.

(2) Add folio_change_private() to change the private data on the folio without adjusting the page refcount or changing the flag. This assumes folio_attach_private() was already called.

Notes:

(*) Should I be using page_mapping() or page_file_mapping()?

(*) Can page_endio() be split into two separate functions, one for read
and one for write? If seems a waste of time to conditionally switch between two different branches.

(*) Is there a better way to implement afs_kill_pages() and
afs_redirty_pages()? I was previously using find_get_pages_contig() into a pagevec, but that doesn't look like it'll work with folios, so I'm now calling filemap_get_folio() a lot more - not that it matters so much, as these are failure paths.

Also, should these be moved into generic code?

(*) Can ->page_mkwrite() see which subpage of a folio got hit?

(*) afs_launder_page() has a bug in it that needs a separate patch.

(*) readahead_folio() puts the page whereas readahead_page() does not.

(*) __filemap_get_folio() should be used instead of
grab_cache_page_write_begin()? What should be done if xa_is_value() returns true on the value returned by that?

With these changes, afs passes -g quick xfstests.

---
 fs/afs/file.c              |   70 +++++----
 fs/afs/internal.h          |   46 +++---
 fs/afs/write.c             |  331 +++++++++++++++++++++------------------------
 fs/netfs/read_helper.c     |  165 +++++++++++-----------
 include/linux/netfs.h      |   12 -
 include/linux/pagemap.h    |   22 ++
 include/trace/events/afs.h |   21 +-
 7 files changed, 340 insertions(+), 327 deletions(-)
fs/afs/file.c +38 -32
index db035ae2a134..8d72ad7571e4 100644
@@ -308,23 +308,24 @@ static void afs_req_issue_op(struct netfs_read_subrequest *subreq)
 
 static int afs_symlink_readpage(struct page *page)
 {
-	struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
+	struct afs_vnode *vnode = AFS_FS_I(page_mapping(page)->host);
 	struct afs_read *fsreq;
+	struct folio *folio = page_folio(page);
 	int ret;
 
 	fsreq = afs_alloc_read(GFP_NOFS);
 	if (!fsreq)
 		return -ENOMEM;
 
-	fsreq->pos	= page->index * PAGE_SIZE;
-	fsreq->len	= PAGE_SIZE;
+	fsreq->pos	= folio_file_pos(folio);
+	fsreq->len	= folio_size(folio);;
 	fsreq->vnode	= vnode;
 	fsreq->iter	= &fsreq->def_iter;
 	iov_iter_xarray(&fsreq->def_iter, READ, &page->mapping->i_pages,
 			fsreq->pos, fsreq->len);
 
 	ret = afs_fetch_data(fsreq->vnode, fsreq);
-	page_endio(page, false, ret);
+	page_endio(&folio->page, false, ret);
 	return ret;
 }
 
@@ -348,7 +349,7 @@ static int afs_begin_cache_operation(struct netfs_read_request *rreq)
 }
 
 static int afs_check_write_begin(struct file *file, loff_t pos, unsigned len,
-				 struct page *page, void **_fsdata)
+				 struct folio *folio, void **_fsdata)
 {
 	struct afs_vnode *vnode = AFS_FS_I(file_inode(file));
 
@@ -371,10 +372,12 @@ const struct netfs_read_request_ops afs_req_ops = {
 
 static int afs_readpage(struct file *file, struct page *page)
 {
+	struct folio *folio = page_folio(page);
+
 	if (!file)
 		return afs_symlink_readpage(page);
 
-	return netfs_readpage(file, page, &afs_req_ops, NULL);
+	return netfs_readpage(file, folio, &afs_req_ops, NULL);
 }
 
 static void afs_readahead(struct readahead_control *ractl)
@@ -386,29 +389,29 @@ static void afs_readahead(struct readahead_control *ractl)
  * Adjust the dirty region of the page on truncation or full invalidation,
  * getting rid of the markers altogether if the region is entirely invalidated.
  */
-static void afs_invalidate_dirty(struct page *page, unsigned int offset,
+static void afs_invalidate_dirty(struct folio *folio, unsigned int offset,
 				 unsigned int length)
 {
-	struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
+	struct afs_vnode *vnode = AFS_FS_I(folio_mapping(folio)->host);
 	unsigned long priv;
 	unsigned int f, t, end = offset + length;
 
-	priv = page_private(page);
+	priv = (unsigned long)folio_get_private(folio);
 
 	/* we clean up only if the entire page is being invalidated */
-	if (offset == 0 && length == thp_size(page))
+	if (offset == 0 && length == folio_size(folio))
 		goto full_invalidate;
 
 	 /* If the page was dirtied by page_mkwrite(), the PTE stays writable
 	  * and we don't get another notification to tell us to expand it
 	  * again.
 	  */
-	if (afs_is_page_dirty_mmapped(priv))
+	if (afs_is_folio_dirty_mmapped(priv))
 		return;
 
 	/* We may need to shorten the dirty region */
-	f = afs_page_dirty_from(page, priv);
-	t = afs_page_dirty_to(page, priv);
+	f = afs_folio_dirty_from(folio, priv);
+	t = afs_folio_dirty_to(folio, priv);
 
 	if (t <= offset || f >= end)
 		return; /* Doesn't overlap */
@@ -426,17 +429,17 @@ static void afs_invalidate_dirty(struct page *page, unsigned int offset,
 	if (f == t)
 		goto undirty;
 
-	priv = afs_page_dirty(page, f, t);
-	set_page_private(page, priv);
-	trace_afs_page_dirty(vnode, tracepoint_string("trunc"), page);
+	priv = afs_folio_dirty(folio, f, t);
+	folio_change_private(folio, (void *)priv);
+	trace_afs_folio_dirty(vnode, tracepoint_string("trunc"), folio);
 	return;
 
 undirty:
-	trace_afs_page_dirty(vnode, tracepoint_string("undirty"), page);
-	clear_page_dirty_for_io(page);
+	trace_afs_folio_dirty(vnode, tracepoint_string("undirty"), folio);
+	folio_clear_dirty_for_io(folio);
 full_invalidate:
-	trace_afs_page_dirty(vnode, tracepoint_string("inval"), page);
-	detach_page_private(page);
+	trace_afs_folio_dirty(vnode, tracepoint_string("inval"), folio);
+	folio_detach_private(folio);
 }
 
 /*
@@ -447,14 +450,16 @@ static void afs_invalidate_dirty(struct page *page, unsigned int offset,
 static void afs_invalidatepage(struct page *page, unsigned int offset,
 			       unsigned int length)
 {
-	_enter("{%lu},%u,%u", page->index, offset, length);
+	struct folio *folio = page_folio(page);
+
+	_enter("{%lu},%u,%u", folio_index(folio), offset, length);
 
 	BUG_ON(!PageLocked(page));
 
 	if (PagePrivate(page))
-		afs_invalidate_dirty(page, offset, length);
+		afs_invalidate_dirty(folio, offset, length);
 
-	wait_on_page_fscache(page);
+	folio_wait_fscache(folio);
 	_leave("");
 }
 
@@ -464,30 +469,31 @@ static void afs_invalidatepage(struct page *page, unsigned int offset,
  */
 static int afs_releasepage(struct page *page, gfp_t gfp_flags)
 {
-	struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
+	struct folio *folio = page_folio(page);
+	struct afs_vnode *vnode = AFS_FS_I(folio_mapping(folio)->host);
 
 	_enter("{{%llx:%llu}[%lu],%lx},%x",
-	       vnode->fid.vid, vnode->fid.vnode, page->index, page->flags,
+	       vnode->fid.vid, vnode->fid.vnode, folio_index(folio), folio->flags,
 	       gfp_flags);
 
 	/* deny if page is being written to the cache and the caller hasn't
 	 * elected to wait */
 #ifdef CONFIG_AFS_FSCACHE
-	if (PageFsCache(page)) {
+	if (folio_test_fscache(folio)) {
 		if (!(gfp_flags & __GFP_DIRECT_RECLAIM) || !(gfp_flags & __GFP_FS))
 			return false;
-		wait_on_page_fscache(page);
+		folio_wait_fscache(folio);
 	}
 #endif
 
-	if (PagePrivate(page)) {
-		trace_afs_page_dirty(vnode, tracepoint_string("rel"), page);
-		detach_page_private(page);
+	if (folio_test_private(folio)) {
+		trace_afs_folio_dirty(vnode, tracepoint_string("rel"), folio);
+		folio_detach_private(folio);
 	}
 
-	/* indicate that the page can be released */
+	/* Indicate that the folio can be released */
 	_leave(" = T");
-	return 1;
+	return true;
 }
 
 /*
fs/afs/internal.h +23 -23

82 lines not on this page · raw message

fs/afs/write.c +158 -173

628 lines not on this page · raw message

fs/netfs/read_helper.c +83 -82

367 lines not on this page · raw message

include/linux/netfs.h +6 -6
index 113b5fa9280c..cb2aee4258ac 100644
@@ -166,13 +166,13 @@ struct netfs_read_request {
 	short			error;		/* 0 or error that occurred */
 	loff_t			i_size;		/* Size of the file */
 	loff_t			start;		/* Start position */
-	pgoff_t			no_unlock_page;	/* Don't unlock this page after read */
+	pgoff_t			no_unlock_folio; /* Don't unlock this folio after read */
 	refcount_t		usage;
 	unsigned long		flags;
 #define NETFS_RREQ_INCOMPLETE_IO	0	/* Some ioreqs terminated short or with error */
 #define NETFS_RREQ_WRITE_TO_CACHE	1	/* Need to write to the cache */
-#define NETFS_RREQ_NO_UNLOCK_PAGE	2	/* Don't unlock no_unlock_page on completion */
-#define NETFS_RREQ_DONT_UNLOCK_PAGES	3	/* Don't unlock the pages on completion */
+#define NETFS_RREQ_NO_UNLOCK_FOLIO	2	/* Don't unlock no_unlock_folio on completion */
+#define NETFS_RREQ_DONT_UNLOCK_FOLIOS	3	/* Don't unlock the folios on completion */
 #define NETFS_RREQ_FAILED		4	/* The request failed */
 #define NETFS_RREQ_IN_PROGRESS		5	/* Unlocked when the request completes */
 	const struct netfs_read_request_ops *netfs_ops;
@@ -190,7 +190,7 @@ struct netfs_read_request_ops {
 	void (*issue_op)(struct netfs_read_subrequest *subreq);
 	bool (*is_still_valid)(struct netfs_read_request *rreq);
 	int (*check_write_begin)(struct file *file, loff_t pos, unsigned len,
-				 struct page *page, void **_fsdata);
+				 struct folio *folio, void **_fsdata);
 	void (*done)(struct netfs_read_request *rreq);
 	void (*cleanup)(struct address_space *mapping, void *netfs_priv);
 };
@@ -240,11 +240,11 @@ extern void netfs_readahead(struct readahead_control *,
 			    const struct netfs_read_request_ops *,
 			    void *);
 extern int netfs_readpage(struct file *,
-			  struct page *,
+			  struct folio *,
 			  const struct netfs_read_request_ops *,
 			  void *);
 extern int netfs_write_begin(struct file *, struct address_space *,
-			     loff_t, unsigned int, unsigned int, struct page **,
+			     loff_t, unsigned int, unsigned int, struct folio **,
 			     void **,
 			     const struct netfs_read_request_ops *,
 			     void *);
include/linux/pagemap.h +21 -1
index 83c1a798265f..72149be4373b 100644
@@ -229,6 +229,25 @@ static inline void folio_attach_private(struct folio *folio, void *data)
 	folio_set_private(folio);
 }
 
+/**
+ * folio_change_private - Change private data on a folio.
+ * @folio: Folio to change the data on.
+ * @data: Data to set on the folio.
+ *
+ * Change the private data attached to a folio and return the old
+ * data.  The page must previously have had data attached and the data
+ * must be detached before the folio will be freed.
+ *
+ * Return: Data that was previously attached to the folio.
+ */
+static inline void *folio_change_private(struct folio *folio, void *data)
+{
+	void *old = folio_get_private(folio);
+
+	folio->private = data;
+	return old;
+}
+
 /**
  * folio_detach_private - Detach private data from a folio.
  * @folio: Folio to detach data from.
@@ -1047,7 +1066,8 @@ static inline struct folio *readahead_folio(struct readahead_control *ractl)
 {
 	struct folio *folio = __readahead_folio(ractl);
 
-	folio_put(folio);
+	if (folio)
+		folio_put(folio);
 	return folio;
 }
 
include/trace/events/afs.h +11 -10
index 3ccf591b2374..d3d8abf3f8df 100644
@@ -955,31 +955,32 @@ TRACE_EVENT(afs_dir_check_failed,
 		      __entry->vnode, __entry->off, __entry->i_size)
 	    );
 
-TRACE_EVENT(afs_page_dirty,
-	    TP_PROTO(struct afs_vnode *vnode, const char *where, struct page *page),
+TRACE_EVENT(afs_folio_dirty,
+	    TP_PROTO(struct afs_vnode *vnode, const char *where, struct folio *folio),
 
-	    TP_ARGS(vnode, where, page),
+	    TP_ARGS(vnode, where, folio),
 
 	    TP_STRUCT__entry(
 		    __field(struct afs_vnode *,		vnode		)
 		    __field(const char *,		where		)
-		    __field(pgoff_t,			page		)
+		    __field(pgoff_t,			index		)
 		    __field(unsigned long,		from		)
 		    __field(unsigned long,		to		)
 			     ),
 
 	    TP_fast_assign(
+		    unsigned long priv = (unsigned long)folio_get_private(folio);
 		    __entry->vnode = vnode;
 		    __entry->where = where;
-		    __entry->page = page->index;
-		    __entry->from = afs_page_dirty_from(page, page->private);
-		    __entry->to = afs_page_dirty_to(page, page->private);
-		    __entry->to |= (afs_is_page_dirty_mmapped(page->private) ?
-				    (1UL << (BITS_PER_LONG - 1)) : 0);
+		    __entry->index = folio_index(folio);
+		    __entry->from  = afs_folio_dirty_from(folio, priv);
+		    __entry->to    = afs_folio_dirty_to(folio, priv);
+		    __entry->to   |= (afs_is_folio_dirty_mmapped(priv) ?
+				      (1UL << (BITS_PER_LONG - 1)) : 0);
 			   ),
 
 	    TP_printk("vn=%p %lx %s %lx-%lx%s",
-		      __entry->vnode, __entry->page, __entry->where,
+		      __entry->vnode, __entry->index, __entry->where,
 		      __entry->from,
 		      __entry->to & ~(1UL << (BITS_PER_LONG - 1)),
 		      __entry->to & (1UL << (BITS_PER_LONG - 1)) ? " M" : "")
maintainer
#1 lore raw
Quoted 9 lines · David Howells

Convert the netfs helper library and the afs filesystem to use folios.

NOTE: This patch will also need to alter the ceph filesystem, but as that's not been done that yet, ceph will fail to build.

The patch makes two alterations to the mm headers:

(1) Fix a bug in readahead_folio() where a NULL return from __readahead_folio() will cause folio_put() to oops.

I'll fold that in.

Quoted 3 lines

(2) Add folio_change_private() to change the private data on the folio without adjusting the page refcount or changing the flag. This assumes folio_attach_private() was already called.

Makes sense.

Quoted 1 line

(*) Should I be using page_mapping() or page_file_mapping()?

Depends if you can have a swapfile on your filesystem. I'd like to get rid of this and only use the directIO path for swap, but that's a far-distant project.

Quoted 3 lines

(*) Can page_endio() be split into two separate functions, one for read
and one for write? If seems a waste of time to conditionally switch between two different branches.

So you'd like a folio_end_write() and folio_end_read()?

Quoted 5 lines

(*) Is there a better way to implement afs_kill_pages() and
afs_redirty_pages()? I was previously using find_get_pages_contig() into a pagevec, but that doesn't look like it'll work with folios, so I'm now calling filemap_get_folio() a lot more - not that it matters so much, as these are failure paths.

I always disliked the _contig variants. Block filesystems tend to follow the pattern

for-each-page-in-range
if page-is-contig-with-prev
append-to-bio
else
start-new-bio

while network filesystems tend to use the pattern

for-range
get-a-batch-of-contig-pages
submit-an-io-using-these-pages

it'd be nice to follow the same pattern for both. Would reduce the amount of duplicated infrastructure.

Quoted 1 line

Also, should these be moved into generic code?

I'd have to figure out what they do to answer this question.

Quoted 1 line

(*) Can ->page_mkwrite() see which subpage of a folio got hit?

It already does -- you're passed a page, not a folio. Are you trying to optimise by only marking part of a folio as dirty? If so, that's a bad idea because we're going to want to, eg, map 64KB chunks of a folio with a single TLB entry on ARM, so you'll only get one notification for that page.

Quoted 3 lines

(*) __filemap_get_folio() should be used instead of
grab_cache_page_write_begin()? What should be done if xa_is_value() returns true on the value returned by that?

If you don't pass FGP_ENTRY, it won't return you an xa_is_value() ...

#2 lore raw
In reply to David Howells
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi David,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on next-20210811]
[cannot apply to tip/perf/core linux/master linus/master v5.14-rc5 v5.14-rc4 v5.14-rc3 v5.14-rc5] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/David-Howells/netfs-afs-ceph-Use-folios/20210811-210906
base:    8ca403f3e7a23c4513046ad8d107adfbe4703362
config: xtensa-randconfig-r026-20210811 (attached as .config)
compiler: xtensa-linux-gcc (GCC) 10.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/a665390ce411c517db3f70ae59cdaa874cb914ba
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review David-Howells/netfs-afs-ceph-Use-folios/20210811-210906 git checkout a665390ce411c517db3f70ae59cdaa874cb914ba
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-10.3.0 make.cross ARCH=xtensa

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

fs/netfs/read_helper.c: In function 'netfs_rreq_unlock':

>> fs/netfs/read_helper.c:435:4: error: implicit declaration of function 'flush_dcache_folio'; did you mean 'flush_dcache_page'? [-Werror=implicit-function-declaration]
     435 |    flush_dcache_folio(folio);
         |    ^~~~~~~~~~~~~~~~~~
         |    flush_dcache_page

cc1: some warnings being treated as errors

vim +435 fs/netfs/read_helper.c

368
369 /*
370 * Unlock the folios in a read operation. We need to set PG_fscache on any
371 * folios we're going to write back before we unlock them.
372 */
373 static void netfs_rreq_unlock(struct netfs_read_request *rreq)
374 {
375 struct netfs_read_subrequest *subreq;
376 struct folio *folio;
377 unsigned int iopos, account = 0;
378 pgoff_t start_page = rreq->start / PAGE_SIZE;
379 pgoff_t last_page = ((rreq->start + rreq->len) / PAGE_SIZE) - 1;
380 bool subreq_failed = false;
381
382 XA_STATE(xas, &rreq->mapping->i_pages, start_page);
383
384 if (test_bit(NETFS_RREQ_FAILED, &rreq->flags)) {
385 __clear_bit(NETFS_RREQ_WRITE_TO_CACHE, &rreq->flags);
386 list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
387 __clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
388 }
389 }
390
391 /* Walk through the pagecache and the I/O request lists simultaneously.
392 * We may have a mixture of cached and uncached sections and we only
393 * really want to write out the uncached sections. This is slightly
394 * complicated by the possibility that we might have huge pages with a
395 * mixture inside.
396 */
397 subreq = list_first_entry(&rreq->subrequests,
398 struct netfs_read_subrequest, rreq_link);
399 iopos = 0;
400 subreq_failed = (subreq->error < 0);
401
402 trace_netfs_rreq(rreq, netfs_rreq_trace_unlock);
403
404 rcu_read_lock();
405 xas_for_each(&xas, folio, last_page) {
406 unsigned int pgpos = (folio_index(folio) - start_page) * PAGE_SIZE;
407 unsigned int pgend = pgpos + folio_size(folio);
408 bool pg_failed = false;
409
410 for (;;) {
411 if (!subreq) {
412 pg_failed = true;
413 break;
414 }
415 if (test_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags))
416 folio_start_fscache(folio);
417 pg_failed |= subreq_failed;
418 if (pgend < iopos + subreq->len)
419 break;
420
421 account += subreq->transferred;
422 iopos += subreq->len;
423 if (!list_is_last(&subreq->rreq_link, &rreq->subrequests)) {
424 subreq = list_next_entry(subreq, rreq_link);
425 subreq_failed = (subreq->error < 0);
426 } else {
427 subreq = NULL;
428 subreq_failed = false;
429 }
430 if (pgend == iopos)
431 break;
432 }
433
434 if (!pg_failed) {

 > 435				flush_dcache_folio(folio);

436 folio_mark_uptodate(folio);
437 }
438
439 if (!test_bit(NETFS_RREQ_DONT_UNLOCK_FOLIOS, &rreq->flags)) {
440 if (folio_index(folio) == rreq->no_unlock_folio &&
441 test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags))
442 _debug("no unlock");
443 else
444 folio_unlock(folio);
445 }
446 }
447 rcu_read_unlock();
448
449 task_io_account_read(account);
450 if (rreq->netfs_ops->done)
451 rreq->netfs_ops->done(rreq);
452 }
453

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#3 lore raw
In reply to David Howells
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi David,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on next-20210811]
[cannot apply to tip/perf/core linux/master linus/master v5.14-rc5 v5.14-rc4 v5.14-rc3 v5.14-rc5] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/David-Howells/netfs-afs-ceph-Use-folios/20210811-210906
base:    8ca403f3e7a23c4513046ad8d107adfbe4703362
config: sh-allmodconfig (attached as .config)
compiler: sh4-linux-gcc (GCC) 10.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/a665390ce411c517db3f70ae59cdaa874cb914ba
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review David-Howells/netfs-afs-ceph-Use-folios/20210811-210906 git checkout a665390ce411c517db3f70ae59cdaa874cb914ba
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-10.3.0 make.cross ARCH=sh

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

>> fs/ceph/addr.c:315:23: error: initialization of 'int (*)(struct file *, loff_t,  unsigned int,  struct folio *, void **)' {aka 'int (*)(struct file *, long long int,  unsigned int,  struct folio *, void **)'} from incompatible pointer type 'int (*)(struct file *, loff_t,  unsigned int,  struct page *, void **)' {aka 'int (*)(struct file *, long long int,  unsigned int,  struct page *, void **)'} [-Werror=incompatible-pointer-types]
     315 |  .check_write_begin = ceph_netfs_check_write_begin,
         |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
   fs/ceph/addr.c:315:23: note: (near initialization for 'ceph_netfs_read_ops.check_write_begin')
   fs/ceph/addr.c: In function 'ceph_readpage':
>> fs/ceph/addr.c:346:30: error: passing argument 2 of 'netfs_readpage' from incompatible pointer type [-Werror=incompatible-pointer-types]
     346 |  return netfs_readpage(file, page, &ceph_netfs_read_ops, NULL);
         |                              ^~~~
         |                              |
         |                              struct page *
   In file included from fs/ceph/addr.c:15:
   include/linux/netfs.h:243:6: note: expected 'struct folio *' but argument is of type 'struct page *'
     243 |      struct folio *,
         |      ^~~~~~~~~~~~~~
   fs/ceph/addr.c: In function 'ceph_write_begin':
>> fs/ceph/addr.c:1255:61: error: passing argument 6 of 'netfs_write_begin' from incompatible pointer type [-Werror=incompatible-pointer-types]
    1255 |  r = netfs_write_begin(file, inode->i_mapping, pos, len, 0, &page, NULL,
         |                                                             ^~~~~
         |                                                             |
         |                                                             struct page **

In file included from fs/ceph/addr.c:15:
include/linux/netfs.h:247:45: note: expected 'struct folio **' but argument is of type 'struct page **'
247 | loff_t, unsigned int, unsigned int, struct folio **,
| ^~~~~~~~~~~~~~~
cc1: some warnings being treated as errors

vim +315 fs/ceph/addr.c

49870056005ca9 Jeff Layton 2020-07-09  307  
675d4d8997ac18 Wei Yongjun 2021-05-14  308  static const struct netfs_read_request_ops ceph_netfs_read_ops = {
f0702876e152f0 Jeff Layton 2020-06-01  309  	.init_rreq		= ceph_init_rreq,
f0702876e152f0 Jeff Layton 2020-06-01  310  	.is_cache_enabled	= ceph_is_cache_enabled,
f0702876e152f0 Jeff Layton 2020-06-01  311  	.begin_cache_operation	= ceph_begin_cache_operation,
f0702876e152f0 Jeff Layton 2020-06-01  312  	.issue_op		= ceph_netfs_issue_op,
f0702876e152f0 Jeff Layton 2020-06-01  313  	.expand_readahead	= ceph_netfs_expand_readahead,
f0702876e152f0 Jeff Layton 2020-06-01  314  	.clamp_length		= ceph_netfs_clamp_length,
d801327d9500c7 Jeff Layton 2020-06-05 @315  	.check_write_begin	= ceph_netfs_check_write_begin,
49870056005ca9 Jeff Layton 2020-07-09  316  	.cleanup		= ceph_readahead_cleanup,
f0702876e152f0 Jeff Layton 2020-06-01  317  };
f0702876e152f0 Jeff Layton 2020-06-01  318  
f0702876e152f0 Jeff Layton 2020-06-01  319  /* read a single page, without unlocking it. */
f0702876e152f0 Jeff Layton 2020-06-01  320  static int ceph_readpage(struct file *file, struct page *page)
f0702876e152f0 Jeff Layton 2020-06-01  321  {
f0702876e152f0 Jeff Layton 2020-06-01  322  	struct inode *inode = file_inode(file);
f0702876e152f0 Jeff Layton 2020-06-01  323  	struct ceph_inode_info *ci = ceph_inode(inode);
f0702876e152f0 Jeff Layton 2020-06-01  324  	struct ceph_vino vino = ceph_vino(inode);
f0702876e152f0 Jeff Layton 2020-06-01  325  	u64 off = page_offset(page);
8ff2d290c8ce77 Jeff Layton 2021-04-05  326  	u64 len = thp_size(page);
f0702876e152f0 Jeff Layton 2020-06-01  327  
f0702876e152f0 Jeff Layton 2020-06-01  328  	if (ci->i_inline_version != CEPH_INLINE_NONE) {
f0702876e152f0 Jeff Layton 2020-06-01  329  		/*
f0702876e152f0 Jeff Layton 2020-06-01  330  		 * Uptodate inline data should have been added
f0702876e152f0 Jeff Layton 2020-06-01  331  		 * into page cache while getting Fcr caps.
f0702876e152f0 Jeff Layton 2020-06-01  332  		 */
f0702876e152f0 Jeff Layton 2020-06-01  333  		if (off == 0) {
f0702876e152f0 Jeff Layton 2020-06-01  334  			unlock_page(page);
f0702876e152f0 Jeff Layton 2020-06-01  335  			return -EINVAL;
f0702876e152f0 Jeff Layton 2020-06-01  336  		}
8ff2d290c8ce77 Jeff Layton 2021-04-05  337  		zero_user_segment(page, 0, thp_size(page));
f0702876e152f0 Jeff Layton 2020-06-01  338  		SetPageUptodate(page);
f0702876e152f0 Jeff Layton 2020-06-01  339  		unlock_page(page);
f0702876e152f0 Jeff Layton 2020-06-01  340  		return 0;
f0702876e152f0 Jeff Layton 2020-06-01  341  	}
f0702876e152f0 Jeff Layton 2020-06-01  342  
f0702876e152f0 Jeff Layton 2020-06-01  343  	dout("readpage ino %llx.%llx file %p off %llu len %llu page %p index %lu\n",
f0702876e152f0 Jeff Layton 2020-06-01  344  	     vino.ino, vino.snap, file, off, len, page, page->index);
f0702876e152f0 Jeff Layton 2020-06-01  345  

f0702876e152f0 Jeff Layton 2020-06-01 @346 return netfs_readpage(file, page, &ceph_netfs_read_ops, NULL);
f0702876e152f0 Jeff Layton 2020-06-01 347 }
f0702876e152f0 Jeff Layton 2020-06-01 348

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

author maintainer
#4 lore raw

Re: [RFC][PATCH] afs: Use folios in directory handling

In reply to David Howells

Convert the AFS directory handling code to use folios.

Notes:

(*) Is it permissible to hold two concurrent kmap_local_folio() mappings
on the same folio?

(*) When modifying an AFS directory, I cache a local copy of it and edit
it locally according to the prescribed rules rather than downloading it again. If I need to grow the directory, is it better to discard all of the constituent pages and redownload so as to be able to have a better chance of packing the content into larger folios?

With these changes, afs passes -g quick xfstests.

cc Matthew Wilcox (Oracle) <[email protected]>
cc Jeff Layton <[email protected]>
cc Marc Dionne <[email protected]>
---
 fs/afs/dir.c      |  229 ++++++++++++++++++++++--------------------------------
 fs/afs/dir_edit.c |  154 ++++++++++++++++++------------------
 2 files changed, 174 insertions(+), 209 deletions(-)
fs/afs/dir.c +95 -134
index 78719f2f567e..db5bc874bf0b 100644
@@ -103,13 +103,13 @@ struct afs_lookup_cookie {
 };
 
 /*
- * Drop the refs that we're holding on the pages we were reading into.  We've
+ * Drop the refs that we're holding on the folios we were reading into.  We've
  * got refs on the first nr_pages pages.
  */
 static void afs_dir_read_cleanup(struct afs_read *req)
 {
 	struct address_space *mapping = req->vnode->vfs_inode.i_mapping;
-	struct page *page;
+	struct folio *folio;
 	pgoff_t last = req->nr_pages - 1;
 
 	XA_STATE(xas, &mapping->i_pages, 0);
@@ -118,65 +118,56 @@ static void afs_dir_read_cleanup(struct afs_read *req)
 		return;
 
 	rcu_read_lock();
-	xas_for_each(&xas, page, last) {
-		if (xas_retry(&xas, page))
+	xas_for_each(&xas, folio, last) {
+		if (xas_retry(&xas, folio))
 			continue;
-		BUG_ON(xa_is_value(page));
-		BUG_ON(PageCompound(page));
-		ASSERTCMP(page->mapping, ==, mapping);
+		BUG_ON(xa_is_value(folio));
+		ASSERTCMP(folio_mapping(folio), ==, mapping);
 
-		put_page(page);
+		folio_put(folio);
 	}
 
 	rcu_read_unlock();
 }
 
 /*
- * check that a directory page is valid
+ * check that a directory folio is valid
  */
-static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
-			       loff_t i_size)
+static bool afs_dir_check_folio(struct afs_vnode *dvnode, struct folio *folio,
+				loff_t i_size)
 {
-	struct afs_xdr_dir_page *dbuf;
-	loff_t latter, off;
-	int tmp, qty;
+	union afs_xdr_dir_block *block;
+	size_t offset, size;
+	loff_t pos;
 
-	/* Determine how many magic numbers there should be in this page, but
+	/* Determine how many magic numbers there should be in this folio, but
 	 * we must take care because the directory may change size under us.
 	 */
-	off = page_offset(page);
-	if (i_size <= off)
+	pos = folio_pos(folio);
+	if (i_size <= pos)
 		goto checked;
 
-	latter = i_size - off;
-	if (latter >= PAGE_SIZE)
-		qty = PAGE_SIZE;
-	else
-		qty = latter;
-	qty /= sizeof(union afs_xdr_dir_block);
-
-	/* check them */
-	dbuf = kmap_atomic(page);
-	for (tmp = 0; tmp < qty; tmp++) {
-		if (dbuf->blocks[tmp].hdr.magic != AFS_DIR_MAGIC) {
-			printk("kAFS: %s(%lx): bad magic %d/%d is %04hx\n",
-			       __func__, dvnode->vfs_inode.i_ino, tmp, qty,
-			       ntohs(dbuf->blocks[tmp].hdr.magic));
-			trace_afs_dir_check_failed(dvnode, off, i_size);
-			kunmap(page);
+	size = min_t(loff_t, folio_size(folio), i_size - pos);
+	for (offset = 0; offset < size; offset += sizeof(*block)) {
+		block = kmap_local_folio(folio, offset);
+		if (block->hdr.magic != AFS_DIR_MAGIC) {
+			printk("kAFS: %s(%lx): [%llx] bad magic %zx/%zx is %04hx\n",
+			       __func__, dvnode->vfs_inode.i_ino,
+			       pos, offset, size, ntohs(block->hdr.magic));
+			trace_afs_dir_check_failed(dvnode, pos + offset, i_size);
+			kunmap_local(block);
 			trace_afs_file_error(dvnode, -EIO, afs_file_error_dir_bad_magic);
 			goto error;
 		}
 
 		/* Make sure each block is NUL terminated so we can reasonably
-		 * use string functions on it.  The filenames in the page
+		 * use string functions on it.  The filenames in the folio
 		 * *should* be NUL-terminated anyway.
 		 */
-		((u8 *)&dbuf->blocks[tmp])[AFS_DIR_BLOCK_SIZE - 1] = 0;
-	}
-
-	kunmap_atomic(dbuf);
+		((u8 *)block)[AFS_DIR_BLOCK_SIZE - 1] = 0;
 
+		kunmap_local(block);
+	}
 checked:
 	afs_stat_v(dvnode, n_read_dir);
 	return true;
@@ -190,11 +181,11 @@ static bool afs_dir_check_page(struct afs_vnode *dvnode, struct page *page,
  */
 static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
 {
-	struct afs_xdr_dir_page *dbuf;
+	union afs_xdr_dir_block *block;
 	struct address_space *mapping = dvnode->vfs_inode.i_mapping;
-	struct page *page;
-	unsigned int i, qty = PAGE_SIZE / sizeof(union afs_xdr_dir_block);
+	struct folio *folio;
 	pgoff_t last = req->nr_pages - 1;
+	size_t offset, size;
 
 	XA_STATE(xas, &mapping->i_pages, 0);
 
@@ -205,30 +196,28 @@ static void afs_dir_dump(struct afs_vnode *dvnode, struct afs_read *req)
 		req->pos, req->nr_pages,
 		req->iter->iov_offset,  iov_iter_count(req->iter));
 
-	xas_for_each(&xas, page, last) {
-		if (xas_retry(&xas, page))
+	xas_for_each(&xas, folio, last) {
+		if (xas_retry(&xas, folio))
 			continue;
 
-		BUG_ON(PageCompound(page));
-		BUG_ON(page->mapping != mapping);
-
-		dbuf = kmap_atomic(page);
-		for (i = 0; i < qty; i++) {
-			union afs_xdr_dir_block *block = &dbuf->blocks[i];
+		BUG_ON(folio_mapping(folio) != mapping);
 
-			pr_warn("[%02lx] %32phN\n", page->index * qty + i, block);
+		size = min_t(loff_t, folio_size(folio), req->actual_len - folio_pos(folio));
+		for (offset = 0; offset < size; offset += sizeof(*block)) {
+			block = kmap_local_folio(folio, offset);
+			pr_warn("[%02lx] %32phN\n", folio_index(folio) + offset, block);
+			kunmap_local(block);
 		}
-		kunmap_atomic(dbuf);
 	}
 }
 
 /*
- * Check all the pages in a directory.  All the pages are held pinned.
+ * Check all the blocks in a directory.  All the folios are held pinned.
  */
 static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
 {
 	struct address_space *mapping = dvnode->vfs_inode.i_mapping;
-	struct page *page;
+	struct folio *folio;
 	pgoff_t last = req->nr_pages - 1;
 	int ret = 0;
 
@@ -238,14 +227,13 @@ static int afs_dir_check(struct afs_vnode *dvnode, struct afs_read *req)
 		return 0;
 
 	rcu_read_lock();
-	xas_for_each(&xas, page, last) {
-		if (xas_retry(&xas, page))
+	xas_for_each(&xas, folio, last) {
+		if (xas_retry(&xas, folio))
 			continue;
 
-		BUG_ON(PageCompound(page));
-		BUG_ON(page->mapping != mapping);
+		BUG_ON(folio_mapping(folio) != mapping);
 
-		if (!afs_dir_check_page(dvnode, page, req->file_size)) {
+		if (!afs_dir_check_folio(dvnode, folio, req->actual_len)) {
 			afs_dir_dump(dvnode, req);
 			ret = -EIO;
 			break;
@@ -274,15 +262,16 @@ static int afs_dir_open(struct inode *inode, struct file *file)
 
 /*
  * Read the directory into the pagecache in one go, scrubbing the previous
- * contents.  The list of pages is returned, pinning them so that they don't
+ * contents.  The list of folios is returned, pinning them so that they don't
  * get reclaimed during the iteration.
  */
 static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
 	__acquires(&dvnode->validate_lock)
 {
+	struct address_space *mapping = dvnode->vfs_inode.i_mapping;
 	struct afs_read *req;
 	loff_t i_size;
-	int nr_pages, i, n;
+	int nr_pages, i;
 	int ret;
 
 	_enter("");
@@ -320,43 +309,30 @@ static struct afs_read *afs_read_dir(struct afs_vnode *dvnode, struct key *key)
 	req->iter = &req->def_iter;
 
 	/* Fill in any gaps that we might find where the memory reclaimer has
-	 * been at work and pin all the pages.  If there are any gaps, we will
+	 * been at work and pin all the folios.  If there are any gaps, we will
 	 * need to reread the entire directory contents.
 	 */
 	i = req->nr_pages;
 	while (i < nr_pages) {
-		struct page *pages[8], *page;
-
-		n = find_get_pages_contig(dvnode->vfs_inode.i_mapping, i,
-					  min_t(unsigned int, nr_pages - i,
-						ARRAY_SIZE(pages)),
-					  pages);
-		_debug("find %u at %u/%u", n, i, nr_pages);
-
-		if (n == 0) {
-			gfp_t gfp = dvnode->vfs_inode.i_mapping->gfp_mask;
+		struct folio *folio;
 
+		folio = filemap_get_folio(mapping, i);
+		if (!folio) {
 			if (test_and_clear_bit(AFS_VNODE_DIR_VALID, &dvnode->flags))
 				afs_stat_v(dvnode, n_inval);
 
 			ret = -ENOMEM;
-			page = __page_cache_alloc(gfp);
-			if (!page)
+			folio = __filemap_get_folio(mapping,
+						    i, FGP_LOCK | FGP_CREAT,
+						    mapping->gfp_mask);
+			if (!folio)
 				goto error;
-			ret = add_to_page_cache_lru(page,
-						    dvnode->vfs_inode.i_mapping,
-						    i, gfp);
-			if (ret < 0)
-				goto error;
-
-			attach_page_private(page, (void *)1);
-			unlock_page(page);
-			req->nr_pages++;
-			i++;
-		} else {
-			req->nr_pages += n;
-			i += n;
+			folio_attach_private(folio, (void *)1);
+			folio_unlock(folio);
 		}
+
+		req->nr_pages += folio_nr_pages(folio);
+		i += folio_nr_pages(folio);
 	}
 
 	/* If we're going to reload, we need to lock all the pages to prevent
@@ -424,7 +400,7 @@ static int afs_dir_iterate_block(struct afs_vnode *dvnode,
 	size_t nlen;
 	int tmp;
 
-	_enter("%u,%x,%p,,",(unsigned)ctx->pos,blkoff,block);
+	_enter("%llx,%x", ctx->pos, blkoff);
 
 	curr = (ctx->pos - blkoff) / sizeof(union afs_xdr_dirent);
 
@@ -513,12 +489,10 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
 			   struct key *key, afs_dataversion_t *_dir_version)
 {
 	struct afs_vnode *dvnode = AFS_FS_I(dir);
-	struct afs_xdr_dir_page *dbuf;
 	union afs_xdr_dir_block *dblock;
 	struct afs_read *req;
-	struct page *page;
-	unsigned blkoff, limit;
-	void __rcu **slot;
+	struct folio *folio;
+	unsigned offset, size;
 	int ret;
 
 	_enter("{%lu},%u,,", dir->i_ino, (unsigned)ctx->pos);
@@ -540,43 +514,30 @@ static int afs_dir_iterate(struct inode *dir, struct dir_context *ctx,
 	/* walk through the blocks in sequence */
 	ret = 0;
 	while (ctx->pos < req->actual_len) {
-		blkoff = ctx->pos & ~(sizeof(union afs_xdr_dir_block) - 1);
-
-		/* Fetch the appropriate page from the directory and re-add it
+		/* Fetch the appropriate folio from the directory and re-add it
 		 * to the LRU.  We have all the pages pinned with an extra ref.
 		 */
-		rcu_read_lock();
-		page = NULL;
-		slot = radix_tree_lookup_slot(&dvnode->vfs_inode.i_mapping->i_pages,
-					      blkoff / PAGE_SIZE);
-		if (slot)
-			page = radix_tree_deref_slot(slot);
-		rcu_read_unlock();
-		if (!page) {
+		folio = __filemap_get_folio(dir->i_mapping, ctx->pos / PAGE_SIZE,
+					    FGP_ACCESSED, 0);
+		if (!folio) {
 			ret = afs_bad(dvnode, afs_file_error_dir_missing_page);
 			break;
 		}
-		mark_page_accessed(page);
 
-		limit = blkoff & ~(PAGE_SIZE - 1);
+		offset = round_down(ctx->pos, sizeof(*dblock)) - folio_file_pos(folio);
+		size = min_t(loff_t, folio_size(folio),
+			     req->actual_len - folio_file_pos(folio));
 
-		dbuf = kmap(page);
-
-		/* deal with the individual blocks stashed on this page */
 		do {
-			dblock = &dbuf->blocks[(blkoff % PAGE_SIZE) /
-					       sizeof(union afs_xdr_dir_block)];
-			ret = afs_dir_iterate_block(dvnode, ctx, dblock, blkoff);
-			if (ret != 1) {
-				kunmap(page);
+			dblock = kmap_local_folio(folio, offset);
+			ret = afs_dir_iterate_block(dvnode, ctx, dblock,
+						    folio_file_pos(folio) + offset);
+			kunmap_local(dblock);
+			if (ret != 1)
 				goto out;
-			}
 
-			blkoff += sizeof(union afs_xdr_dir_block);
+		} while (offset += sizeof(*dblock), offset < size);
 
-		} while (ctx->pos < dir->i_size && blkoff < limit);
-
-		kunmap(page);
 		ret = 0;
 	}
 
@@ -2056,42 +2017,42 @@ static int afs_rename(struct user_namespace *mnt_userns, struct inode *old_dir,
 }
 
 /*
- * Release a directory page and clean up its private state if it's not busy
- * - return true if the page can now be released, false if not
+ * Release a directory folio and clean up its private state if it's not busy
+ * - return true if the folio can now be released, false if not
  */
-static int afs_dir_releasepage(struct page *page, gfp_t gfp_flags)
+static int afs_dir_releasepage(struct page *subpage, gfp_t gfp_flags)
 {
-	struct afs_vnode *dvnode = AFS_FS_I(page->mapping->host);
+	struct folio *folio = page_folio(subpage);
+	struct afs_vnode *dvnode = AFS_FS_I(folio_mapping(folio)->host);
 
-	_enter("{{%llx:%llu}[%lu]}", dvnode->fid.vid, dvnode->fid.vnode, page->index);
+	_enter("{{%llx:%llu}[%lu]}", dvnode->fid.vid, dvnode->fid.vnode, folio_index(folio));
 
-	detach_page_private(page);
+	folio_detach_private(folio);
 
 	/* The directory will need reloading. */
 	if (test_and_clear_bit(AFS_VNODE_DIR_VALID, &dvnode->flags))
 		afs_stat_v(dvnode, n_relpg);
-	return 1;
+	return true;
 }
 
 /*
- * invalidate part or all of a page
- * - release a page and clean up its private data if offset is 0 (indicating
- *   the entire page)
+ * Invalidate part or all of a folio.
  */
-static void afs_dir_invalidatepage(struct page *page, unsigned int offset,
+static void afs_dir_invalidatepage(struct page *subpage, unsigned int offset,
 				   unsigned int length)
 {
-	struct afs_vnode *dvnode = AFS_FS_I(page->mapping->host);
+	struct folio *folio = page_folio(subpage);
+	struct afs_vnode *dvnode = AFS_FS_I(folio_mapping(folio)->host);
 
-	_enter("{%lu},%u,%u", page->index, offset, length);
+	_enter("{%lu},%u,%u", folio_index(folio), offset, length);
 
-	BUG_ON(!PageLocked(page));
+	BUG_ON(!folio_test_locked(folio));
 
 	/* The directory will need reloading. */
 	if (test_and_clear_bit(AFS_VNODE_DIR_VALID, &dvnode->flags))
 		afs_stat_v(dvnode, n_inval);
 
-	/* we clean up only if the entire page is being invalidated */
-	if (offset == 0 && length == thp_size(page))
-		detach_page_private(page);
+	/* we clean up only if the entire folio is being invalidated */
+	if (offset == 0 && length == folio_size(folio))
+		folio_detach_private(folio);
 }
fs/afs/dir_edit.c +79 -75

291 lines not on this page · raw message

maintainer
#5 lore raw
In reply to David Howells
Quoted 3 lines · David Howells

(*) Can page_endio() be split into two separate functions, one for read
and one for write? If seems a waste of time to conditionally switch between two different branches.

At this point I'm thinking ...

static inline void folio_end_read(struct folio *folio, int err)

{
	if (!err)
		folio_set_uptodate(folio);
	folio_unlock(folio);
}

Clearly the page isn't uptodate at this point, or ->readpage wouldn't've been called. So there's no need to clear it. And PageError is completely useless.

Part of this exercise was to find gaps in the API. It looks like there aren't too many places I missed that AFS hits.

fs/afs/write.c int afs_write_begin(struct file *file, struct address_space *mapping, · line 81

Quoted 6 lines
fs/afs/write.c +1 -1
@@ -78,7 +78,7 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
 			goto flush_conflicting_write;
 	}
 
-	*_page = page;
+	*_page = &folio->page;

Can't do anything about this one; the write_begin API needs to be fixed.

fs/afs/write.c int afs_write_begin(struct file *file, struct address_space *mapping, · line 90

Quoted 6 lines
fs/afs/write.c +1 -1
@@ -87,17 +87,17 @@ int afs_write_begin(struct file *file, struct address_space *mapping,
 	 */
 flush_conflicting_write:
 	_debug("flush conflict");
-	ret = write_one_page(page);
+	ret = write_one_page(&folio->page);

I have folio_write_one() in my tree:

https://git.infradead.org/users/willy/pagecache.git/commitdiff/82b9f3c7b258de31bf3d3fa4cc587a6d17b5fe40

it's in the "nice to have", rather than "filesystems depend on it" pile, so I should move it over.

Quoted 1 line
(quoted diff) +0 -0
@@ -174,40 +175,32 @@ static void afs_kill_pages(struct address_space *mapping,

[...]

fs/afs/write.c static void afs_kill_pages(struct address_space *mapping, · line 201

Quoted 6 lines
fs/afs/write.c +6 -0
+		folio_clear_uptodate(folio);
+		folio_end_writeback(folio);
+		folio_lock(folio);
+		generic_error_remove_page(mapping, &folio->page);
+		folio_unlock(folio);
+		folio_put(folio);

This one I'm entirely missing. It's awkward. I'll work on it.

fs/afs/write.c static void afs_extend_writeback(struct address_space *mapping, · line 484

Quoted 8 lines
fs/afs/write.c +2 -2
@@ -497,8 +480,8 @@ static void afs_extend_writeback(struct address_space *mapping,
 			else if (t == psize || new_content)
 				stop = false;
 
-			index += thp_nr_pages(page);
-			if (!pagevec_add(&pvec, page))
+			index += folio_nr_pages(folio);
+			if (!pagevec_add(&pvec, &folio->page))

Pagevecs are also awkward. I haven't quite figured out how to transition them to folios.

Quoted 16 lines
fs/afs/write.c +3 -7
@@ -933,29 +919,28 @@ int afs_launder_page(struct page *page)
 	unsigned int f, t;
 	int ret = 0;
 
-	_enter("{%lx}", page->index);
+	_enter("{%lx}", folio_index(folio));
 
-	priv = page_private(page);
-	if (clear_page_dirty_for_io(page)) {
+	priv = (unsigned long)folio_get_private(folio);
+	if (folio_clear_dirty_for_io(folio)) {
 		f = 0;
-		t = thp_size(page);
-		if (PagePrivate(page)) {
-			f = afs_page_dirty_from(page, priv);
-			t = afs_page_dirty_to(page, priv);

fs/afs/write.c int afs_launder_page(struct page *page) · line 935

Quoted 10 lines
fs/afs/write.c +5 -1
+		t = folio_size(folio);
+		if (folio_test_private(folio)) {
+			f = afs_folio_dirty_from(folio, priv);
+			t = afs_folio_dirty_to(folio, priv);
 		}
 
-		bv[0].bv_page = page;
+		bv[0].bv_page = &folio->page;
 		bv[0].bv_offset = f;
 		bv[0].bv_len = t - f;

I should probably add a wrapper to init a bvec.

fs/netfs/read_helper.c int netfs_readpage(struct file *file, · line 1045

Quoted 3 lines
fs/netfs/read_helper.c +1 -1
 zero_out:
-	zero_user_segments(page, 0, offset, offset + len, thp_size(page));
+	zero_user_segments(&folio->page, 0, offset, offset + len, folio_size(folio));

Yeah, that's ugly.

author maintainer
#6 lore raw
Quoted 13 lines · Matthew Wilcox
Quoted 3 lines

(*) Can page_endio() be split into two separate functions, one for read
and one for write? If seems a waste of time to conditionally switch between two different branches.

At this point I'm thinking ...

static inline void folio_end_read(struct folio *folio, int err)

{
	if (!err)
		folio_set_uptodate(folio);
	folio_unlock(folio);
}
Quoted 3 lines

Clearly the page isn't uptodate at this point, or ->readpage wouldn't've been called. So there's no need to clear it. And PageError is completely useless.

Seems reasonable.

Quoted 4 lines

fs/afs/write.c int afs_write_begin(struct file *file, struct address_space *mapping, · line 81

Quoted 2 lines
fs/afs/write.c +1 -1
-	*_page = page;
+	*_page = &folio->page;

Can't do anything about this one; the write_begin API needs to be fixed.

That's fine. I expected things like this at this stage.

Quoted 10 lines
Quoted 1 line
(quoted diff) +0 -0
@@ -174,40 +175,32 @@ static void afs_kill_pages(struct address_space *mapping,

[...]

fs/afs/write.c static void afs_kill_pages(struct address_space *mapping, · line 201

Quoted 6 lines
fs/afs/write.c +6 -0
+		folio_clear_uptodate(folio);
+		folio_end_writeback(folio);
+		folio_lock(folio);
+		generic_error_remove_page(mapping, &folio->page);
+		folio_unlock(folio);
+		folio_put(folio);

This one I'm entirely missing. It's awkward. I'll work on it.

afs_kill_pages() is just a utility to end writeback, clear uptodate and do generic_error_remove_page() over a range of pages and afs_redirty_pages() is a utility that to end writeback and redirty a range of pages - hence why I was thinking it might make sense to put them into common code.

Quoted 7 lines

fs/afs/write.c static void afs_extend_writeback(struct address_space *mapping, · line 484

Quoted 4 lines
fs/afs/write.c +2 -2
-			index += thp_nr_pages(page);
-			if (!pagevec_add(&pvec, page))
+			index += folio_nr_pages(folio);
+			if (!pagevec_add(&pvec, &folio->page))

Pagevecs are also awkward. I haven't quite figured out how to transition them to folios.

Maybe provide pagevec_add_folio(struct pagevec *, struct folio *)?

Quoted 5 lines

fs/netfs/read_helper.c int netfs_readpage(struct file *file, · line 1045

Quoted 3 lines

zero_out:

fs/netfs/read_helper.c +1 -1
-	zero_user_segments(page, 0, offset, offset + len, thp_size(page));
+	zero_user_segments(&folio->page, 0, offset, offset + len, folio_size(folio));

Yeah, that's ugly.

Maybe:

	folio_clear_around(folio, keep_from, keep_to);

clearing the bits of the folio outside the specified section?

David

#7 lore raw
In reply to Matthew Wilcox
Quoted 14 lines · Matthew Wilcox
Quoted 3 lines · David Howells

(*) Can page_endio() be split into two separate functions, one for read
and one for write? If seems a waste of time to conditionally switch between two different branches.

At this point I'm thinking ...

static inline void folio_end_read(struct folio *folio, int err)

{
	if (!err)
		folio_set_uptodate(folio);
	folio_unlock(folio);
}
Quoted 3 lines

Clearly the page isn't uptodate at this point, or ->readpage wouldn't've been called. So there's no need to clear it. And PageError is completely useless.

Just opencoding the above makes a lot more sense. No need to turn err into some acceptable form, and trivial to follow. Not all little convenience helpers are good.

Quoted 6 lines

fs/afs/write.c int afs_write_begin(struct file *file, struct address_space *mapping, · line 81

Quoted 4 lines
fs/afs/write.c +1 -1
 	}
 
-	*_page = page;
+	*_page = &folio->page;

Can't do anything about this one; the write_begin API needs to be fixed.

It actually needs to go away. There's not real good use for that level of API. netfs should just open code the releavant parts of generic_perform_write, similar to iomap.

author maintainer
#8 lore raw
Quoted 3 lines · Christoph Hellwig

It actually needs to go away. There's not real good use for that level of API. netfs should just open code the releavant parts of generic_perform_write, similar to iomap.

I'm working on doing that in netfs lib, with the intent of sharing it between at least afs, ceph, cifs and 9p. It reduces the cost of accessing fscache for large writes that span multiple pages.

David