Skip to content

netfs, afs, 9p, cifs: Rework netfs to use ->writepages() to copy to cache

David Howells ceph-devel linux-cifs linux-erofs linux-fsdevel linux-mm linux-nfs lkml netdev netfs v9fs 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest22 patches 3 repliesIn review
  2. v126 patches 35 repliesApplied

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/22netfs: Update i_blocks when write committed to pagecachefs/netfs/buffered_write.c+34 -110 replies
2/22netfs: Replace PG_fscache by setting folio->private and marking dirtyfs/ceph/addr.c fs/ceph/inode.c fs/netfs/buffered_read.c +11 more+143 -970 replies
3/22mm: Remove the PG_fscache alias for PG_private_2fs/ceph/addr.c fs/netfs/buffered_read.c fs/netfs/fscache_io.c +7 more+33 -1040 replies
4/22netfs: Remove deprecated use of PG_private_2 as a second writeback flagfs/ceph/addr.c fs/netfs/buffered_read.c fs/netfs/io.c+2 -1690 replies
5/22netfs: Make netfs_io_request::subreq_counter an atomic_tfs/netfs/output.c include/linux/netfs.h+2 -20 replies
6/22netfs: Use subreq_counter to allocate subreq debug_index valuesfs/netfs/io.c fs/netfs/objects.c fs/netfs/output.c+3 -60 replies
7/22mm: Provide a means of invalidation without using launder_foliofs/netfs/direct_write.c include/linux/pagemap.h mm/filemap.c+80 -42 replies
8/229p: Use alternative invalidation to using launder_foliofs/9p/vfs_addr.c+0 -20 replies
9/22afs: Use alternative invalidation to using launder_foliofs/afs/file.c fs/afs/internal.h fs/afs/validation.c +1 more+5 -110 replies
10/22netfs: Remove ->launder_folio() supportfs/netfs/buffered_write.c fs/netfs/main.c include/linux/netfs.h +1 more+0 -800 replies
11/22netfs: Use mempools for allocating requests and subrequestsfs/netfs/internal.h fs/netfs/main.c fs/netfs/objects.c +1 more+89 -280 replies
12/22mm: Export writeback_iter()mm/page-writeback.c+1 -00 replies
13/22netfs: Switch to using unsigned long long rather than loff_tfs/cachefiles/io.c fs/ceph/addr.c fs/netfs/buffered_read.c +6 more+24 -200 replies
14/22netfs: New writeback implementationfs/netfs/Makefile fs/netfs/buffered_write.c fs/netfs/internal.h +5 more+1820 -91 reply
15/22netfs: Add some write-side stats and clean up some stat namesfs/netfs/buffered_read.c fs/netfs/buffered_write.c fs/netfs/direct_write.c +3 more+23 -90 replies
16/22netfs, afs: Implement helpers for new write codefs/afs/file.c fs/afs/internal.h fs/afs/write.c+60 -00 replies
17/22netfs, 9p: Implement helpers for new write codefs/9p/vfs_addr.c include/net/9p/client.h net/9p/Kconfig +1 more+100 -00 replies
18/22netfs, cachefiles: Implement helpers for new write codefs/cachefiles/io.c+74 -00 replies
19/22netfs: Cut over to using new writeback codefs/9p/vfs_addr.c fs/afs/file.c fs/afs/internal.h +8 more+64 -780 replies
20/22netfs: Remove the old writeback codefs/9p/vfs_addr.c fs/afs/write.c fs/netfs/buffered_write.c +1 more+0 -11800 replies
21/22netfs: Miscellaneous tidy upsfs/netfs/buffered_write.c include/trace/events/netfs.h+2 -60 replies
22/22netfs, afs: Use writeback retry to deal with alternate keysfs/afs/file.c fs/afs/internal.h fs/afs/write.c +2 more+104 -1000 replies

Reviews trailers per patch

17 of 22 patches have a Reviewed-by or Acked-by.

PatchJeff Layton
1/22R-b
2/22R-b
3/22R-b
4/22R-b
5/22R-b
6/22
7/22
8/22
9/22
10/22R-b
11/22R-b
12/22R-b
13/22R-b
14/22R-b
15/22R-b
16/22R-b
17/22R-b
18/22R-b
19/22R-b
20/22
21/22R-b
22/22R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 00/22] netfs, afs, 9p, cifs: Rework netfs to use ->writepages() to copy to cache

Hi Christian, Willy,

Here's an updated version of my netfs writeback rewrite. There haven't been any major fixes. So far...

The primary purpose of these patches is to rework the netfslib writeback implementation such that pages read from the cache are written to the cache through ->writepages(), thereby allowing the fscache page flag to be retired.

The reworking also:

(1) builds on top of the new writeback_iter() infrastructure;

(2) makes it possible to use vectored write RPCs as discontiguous streams

     of pages can be accommodated;

(3) makes it easier to do simultaneous content crypto and stream division.

(4) provides support for retrying writes and re-dividing a stream;

(5) replaces the ->launder_folio() op, so that ->writepages() is used

     instead;

(6) uses mempools to allocate the netfs_io_request and netfs_io_subrequest structs to avoid allocation failure in the writeback path.

Some code that uses the fscache page flag is retained for compatibility purposes with nfs and ceph. The code is switched to using the synonymous private_2 label instead and marked with deprecation comments. I have a separate set of patches that convert cifs to use this code.

-~-

In this new implementation, writeback_iter() is used to pump folios, progressively creating two parallel, but separate streams. Either or both streams can contain gaps, and the subrequests in each stream can be of variable size, don't need to align with each other and don't need to align with the folios. (Note that more streams can be added if we have multiple servers to duplicate data to).

Indeed, subrequests can cross folio boundaries, may cover several folios or a folio may be spanned by multiple subrequests, e.g.:

         +---+---+-----+-----+---+----------+
Folios:  |   |   |     |     |   |          |
         +---+---+-----+-----+---+----------+
           +------+------+     +----+----+
Upload:    |      |      |.....|    |    |
           +------+------+     +----+----+
         +------+------+------+------+------+
Cache:   |      |      |      |      |      |
         +------+------+------+------+------+

Data that got read from the server that needs copying to the cache is stored in folios that are marked dirty and have folio->private set to a special value.

The progressive subrequest construction permits the algorithm to be preparing both the next upload to the server and the next write to the cache whilst the previous ones are already in progress. Throttling can be applied to control the rate of production of subrequests - and, in any case, we probably want to write them to the server in ascending order, particularly if the file will be extended.

Content crypto can also be prepared at the same time as the subrequests and run asynchronously, with the prepped requests being stalled until the crypto catches up with them. This might also be useful for transport crypto, but that happens at a lower layer, so probably would be harder to pull off.

The algorithm is split into three parts:

(1) The issuer. This walks through the data, packaging it up, encrypting it and creating subrequests. The part of this that generates subrequests only deals with file positions and spans and so is usable for DIO/unbuffered writes as well as buffered writes.

(2) The collector. This asynchronously collects completed subrequests, unlocks folios, frees crypto buffers and performs any retries. This runs in a work queue so that the issuer can return to the caller for writeback (so that the VM can have its kswapd thread back) or async writes.

Collection is slightly complex as the collector has to work out where discontiguities happen in the folio list so that it doesn't try and collect folios that weren't included in the write out.

(3) The retryer. This pauses the issuer, waits for all outstanding subrequests to complete and then goes through the failed subrequests to reissue them. This may involve reprepping them (with cifs, the credits must be renegotiated and a subrequest may need splitting), and doing RMW for content crypto if there's a conflicting change on the server.

The patches can be found here also:

https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-writeback

David

Changes
=======
ver #2)
- Rebase on v6.9-rc6 to accommodate some fixes that went upstream.
- Make netfs_begin_writethrough() actually return an error.
- Move the cifs invalidation patch to the cifs-netfs branch.
- Mark writeback_iter() EXPORT_SYMBOL_GPL.
- In p9_client_write_subreq(), use 'int len' rather than 'size_t len' because of the varargs packet formatter.
- In netfs_perform_write(), don't wait for writeback if we don't need to.
- Don't do the AFS StoreData RPC inline, but keep doing it from a workqueue.
- Remove a couple of redundant checks where the second used to be inside a lock.
- Add missing linux/bio.h for BIO_MAX_VECS in cachefiles.
- Change a comma ending a statement to a semicolon.
- Make filemap_invalidate_inode() take a range.
- Make netfs_unbuffered_write_iter() use filemap_invalidate_inode().

David Howells (22):
netfs: Update i_blocks when write committed to pagecache
netfs: Replace PG_fscache by setting folio->private and marking dirty
mm: Remove the PG_fscache alias for PG_private_2
netfs: Remove deprecated use of PG_private_2 as a second writeback
flag
netfs: Make netfs_io_request::subreq_counter an atomic_t
netfs: Use subreq_counter to allocate subreq debug_index values
mm: Provide a means of invalidation without using launder_folio 9p: Use alternative invalidation to using launder_folio
afs: Use alternative invalidation to using launder_folio
netfs: Remove ->launder_folio() support
netfs: Use mempools for allocating requests and subrequests
mm: Export writeback_iter()
netfs: Switch to using unsigned long long rather than loff_t
netfs: New writeback implementation
netfs: Add some write-side stats and clean up some stat names netfs, afs: Implement helpers for new write code
netfs, 9p: Implement helpers for new write code
netfs, cachefiles: Implement helpers for new write code
netfs: Cut over to using new writeback code
netfs: Remove the old writeback code
netfs: Miscellaneous tidy ups
netfs, afs: Use writeback retry to deal with alternate keys

 fs/9p/vfs_addr.c             |  60 +--
 fs/afs/file.c                |   8 +-
 fs/afs/internal.h            |   6 +-
 fs/afs/validation.c          |   4 +-
 fs/afs/write.c               | 189 ++++----
 fs/cachefiles/io.c           |  76 +++-
 fs/ceph/addr.c               |  24 +-
 fs/ceph/inode.c              |   2 +
 fs/netfs/Makefile            |   3 +-
 fs/netfs/buffered_read.c     |  40 +-
 fs/netfs/buffered_write.c    | 825 +++--------------------------------
 fs/netfs/direct_write.c      |  56 ++-
 fs/netfs/fscache_io.c        |  14 +-
 fs/netfs/internal.h          |  55 ++-
 fs/netfs/io.c                | 155 +------
 fs/netfs/main.c              |  55 ++-
 fs/netfs/misc.c              |  10 +-
 fs/netfs/objects.c           |  81 +++-
 fs/netfs/output.c            | 478 --------------------
 fs/netfs/stats.c             |  17 +-
 fs/netfs/write_collect.c     | 808 ++++++++++++++++++++++++++++++++++
 fs/netfs/write_issue.c       | 675 ++++++++++++++++++++++++++++
 fs/nfs/file.c                |   8 +-
 fs/nfs/fscache.h             |   6 +-
 fs/nfs/write.c               |   4 +-
 fs/smb/client/file.c         |  16 +-
 include/linux/fscache.h      |  22 +-
 include/linux/netfs.h        | 196 +++++----
 include/linux/pagemap.h      |   2 +
 include/net/9p/client.h      |   2 +
 include/trace/events/netfs.h | 249 ++++++++++-
 mm/filemap.c                 |  60 ++-
 mm/page-writeback.c          |   1 +
 net/9p/Kconfig               |   1 +
 net/9p/client.c              |  49 +++
 35 files changed, 2502 insertions(+), 1755 deletions(-)
 delete mode 100644 fs/netfs/output.c
 create mode 100644 fs/netfs/write_collect.c
 create mode 100644 fs/netfs/write_issue.c