Skip to content

netfs: Read/write improvements

David Howells ceph-devel linux-cifs linux-erofs linux-fsdevel linux-mm linux-nfs lkml netdev netfs stable v9fs 1 version Applied

Status from this message: Applied to the vfs.fixes branch of the vfs/vfs.git tree.

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest24 patches 6 repliesApplied

Patches in v1 Applied

#Subject and filesSizeRepliesApplied
1/24fs/netfs/fscache_cookie: add missing "n_accesses" checkfs/netfs/fscache_cookie.c+4 -00 replies
2/24cachefiles: Fix non-taking of sb_writers around set/removexattrfs/cachefiles/xattr.c+26 -80 replies
3/24netfs: Adjust labels in /proc/fs/netfs/statsfs/netfs/stats.c+8 -80 replies
4/24netfs: Record contention stats for writeback lockfs/netfs/internal.h fs/netfs/stats.c fs/netfs/write_issue.c+14 -30 replies
5/24netfs: Reduce number of conditional branches in netfs_perform_write()fs/netfs/buffered_write.c include/trace/events/netfs.h+134 -1670 replies
6/24netfs, cifs: Move CIFS_INO_MODIFIED_ATTR to netfs_inodefs/netfs/buffered_write.c fs/smb/client/cifsglob.h fs/smb/client/file.c +1 more+10 -110 replies
7/24netfs: Move max_len/max_nr_segs from netfs_io_subrequest to netfs_io_streamfs/afs/write.c fs/cachefiles/io.c fs/netfs/io.c +4 more+30 -260 replies
8/24netfs: Reserve netfs_sreq_source 0 as unset/unknowninclude/linux/netfs.h include/trace/events/netfs.h+2 -00 replies
9/24netfs: Remove NETFS_COPY_TO_CACHEfs/netfs/main.c include/linux/netfs.h include/trace/events/netfs.h+3 -60 replies
10/24netfs: Set the request work function upon allocationfs/netfs/internal.h fs/netfs/io.c fs/netfs/objects.c +1 more+10 -50 replies
11/24netfs: Use bh-disabling spinlocks for rreq->lockfs/netfs/write_collect.c fs/netfs/write_issue.c+4 -40 replies
12/24mm: Define struct folio_queue and ITER_FOLIOQ to handle a sequence of foliosinclude/linux/folio_queue.h include/linux/iov_iter.h include/linux/uio.h +3 more+760 -40 replies
13/24cifs: Provide the capability to extract from ITER_FOLIOQ to RDMA SGEsfs/smb/client/smbdirect.c+68 -30 replies
14/24netfs: Use new folio_queue data type and iterator instead of xarray iterfs/netfs/internal.h fs/netfs/misc.c fs/netfs/objects.c +5 more+150 -610 replies
15/24netfs: Provide an iterator-reset functionfs/netfs/internal.h fs/netfs/misc.c fs/netfs/write_collect.c +1 more+23 -80 replies
16/24netfs: Simplify the writeback codefs/netfs/write_collect.c fs/netfs/write_issue.c include/linux/netfs.h +1 more+45 -1710 replies
17/24afs: Make read subreqs asyncfs/afs/file.c+8 -10 replies
18/24netfs: Speed up buffered readingfs/9p/vfs_addr.c fs/afs/file.c fs/afs/fsclient.c +19 more+1707 -3374 replies
19/24netfs: Remove fs/netfs/io.cfs/netfs/io.c+0 -6450 replies
20/24cachefiles, netfs: Fix write to partial block at EOFfs/cachefiles/io.c fs/netfs/write_issue.c include/linux/netfs.h+17 -40 replies
21/24netfs: Cancel dirty folios that have no storage destinationfs/netfs/write_issue.c include/trace/events/netfs.h+6 -10 replies
22/24cifs: Use iterate_and_advance*() routines directly for hashingfs/smb/client/cifsencrypt.c include/linux/iov_iter.h+70 -860 replies
23/24cifs: Switch crypto buffer to use a folio_queue rather than an xarrayfs/smb/client/cifsglob.h fs/smb/client/smb2ops.c include/linux/uio.h+127 -990 replies
24/24cifs: Don't support ITER_XARRAYfs/smb/client/cifsencrypt.c fs/smb/client/smbdirect.c+0 -1000 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/24] netfs: Read/write improvements

Hi Christian, Steve, Willy,

This set of patches includes one fscache fix and one cachefiles fix

(1) Fix a cookie access race in fscache.

(2) Fix the setxattr/removexattr syscalls to pull their arguments into kernel space before taking the sb_writers lock to avoid a deadlock against mm->mmap_lock.

A couple of adjustments to the /proc/fs/netfs/stats file:

(3) All the netfs stats lines begin 'Netfs:'. Change this to something a bit more useful.

(4) Add a couple of stats counters to track the numbers of skips and waits on the per-inode writeback serialisation lock to make it easier to check for this as a source of performance loss.

Some miscellaneous bits:

(5) Reduce the number of conditional branches in netfs_perform_write().

(6) Move the CIFS_INO_MODIFIED_ATTR flag to the netfs_inode struct and remove cifs_post_modify().

(7) Move the max_len/max_nr_segs members from netfs_io_subrequest to netfs_io_request as they're only needed for one subreq at a time.

(8) Add an 'unknown' source value for tracing purposes.

(9) Remove NETFS_COPY_TO_CACHE as it's no longer used.

(10) Set the request work function up front at allocation time.

(11) Use bh-disabling spinlocks for rreq->lock as cachefiles completion may be run from block-filesystem DIO completion in softirq context.

Then there's the main performance enhancing changes:

(12) Define a structure, struct folio_queue, and a new iterator type, ITER_FOLIOQ, to hold a buffer as a replacement for ITER_XARRAY. See that patch for questions about naming and form.

(13) Make cifs RDMA support ITER_FOLIOQ.

(14) Add a function to reset the iterator in a subrequest.

(15) Use folio queues in the write-side helpers instead of xarrays.

(16) Simplify the write-side helpers to use sheaves to skip gaps rather than trying to work out where gaps are.

(17) In afs, make the read subrequests asynchronous, putting them into work items to allow the next patch to do progressive unlocking/reading.

(18) Overhaul the read-side helpers to improve performance.

(19) Remove fs/netfs/io.c.

(20) Fix the caching of a partial block at the end of a file.

(21) Allow a store to be cancelled.

Then some changes for cifs to make it use folio queues instead of xarrays for crypto bufferage:

(22) Use raw iteration functions rather than manually coding iteration when hashing data.

(23) Switch to using folio_queue for crypto buffers.

(24) Remove the xarray bits.

David

David Howells (23):
cachefiles: Fix non-taking of sb_writers around set/removexattr
netfs: Adjust labels in /proc/fs/netfs/stats
netfs: Record contention stats for writeback lock
netfs: Reduce number of conditional branches in netfs_perform_write() netfs, cifs: Move CIFS_INO_MODIFIED_ATTR to netfs_inode
netfs: Move max_len/max_nr_segs from netfs_io_subrequest to
netfs_io_stream
netfs: Reserve netfs_sreq_source 0 as unset/unknown
netfs: Remove NETFS_COPY_TO_CACHE
netfs: Set the request work function upon allocation
netfs: Use bh-disabling spinlocks for rreq->lock
mm: Define struct folio_queue and ITER_FOLIOQ to handle a sequence of
folios
cifs: Provide the capability to extract from ITER_FOLIOQ to RDMA SGEs
netfs: Use new folio_queue data type and iterator instead of xarray
iter
netfs: Provide an iterator-reset function
netfs: Simplify the writeback code
afs: Make read subreqs async
netfs: Speed up buffered reading
netfs: Remove fs/netfs/io.c
cachefiles, netfs: Fix write to partial block at EOF
netfs: Cancel dirty folios that have no storage destination
cifs: Use iterate_and_advance*() routines directly for hashing
cifs: Switch crypto buffer to use a folio_queue rather than an xarray
cifs: Don't support ITER_XARRAY

Max Kellermann (1):
fs/netfs/fscache_cookie: add missing "n_accesses" check

 fs/9p/vfs_addr.c             |   5 +-
 fs/afs/file.c                |  29 +-
 fs/afs/fsclient.c            |   9 +-
 fs/afs/write.c               |   4 +-
 fs/afs/yfsclient.c           |   9 +-
 fs/cachefiles/io.c           |  19 +-
 fs/cachefiles/xattr.c        |  34 +-
 fs/ceph/addr.c               |  72 ++--
 fs/netfs/Makefile            |   3 +-
 fs/netfs/buffered_read.c     | 677 ++++++++++++++++++++++++-----------
 fs/netfs/buffered_write.c    | 309 ++++++++--------
 fs/netfs/direct_read.c       | 147 +++++++-
 fs/netfs/fscache_cookie.c    |   4 +
 fs/netfs/internal.h          |  33 +-
 fs/netfs/io.c                | 647 ---------------------------------
 fs/netfs/iterator.c          |  50 +++
 fs/netfs/main.c              |   6 +-
 fs/netfs/misc.c              |  94 +++++
 fs/netfs/objects.c           |  16 +-
 fs/netfs/read_collect.c      | 540 ++++++++++++++++++++++++++++
 fs/netfs/read_retry.c        | 256 +++++++++++++
 fs/netfs/stats.c             |  23 +-
 fs/netfs/write_collect.c     | 243 ++++---------
 fs/netfs/write_issue.c       |  92 ++---
 fs/nfs/fscache.c             |  19 +-
 fs/nfs/fscache.h             |   7 +-
 fs/smb/client/cifsencrypt.c  | 144 +-------
 fs/smb/client/cifsglob.h     |   3 +-
 fs/smb/client/cifssmb.c      |   6 +-
 fs/smb/client/file.c         |  71 ++--
 fs/smb/client/smb2ops.c      | 218 ++++++-----
 fs/smb/client/smb2pdu.c      |  10 +-
 fs/smb/client/smbdirect.c    |  82 +++--
 include/linux/folio_queue.h  | 138 +++++++
 include/linux/iov_iter.h     | 104 ++++++
 include/linux/netfs.h        |  44 ++-
 include/linux/uio.h          |  18 +
 include/trace/events/netfs.h | 140 ++++++--
 lib/iov_iter.c               | 229 +++++++++++-
 lib/kunit_iov_iter.c         | 259 ++++++++++++++
 lib/scatterlist.c            |  69 +++-
 41 files changed, 3175 insertions(+), 1707 deletions(-)
 delete mode 100644 fs/netfs/io.c
 create mode 100644 fs/netfs/read_collect.c
 create mode 100644 fs/netfs/read_retry.c
 create mode 100644 include/linux/folio_queue.h