Skip to content

smb3: Use iov_iters down to the network transport and fix DIO page pinning

David Howells linux-cifs linux-crypto linux-fsdevel linux-rdma lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest12 patches 9 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/12netfs: Add a function to extract a UBUF or IOVEC into a BVEC iteratorfs/netfs/Makefile fs/netfs/iterator.c include/linux/netfs.h+105 -00 replies
2/12netfs: Add a function to extract an iterator into a scatterlistfs/netfs/iterator.c include/linux/netfs.h mm/vmalloc.c+274 -00 replies
3/12cifs: Implement splice_read to pass down ITER_BVEC not ITER_PIPEfs/cifs/cifsfs.c fs/cifs/cifsfs.h fs/cifs/file.c +1 more+102 -61 reply
4/12cifs: Add a function to build an RDMA SGE list from an iteratorfs/cifs/smbdirect.c+220 -00 replies
5/12cifs: Add a function to Hash the contents of an iteratorfs/cifs/cifsencrypt.c+144 -04 replies
6/12cifs: Add some helper functionsfs/cifs/cifsfs.h fs/cifs/file.c+96 -00 replies
7/12cifs: Add a function to read into an iter from a socketfs/cifs/cifsproto.h fs/cifs/connect.c+17 -00 replies
8/12cifs: Change the I/O paths to use an iterator rather than a page listfs/cifs/Kconfig fs/cifs/cifsencrypt.c fs/cifs/cifsglob.h +11 more+1123 -10890 replies
9/12cifs: Build the RDMA SGE list directly from an iteratorfs/cifs/smbdirect.c fs/cifs/smbdirect.h+65 -1010 replies
10/12cifs: Remove unused codefs/cifs/file.c+0 -6060 replies
11/12cifs: Fix problem with encrypted RDMA data readfs/cifs/smb2ops.c+3 -04 replies
12/12cifs: DIO to/from KVEC-type iterators should now workfs/cifs/file.c+0 -200 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/12] smb3: Use iov_iters down to the network transport and fix DIO page pinning

Hi Steve,

Here's an updated version of my patchset to make the cifs/smb3 driver pass iov_iters down to the lowest layers where they can be passed directly to the network transport rather than passing lists of pages around.

I've fixed a couple of bugs and rebased on top of a merge of my two iov_iter_extract_pages() patches onto your for-next branch. The merge is so that the same commits are used as are in the linux-block tree.

The series also deals with some other issues:

(*) By pinning pages, it fixes the race between concurrent DIO read and
fork, whereby the pages containing the DIO read buffer may end up belonging to the child process and not the parent - with the result that the parent might not see the retrieved data.

(*) cifs shouldn't take refs on pages extracted from non-user-backed
iterators (eg. KVEC). With these changes, cifs will apply the appropriate cleanup. Note that there is the possibility the network transport might, but that's beyond the scope of this patchset.

(*) Making it easier to transition to using folios in cifs rather than
pages by dealing with them through BVEC and XARRAY iterators.

The first couple of patches to provide function to pin or leave unpinned the pages from an iterator (and not take a ref on them).

(1) Define qualifying flags for extraction functions.

(2) Define iov_iter_extract_pages() to do the extraction and iov_iter_extract_will_pin() to indicate how it should be cleaned up.

Then there are a couple of patches that add stuff to netfslib that I want to use there as well as in cifs:

(3) Add a netfslib function to extract and pin pages from an ITER_IOBUF or ITER_UBUF iterator into an ITER_BVEC iterator.

(4) Add a netfslib function to extract pages from an iterator that's of type ITER_UBUF/IOVEC/BVEC/KVEC/XARRAY and add them to a scatterlist. The cleanup will need to be done as for iov_iter_extract_pages().

BVEC, KVEC and XARRAY iterators can be rendered into elements that span multiple pages.

Then a fix:

(5) Fix oops due to uncleared server->smbd_conn in reconnect

Then there are some cifs helpers that work with iterators:

(6) Implement cifs_splice_read() to use an ITER_BVEC rather than an ITER_PIPE, bulk-allocating the pages, attaching them to the bvec, doing the I/O and then pushing the pages into the pipe. This avoids the problem with cifs wanting to split the pipe iterator in a later patch.

(7) Add a function to walk through an ITER_BVEC/KVEC/XARRAY iterator and add elements to an RDMA SGE list. Only the DMA addresses are stored, and an element may span multiple pages (say if an xarray contains a multipage folio).

(8) Add a function to walk through an ITER_BVEC/KVEC/XARRAY iterator and pass the contents into a shash function.

(9) Add functions to walk through an ITER_XARRAY iterator and perform various sorts of cleanup on the folios held therein, to be used on I/O completion.

(10) Add a function to read from the transport TCP socket directly into an iterator.

Then come the patches that actually do the work of iteratorising cifs:

(11) The main patch. Replace page lists with iterators. It extracts the pages from ITER_UBUF and ITER_IOVEC iterators to an ITER_BVEC iterator, pinning or getting refs on them, before passing them down as the I/O may be done from a worker thread.

The iterator is extracted into a scatterlist in order to talk to the crypto interface or to do RDMA.

(12) In the cifs RDMA code, extract the iterator into an RDMA SGE[] list, removing the scatterlist intermediate - at least for smbd_send(). There appear to be other ways for cifs to talk to the RDMA layer that don't go through that that I haven't managed to work out.

(13) Remove a chunk of now-unused code.

(14) Fix a problem with encrypted RDMA data read.

(15) Allow DIO to/from KVEC-type iterators.

I've pushed the patches here also:

https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-cifs

David

David Howells (12):
netfs: Add a function to extract a UBUF or IOVEC into a BVEC iterator
netfs: Add a function to extract an iterator into a scatterlist
cifs: Implement splice_read to pass down ITER_BVEC not ITER_PIPE
cifs: Add a function to build an RDMA SGE list from an iterator
cifs: Add a function to Hash the contents of an iterator
cifs: Add some helper functions
cifs: Add a function to read into an iter from a socket
cifs: Change the I/O paths to use an iterator rather than a page list
cifs: Build the RDMA SGE list directly from an iterator
cifs: Remove unused code
cifs: Fix problem with encrypted RDMA data read
cifs: DIO to/from KVEC-type iterators should now work

 fs/cifs/Kconfig       |    1 +
 fs/cifs/cifsencrypt.c |  172 +++-
 fs/cifs/cifsfs.c      |   12 +-
 fs/cifs/cifsfs.h      |    6 +
 fs/cifs/cifsglob.h    |   66 +-
 fs/cifs/cifsproto.h   |   11 +-
 fs/cifs/cifssmb.c     |   15 +-
 fs/cifs/connect.c     |   14 +
 fs/cifs/file.c        | 1848 +++++++++++++++++++----------------------
 fs/cifs/fscache.c     |   22 +-
 fs/cifs/fscache.h     |   10 +-
 fs/cifs/misc.c        |  128 +--
 fs/cifs/smb2ops.c     |  365 ++++----
 fs/cifs/smb2pdu.c     |   53 +-
 fs/cifs/smbdirect.c   |  535 +++++++-----
 fs/cifs/smbdirect.h   |    7 +-
 fs/cifs/transport.c   |   54 +-
 fs/netfs/Makefile     |    1 +
 fs/netfs/iterator.c   |  371 +++++++++
 fs/splice.c           |    1 +
 include/linux/netfs.h |    6 +
 mm/vmalloc.c          |    1 +
 22 files changed, 2013 insertions(+), 1686 deletions(-)
 create mode 100644 fs/netfs/iterator.c