Skip to contentSkip to conversation

netfs, iov_iter: Use a chain of bio_vec arrays instead of folio_queue

PATCH Superseded David Howells 2 participants 35 messages Nov 11 – Nov 12

Subsystems (15)

v17 of 19 Superseded Latest is v19 →

lore
Share & tools
lore

David Howells

author maintainer
#0 lore raw

v17 cover netfs, iov_iter: Use a chain of bio_vec arrays instead of folio_queue

Hi Christian,

Could you pull these patches please into your vfs-7.5.netfs branch? This is the third of five batches (two already having gone upstream), in this case replacing the folio_queue struct with a bvecq struct. It has been rebased on v7.4-rc1. This was split from v11 of a larger netfslib series[1].

This series adds a new type, struct bvecq. This points to a bounded array of bio_vecs with information about how to clean up the memory they point to. A number of bvecq structs can be linked together to form a chain and an iterator type, ITER_BVECQ, is provided that can be pointed at the chain and can walk it. Netfslib is then switched to use the new type and folio_queue is removed along with ITER_FOLIOQ.

This is intended as a more flexible replacement for struct folio_queue as the latter can only point to whole folios and cannot point to partial pages that might be extracted from an ITER_IOVEC/ITER_UBUF for direct or unbuffered I/O purposes.

These patches make a more or less straight swap between struct folio_queue and struct bvecq. The change is not quite trivial as the folio_queue struct contains a folio_batch struct and is basically a list of whole folios only, whereas the new bvecq struct is a chain of bio_vec arrays.

A further patchset will switch the netfslib unbuffered I/O code to using bvecqs and then the bvecqs will be passed down into the filesystem rather than passing an iterator.

The primary reason behind making this change is so that bvecq chains can be used in the assembly of network filesystem messages. Netfslib can furnish the filesystem with bvecq chains representing the data buffers that need to be read or written and then, for example, for a write RPC, the filesystem can add protocol headers and trailers onto the chain without the need to copy it and can also glue multiple buffers together to form compound operations and/or perform sparse operations.

One option here is to add two more fields to the bvecq struct, one to increase the offset of the first bio_vec and one to decrease the length of the last, thereby allowing a bvecq struct to point at a slice of another bio_vec array without having to copy the array - but at the cost of adding one or two extra conditional ops when getting the offset or length of a segment.

I have in-progress patch sets to rewrite the cifs transport[2] and the ceph and rbd transport[3] to make use of bvecq chains. For the cifs transport, the idea is to extend the SMB message concept further up the stack, where the PDU-marshalling routines will create individual RPC request blobs that the transport can then automatically chain together when it is assembling a compound SMB request for transmission.

With the ceph transport, the idea is to convert all the different data containers it has into just passing around bvecqs. These can then be chained together in order to transmit them.

This improves efficiency in the TCP stack as we no longer need to cork the TCP socket, call sendmsg() multiple times and then uncork; rather we can preassemble the message in a bvecq chain and just send the entire message in one shot with a single sendmsg() and reduce the number of places doing loops. Using a single sendmsg got an improvement in nfsd performance[4].

I also have some changes on the TCP receive side for the cifs transport (which is also likely applicable to the ceph TCP transport) whereby the receive buffers are 'spliced' out into a bvecq in the cifs I/O thread rather than being copied. Using a bvecq chain here is advantageous as we don't know in advance how many segments we're going to have. This allows us firstly to avoid copying data with the socket lock held (thus holding up sendmsg) and secondly to avoid copying data in the I/O thread (copying can be offloaded to the app thread). The last time I benchmarked this, it appeared to get fio reading tests on cifs a 5% speedup.

Unfortunately, this doesn't help AFS much as that uses a UDP transport, but it might also help 9P, at least with its TCP transport.

The patches can also be found here:

https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next-3

Thanks,
David

Changes
=======
ver #17)
- Fixed issues found by Sashiko[8]:
- Fix bvecq_alloc_buffer2() to set mem_type if zero slots requested.
- Fix folio leak in the kunit tests.
- Don't mark kunit functions as __init.
- Make subreqs pin the bvecq chain to avoid retry of a partially complete subreq seeing released memory.
- Fix handling of concurrent writeback of a single-type object and afs vnode status update.
- Fixed issues found by local review with claude-opus-5-5:
- Pass len from netfs_create_write_req() to netfs_alloc_request().
- Added a patch to pass the request length in for unbuffered/DIO writes

    and single-type object writeback to request creation, to set wreq->len.

- In netfs_writeback_single() set wreq->len to the specified length, not padded buffer size.
- Added a fix patch to remove __init and __refdata marks from kunit_iov_iter.c.
- Removed __refdata from kunit_bvecq.c too.
- Extended the bvecq chain pinning to the retry functions too.

ver #16)
- Rebase on v7.4-rc1.
- Add docs for the bvecq tools.
- Add some kunit tests for the bvecq iterator.
- Fixed issues found by local review with claude-opus-5-5:
- Added missing linux/mempool.h inclusion.
- Fix bvecq_alloc_one() to check the rounded-up allocation size against the pool allocation size, not the struct+array size which may have a gap.
- Fix bvecq_alloc_one() to limit the max slots to USHRT_MAX to not overrun the slot counters.
- Make bvec_kmap_partial()'s bvec pointer const.
- Added an iter_to_sg kunit test for bvecq.
- Added a patch to make afs symlink's use of ->directory_size consistent.
- In netfs_writeback_single(), fix the value ->submit_extendable_to is set to (though a future patch series will remove this).
- Split the afs dir bvecq patch into three:
- A patch to pass the content length to netfs_writeback_single(), separately from the buffer size in the iov_iter. The filesystem is then responsible for providing zeroed padding.
- A check is added to make sure the buffer is not smaller than the content.
- The API doc is updated.
- A patch to simplify netfs_writeback_single():
- No looking inside the iov_iter.
- Effectively do DIO rather than buffered I/O (pagecache not used).
- Pass a single op to the cache.
- Wait synchronously for op completion for now as the calling filesystem should otherwise do a wait.
- Removed the bits to set up a write to the server since it should only write to the cache.
- A patch to switch afs dir (and symlink) buffering to use bvecq instead of folio_queue.
- Clear new pages as added and clear the tail if shrunk by a third party for AFS directories.
- Clear the buffer tail before calling netfs_writeback_single() for AFS symlinks.
- Fix netfs_limit_bvecq() to handle empty bvecqs and short chains.
- Added missing SPDX to new doc.
- Added new files to MAINTAINERS.

ver #15)
- Fix issues from Sashiko[7]:
- Don't need to set page counts after calling alloc_pages_bulk().
- Adjust netfs_writeback_single() to return 0 on success, not the amount written.
- Remove a couple of unused constants.

ver #14)
- Fix issues from Sashiko[6]:
- In iter_bvecq_get_pages(), use folio_get() not get_pages.
- In netfs_read_set_unlock_at(), netfs_read_unlock_folios(), netfs_pgpriv2_unlock_copied_folios() and netfs_writeback_unlock_folios(), skip wholly unoccupied bvecqs.
- Remove some now-unused trace bits.

ver #13)
- Fix issues from Sashiko[5]:
- In iter_bvecq_get_pages(), need to add bv_offset when calling

    want_pages_array().

- In iter_bvecq_get_pages(), make the getting of a ref on the page dependent on the page being non-slab.
- In bvecq_alloc_one(), use mempool_alloc(), not mempool_alloc_noreserve().
- In bvecq_expand_buffer(), don't add __GFP_NORETRY|__GFP_WARN for order 0 allocations.
- Drop the netfs_bv_slot tracepoint as it's not used in these patches.
- In afs_do_read_single(), afs_dir_get_block() and afs_do_read_symlink(), use mapping_gfp_mask() rather than GFP_KERNEL.
- In netfs_writeback_single(), set ->submitted so that netfs_wait_for_write() doesn't then just return -EIO.
- In netfs_unlock_abandoned_read_pages(), reset ->first_tail_slot when moving on to next node.
- Remove netfs_folioq_trace(s) trace info enum and string set as their tracepoint got removed.

ver #12)
- Split from v11 of "netfs: Keep track of folios in a segmented bio_vec[] chain"[1]

[1] https://lore.kernel.org/r/[email protected]/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=cifs-experimental
[3] https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=ceph-iter
[4] https://lore.kernel.org/netdev/168979108540.1905271.9720708849149797793.stgit@morisot.1015granger.net/
[5] https://sashiko.dev/#/patchset/20260929075913.2740968-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260929103412.2807494-1-dhowells%40redhat.com
[7] https://sashiko.dev/#/patchset/20260930125247.3232130-1-dhowells%40redhat.com
[8] https://sashiko.dev/#/patchset/20261107160823.1465553-1-dhowells%40redhat.com

David Howells (17):
iov_iter: Fix kunit test to remove __init and __refdata marks Add a function to kmap one page of a multipage bio_vec
iov_iter: Add a segmented queue of bio_vec[]
netfs: Add some tools for managing bvecq chains
iov_iter: Add an API document for the bvecq structure
iov_iter: Add a bvecq kunit test
afs: Fix an inconsistency in the symlink code
netfs: Explicitly pass tail padding to netfs_writeback_single()
netfs: Change netfs_writeback_single() to not look inside the iterator
netfs: Pass the write length to request creation where appropriate
afs: Use a bvecq to hold dir content rather than folioq
cifs: Use a bvecq for buffering instead of a folioq
smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
netfs: Switch folioq to bvecq
smbdirect: Remove support for ITER_FOLIOQ from
smbdirect_map_sges_from_iter()
iov_iter: Remove ITER_FOLIOQ
netfs: Remove folio_queue

 Documentation/core-api/bvecq.rst            | 229 ++++++++
 Documentation/core-api/folio_queue.rst      | 209 -------
 Documentation/core-api/index.rst            |   2 +-
 Documentation/filesystems/netfs_library.rst |  47 +-
 MAINTAINERS                                 |   3 +
 fs/afs/dir.c                                |  55 +-
 fs/afs/dir_edit.c                           |  44 +-
 fs/afs/dir_search.c                         |  33 +-
 fs/afs/inode.c                              |   2 +-
 fs/afs/internal.h                           |   6 +-
 fs/afs/symlink.c                            |  55 +-
 fs/netfs/Makefile                           |   1 +
 fs/netfs/buffered_read.c                    |  36 +-
 fs/netfs/bvecq.c                            | 355 ++++++++++++
 fs/netfs/direct_write.c                     |   4 +-
 fs/netfs/internal.h                         |  10 +-
 fs/netfs/iterator.c                         |  42 +-
 fs/netfs/main.c                             |  13 +-
 fs/netfs/misc.c                             |  96 ----
 fs/netfs/objects.c                          |   6 +-
 fs/netfs/read_collect.c                     |  48 +-
 fs/netfs/read_pgpriv2.c                     |  28 +-
 fs/netfs/read_retry.c                       |  53 +-
 fs/netfs/rolling_buffer.c                   | 191 ++-----
 fs/netfs/stats.c                            |   6 +-
 fs/netfs/write_collect.c                    |  26 +-
 fs/netfs/write_issue.c                      | 214 ++-----
 fs/netfs/write_retry.c                      |  15 +-
 fs/smb/client/cifsglob.h                    |   2 +-
 fs/smb/client/smb2ops.c                     |  78 ++-
 fs/smb/smbdirect/connection.c               | 135 +++--
 include/linux/bvec.h                        |  18 +
 include/linux/bvecq.h                       | 176 ++++++
 include/linux/folio_queue.h                 | 282 ---------
 include/linux/iov_iter.h                    |  87 +--
 include/linux/netfs.h                       |  19 +-
 include/linux/rolling_buffer.h              |  42 +-
 include/linux/uio.h                         |  17 +-
 include/trace/events/netfs.h                |  37 +-
 kernel/bpf/btf.c                            |   2 -
 lib/Kconfig.debug                           |  13 +
 lib/iov_iter.c                              | 517 ++++++++++++-----
 lib/scatterlist.c                           |  82 ++-
 lib/tests/Makefile                          |   1 +
 lib/tests/kunit_bvecq.c                     | 599 ++++++++++++++++++++
 lib/tests/kunit_iov_iter.c                  | 208 +++----
 46 files changed, 2506 insertions(+), 1638 deletions(-)
 create mode 100644 Documentation/core-api/bvecq.rst
 delete mode 100644 Documentation/core-api/folio_queue.rst
 create mode 100644 fs/netfs/bvecq.c
 create mode 100644 include/linux/bvecq.h
 delete mode 100644 include/linux/folio_queue.h
 create mode 100644 lib/tests/kunit_bvecq.c
author maintainer
#1 lore raw

PATCH v17 1/17 iov_iter: Fix kunit test to remove __init and __refdata marks

Remove the __init marks from the functions in the iov_iter kunit test so that they don't get removed after the module is loaded or if the test is built in as the tests can be triggered through debugfs too.

Also remove the __refdata mark which is unnecessary if the table so marked doesn't point at any __init funcs.

This was found by claude-opus-5-5 run locally.

Fixes 2d71340ff1d4 ("iov_iter: Kunit tests for copying to/from an iterator")
cc Paulo Alcantara (Red Hat) <[email protected]>
cc Matthew Wilcox <[email protected]>
cc Jens Axboe <[email protected]>
cc Christoph Hellwig <[email protected]>
---
 lib/tests/kunit_iov_iter.c | 96 +++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 49 deletions(-)
lib/tests/kunit_iov_iter.c +47 -49
index 32e42d8c7ca1..b4ad38d1a4bd 100644
@@ -48,9 +48,9 @@ static void iov_kunit_unmap(void *data)
 	vfree(data);
 }
 
-static void *__init iov_kunit_create_buffer(struct kunit *test,
-					    struct page ***ppages,
-					    size_t npages)
+static void *iov_kunit_create_buffer(struct kunit *test,
+				     struct page ***ppages,
+				     size_t npages)
 {
 	struct page **pages;
 	unsigned long got, last;
@@ -90,11 +90,11 @@ static void *__init iov_kunit_create_buffer(struct kunit *test,
 	return buffer;
 }
 
-static void __init iov_kunit_load_kvec(struct kunit *test,
-				       struct iov_iter *iter, int dir,
-				       struct kvec *kvec, unsigned int kvmax,
-				       void *buffer, size_t bufsize,
-				       const struct kvec_test_range *pr)
+static void iov_kunit_load_kvec(struct kunit *test,
+				struct iov_iter *iter, int dir,
+				struct kvec *kvec, unsigned int kvmax,
+				void *buffer, size_t bufsize,
+				const struct kvec_test_range *pr)
 {
 	size_t size = 0;
 	int i;
@@ -116,7 +116,7 @@ static void __init iov_kunit_load_kvec(struct kunit *test,
 /*
  * Test copying to a ITER_KVEC-type iterator.
  */
-static void __init iov_kunit_copy_to_kvec(struct kunit *test)
+static void iov_kunit_copy_to_kvec(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -166,7 +166,7 @@ static void __init iov_kunit_copy_to_kvec(struct kunit *test)
 /*
  * Test copying from a ITER_KVEC-type iterator.
  */
-static void __init iov_kunit_copy_from_kvec(struct kunit *test)
+static void iov_kunit_copy_from_kvec(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -234,12 +234,12 @@ static const struct bvec_test_range bvec_test_ranges[] = {
 	{ -1, -1, -1 }
 };
 
-static void __init iov_kunit_load_bvec(struct kunit *test,
-				       struct iov_iter *iter, int dir,
-				       struct bio_vec *bvec, unsigned int bvmax,
-				       struct page **pages, size_t npages,
-				       size_t bufsize,
-				       const struct bvec_test_range *pr)
+static void iov_kunit_load_bvec(struct kunit *test,
+				struct iov_iter *iter, int dir,
+				struct bio_vec *bvec, unsigned int bvmax,
+				struct page **pages, size_t npages,
+				size_t bufsize,
+				const struct bvec_test_range *pr)
 {
 	struct page *can_merge = NULL, *page;
 	size_t size = 0;
@@ -275,7 +275,7 @@ static void __init iov_kunit_load_bvec(struct kunit *test,
 /*
  * Test copying to a ITER_BVEC-type iterator.
  */
-static void __init iov_kunit_copy_to_bvec(struct kunit *test)
+static void iov_kunit_copy_to_bvec(struct kunit *test)
 {
 	const struct bvec_test_range *pr;
 	struct iov_iter iter;
@@ -328,7 +328,7 @@ static void __init iov_kunit_copy_to_bvec(struct kunit *test)
 /*
  * Test copying from a ITER_BVEC-type iterator.
  */
-static void __init iov_kunit_copy_from_bvec(struct kunit *test)
+static void iov_kunit_copy_from_bvec(struct kunit *test)
 {
 	const struct bvec_test_range *pr;
 	struct iov_iter iter;
@@ -392,10 +392,10 @@ static void iov_kunit_destroy_folioq(void *data)
 	}
 }
 
-static void __init iov_kunit_load_folioq(struct kunit *test,
-					struct iov_iter *iter, int dir,
-					struct folio_queue *folioq,
-					struct page **pages, size_t npages)
+static void iov_kunit_load_folioq(struct kunit *test,
+				  struct iov_iter *iter, int dir,
+				  struct folio_queue *folioq,
+				  struct page **pages, size_t npages)
 {
 	struct folio_queue *p = folioq;
 	size_t size = 0;
@@ -429,7 +429,7 @@ static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)
 /*
  * Test copying to a ITER_FOLIOQ-type iterator.
  */
-static void __init iov_kunit_copy_to_folioq(struct kunit *test)
+static void iov_kunit_copy_to_folioq(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -491,7 +491,7 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
 /*
  * Test copying from a ITER_FOLIOQ-type iterator.
  */
-static void __init iov_kunit_copy_from_folioq(struct kunit *test)
+static void iov_kunit_copy_from_folioq(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -560,10 +560,10 @@ static void iov_kunit_destroy_xarray(void *data)
 	kfree(xarray);
 }
 
-static void __init iov_kunit_load_xarray(struct kunit *test,
-					 struct iov_iter *iter, int dir,
-					 struct xarray *xarray,
-					 struct page **pages, size_t npages)
+static void iov_kunit_load_xarray(struct kunit *test,
+				  struct iov_iter *iter, int dir,
+				  struct xarray *xarray,
+				  struct page **pages, size_t npages)
 {
 	size_t size = 0;
 	int i;
@@ -591,7 +591,7 @@ static struct xarray *iov_kunit_create_xarray(struct kunit *test)
 /*
  * Test copying to a ITER_XARRAY-type iterator.
  */
-static void __init iov_kunit_copy_to_xarray(struct kunit *test)
+static void iov_kunit_copy_to_xarray(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -649,7 +649,7 @@ static void __init iov_kunit_copy_to_xarray(struct kunit *test)
 /*
  * Test copying from a ITER_XARRAY-type iterator.
  */
-static void __init iov_kunit_copy_from_xarray(struct kunit *test)
+static void iov_kunit_copy_from_xarray(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -712,7 +712,7 @@ static void __init iov_kunit_copy_from_xarray(struct kunit *test)
 /*
  * Test the extraction of ITER_KVEC-type iterators.
  */
-static void __init iov_kunit_extract_pages_kvec(struct kunit *test)
+static void iov_kunit_extract_pages_kvec(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -792,7 +792,7 @@ static void __init iov_kunit_extract_pages_kvec(struct kunit *test)
 /*
  * Test the extraction of ITER_BVEC-type iterators.
  */
-static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
+static void iov_kunit_extract_pages_bvec(struct kunit *test)
 {
 	const struct bvec_test_range *pr;
 	struct iov_iter iter;
@@ -870,7 +870,7 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
 /*
  * Test the extraction of ITER_FOLIOQ-type iterators.
  */
-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
+static void iov_kunit_extract_pages_folioq(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct folio_queue *folioq;
@@ -949,7 +949,7 @@ static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
 /*
  * Test the extraction of ITER_XARRAY-type iterators.
  */
-static void __init iov_kunit_extract_pages_xarray(struct kunit *test)
+static void iov_kunit_extract_pages_xarray(struct kunit *test)
 {
 	const struct kvec_test_range *pr;
 	struct iov_iter iter;
@@ -1033,8 +1033,7 @@ struct iov_kunit_iter_to_sg_data {
 	size_t npages;
 };
 
-static void __init
-iov_kunit_iter_unpin_sgt(void *data)
+static void iov_kunit_iter_unpin_sgt(void *data)
 {
 	struct sg_table *sgt = data;
 
@@ -1042,9 +1041,9 @@ iov_kunit_iter_unpin_sgt(void *data)
 		unpin_user_page(sg_page(&sgt->sgl[i]));
 }
 
-static void __init
-iov_kunit_iter_to_sg_init(struct kunit *test, size_t bufsize, bool user,
-			  struct iov_kunit_iter_to_sg_data *data)
+static void iov_kunit_iter_to_sg_init(struct kunit *test, size_t bufsize,
+				      bool user,
+				      struct iov_kunit_iter_to_sg_data *data)
 {
 	struct page **spages;
 	struct scatterlist *sg;
@@ -1080,10 +1079,9 @@ iov_kunit_iter_to_sg_init(struct kunit *test, size_t bufsize, bool user,
 	memset(data->scratch, 0, bufsize);
 }
 
-static void __init
-iov_kunit_iter_to_sg_check(struct kunit *test, struct iov_iter *iter,
-			   size_t bufsize,
-			   struct iov_kunit_iter_to_sg_data *data)
+static void iov_kunit_iter_to_sg_check(struct kunit *test, struct iov_iter *iter,
+				       size_t bufsize,
+				       struct iov_kunit_iter_to_sg_data *data)
 {
 	static const size_t tail = 16 * PAGE_SIZE;
 	size_t i;
@@ -1128,7 +1126,7 @@ iov_kunit_iter_to_sg_check(struct kunit *test, struct iov_iter *iter,
 	KUNIT_EXPECT_EQ(test, i, bufsize);
 }
 
-static void __init iov_kunit_iter_to_sg_kvec(struct kunit *test)
+static void iov_kunit_iter_to_sg_kvec(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
 	struct iov_iter iter;
@@ -1145,7 +1143,7 @@ static void __init iov_kunit_iter_to_sg_kvec(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
+static void iov_kunit_iter_to_sg_bvec(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
 	struct page *p, *can_merge = NULL;
@@ -1173,7 +1171,7 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
+static void iov_kunit_iter_to_sg_folioq(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
 	struct folio_queue *folioq;
@@ -1190,7 +1188,7 @@ static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
+static void iov_kunit_iter_to_sg_xarray(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
 	struct xarray *xarray;
@@ -1207,7 +1205,7 @@ static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static void __init iov_kunit_iter_to_sg_ubuf(struct kunit *test)
+static void iov_kunit_iter_to_sg_ubuf(struct kunit *test)
 {
 	struct iov_kunit_iter_to_sg_data data;
 	struct iov_iter iter;
@@ -1221,7 +1219,7 @@ static void __init iov_kunit_iter_to_sg_ubuf(struct kunit *test)
 	iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
 }
 
-static struct kunit_case __refdata iov_kunit_cases[] = {
+static struct kunit_case iov_kunit_cases[] = {
 	KUNIT_CASE(iov_kunit_copy_to_kvec),
 	KUNIT_CASE(iov_kunit_copy_from_kvec),
 	KUNIT_CASE(iov_kunit_copy_to_bvec),
#2 lore raw

Re: [PATCH v17 01/17] iov_iter: Fix kunit test to remove __init and __refdata marks

Sashiko has reviewed this patch and found no issues.
Quoted 11 lines

Remove the __init marks from the functions in the iov_iter kunit test so that they don't get removed after the module is loaded or if the test is built in as the tests can be triggered through debugfs too.

Also remove the __refdata mark which is unnecessary if the table so marked doesn't point at any __init funcs.

This was found by claude-opus-5-5 run locally.

Fixes 2d71340ff1d4 ("iov_iter: Kunit tests for copying to/from an iterator")

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v17 2/17 Add a function to kmap one page of a multipage bio_vec

In reply to David Howells

Add a function to kmap one page of a multipage bio_vec by offset (which is added to the offset in the bio_vec internally). The caller is responsible for calculating how much of the page is then available.

---
 include/linux/bvec.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
include/linux/bvec.h +18 -0
index b63914ff56e3..7ea62643cfb7 100644
@@ -344,4 +344,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
 	return page_to_phys(bvec->bv_page) + bvec->bv_offset;
 }
 
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space.  The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(const struct bio_vec *bvec, size_t offset)
+{
+	offset += bvec->bv_offset;
+
+	return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+		(offset & ~PAGE_MASK);
+}
+
 #endif /* __LINUX_BVEC_H */
#4 lore raw

Re: [PATCH v17 02/17] Add a function to kmap one page of a multipage bio_vec

Sashiko has reviewed this patch and found no issues.
Quoted 5 lines

Add a function to kmap one page of a multipage bio_vec by offset (which is added to the offset in the bio_vec internally). The caller is responsible for calculating how much of the page is then available.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#5 lore raw

PATCH v17 3/17 iov_iter: Add a segmented queue of bio_vec[]

In reply to David Howells

Add the concept of a segmented queue of bio_vec[] arrays. This allows an indefinite quantity of elements to be handled and allows things like network filesystems and crypto drivers to glue bits on the ends without having to reallocate the array.

The bvecq struct that defines each segment also carries capacity/usage information along with flags indicating whether the constituent memory regions need freeing or unpinning. The bvecq structs are refcounted to allow a queue to be extracted in batches and split between a number of subrequests.

The bvecq can have the bio_vec[] it manages allocated in with it, but this is not required. A flag is provided for if this is the case as comparing ->bv to ->__bv is not sufficient to detect this case.

Add an iterator type ITER_BVECQ for it. This is intended to replace ITER_FOLIOQ (and ITER_XARRAY).

Note that the prev pointer is only really needed for iov_iter_revert() and could be dispensed with if struct iov_iter contained the head information as well as the current point.

---
 MAINTAINERS                |   1 +
 include/linux/bvecq.h      |  50 +++++
 include/linux/iov_iter.h   |  74 ++++++-
 include/linux/uio.h        |  11 +
 lib/iov_iter.c             | 435 ++++++++++++++++++++++++++++++++++++-
 lib/scatterlist.c          |  67 +++++-
 lib/tests/kunit_iov_iter.c | 277 +++++++++++++++++++++++
 7 files changed, 909 insertions(+), 6 deletions(-)
 create mode 100644 include/linux/bvecq.h
MAINTAINERS +1 -0
index 93abbb315065..6fa777744163 100644
@@ -10385,6 +10385,7 @@ S:	Supported
 F:	Documentation/filesystems/caching/
 F:	Documentation/filesystems/netfs_library.rst
 F:	fs/netfs/
+F:	include/linux/bvecq.h
 F:	include/linux/fscache*.h
 F:	include/linux/netfs.h
 F:	include/trace/events/fscache.h
include/linux/bvecq.h new file +50 -0

50 lines not on this page · raw message

include/linux/iov_iter.h +72 -2

104 lines not on this page · raw message

include/linux/uio.h +11 -0
index 638c1116e912..f87e05033eac 100644
@@ -27,6 +27,7 @@ enum iter_type {
 	ITER_IOVEC,
 	ITER_BVEC,
 	ITER_KVEC,
+	ITER_BVECQ,
 	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
@@ -70,6 +71,7 @@ struct iov_iter {
 				const struct iovec *__iov;
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
+				const struct bvecq *bvecq;
 				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
@@ -80,6 +82,7 @@ struct iov_iter {
 	};
 	union {
 		unsigned long nr_segs;
+		u16 bvecq_slot;
 		u8 folioq_slot;
 		loff_t xarray_start;
 	};
@@ -148,6 +151,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_DISCARD;
 }
 
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+	return iov_iter_type(i) == ITER_BVECQ;
+}
+
 static inline bool iov_iter_is_folioq(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_FOLIOQ;
@@ -303,6 +311,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
 void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
 			unsigned long nr_segs, size_t count);
 void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+			 const struct bvecq *bvecq,
+			 unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
 			  const struct folio_queue *folioq,
 			  unsigned int first_slot, unsigned int offset, size_t count);
lib/iov_iter.c +433 -2

520 lines not on this page · raw message

lib/scatterlist.c +65 -2

91 lines not on this page · raw message

lib/tests/kunit_iov_iter.c +277 -0

317 lines not on this page · raw message

#6 lore raw

Re: [PATCH v17 03/17] iov_iter: Add a segmented queue of bio_vec[]

Sashiko has reviewed this patch and found no issues.
Quoted 15 lines

Add the concept of a segmented queue of bio_vec[] arrays. This allows an indefinite quantity of elements to be handled and allows things like network filesystems and crypto drivers to glue bits on the ends without having to reallocate the array.

The bvecq struct that defines each segment also carries capacity/usage information along with flags indicating whether the constituent memory regions need freeing or unpinning. The bvecq structs are refcounted to allow a queue to be extracted in batches and split between a number of subrequests.

The bvecq can have the bio_vec[] it manages allocated in with it, but this is not required. A flag is provided for if this is the case as comparing ->bv to ->__bv is not sufficient to detect this case.

Quoted 8 lines

Add an iterator type ITER_BVECQ for it. This is intended to replace ITER_FOLIOQ (and ITER_XARRAY).

Note that the prev pointer is only really needed for iov_iter_revert() and could be dispensed with if struct iov_iter contained the head information as well as the current point.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author maintainer
#7 lore raw

PATCH v17 4/17 netfs: Add some tools for managing bvecq chains

In reply to David Howells

Provide a selection of tools for managing bvec queue chains and a memory pool for writeback to draw upon if needed. The tools include the allocation, prepopulation and refcounting of bvecqs and bvecq chains. When a chain is being cleaned up, the memory segments will be appropriately disposed off according to the flags on each bvecq in that chain, allowing for mixed memory cleanup types.

The main use for this is to replace the use of both folio_queues and kvmalloc'd bio_vec arrays in netfslib and creating a structure that allows network filesystems to easily build an entire message by gluing bits of protocol on either side of the bvecq chains and also glue chains together for sparse writes and compound operations. This allows TCP-based transports to pass a message in a single sendmsg() call and avoid corking as this makes things more efficient.

This will also be used to do things like creating an encryption buffer in cifs or a directory content buffer in afs.

Note that these tools are placed in netfslib for the moment, but can be moved to lib/ later if any non-netfslib code starts to use them. The iov_iter bits don't depend on these tools and must live in the core.

---
 fs/netfs/Makefile         |   1 +
 fs/netfs/bvecq.c          | 353 ++++++++++++++++++++++++++++++++++++++
 fs/netfs/internal.h       |   3 +
 fs/netfs/main.c           |   8 +-
 fs/netfs/objects.c        |   1 -
 fs/netfs/rolling_buffer.c |   1 -
 fs/netfs/stats.c          |   4 +-
 include/linux/bvecq.h     | 124 +++++++++++++
 include/linux/netfs.h     |   1 +
 9 files changed, 492 insertions(+), 4 deletions(-)
 create mode 100644 fs/netfs/bvecq.c
fs/netfs/Makefile +1 -0
index 54834cde7e56..b1ea4439c1bb 100644
@@ -3,6 +3,7 @@
 netfs-y := \
 	buffered_read.o \
 	buffered_write.o \
+	bvecq.o \
 	direct_read.o \
 	direct_write.o \
 	iterator.o \
fs/netfs/bvecq.c new file +353 -0

353 lines not on this page · raw message

fs/netfs/internal.h +3 -0
index 6b583152132f..d554468bf694 100644
@@ -11,6 +11,7 @@
 #include <linux/netfs.h>
 #include <linux/fscache.h>
 #include <linux/fscache-cache.h>
+#include <linux/mempool.h>
 #include <trace/events/netfs.h>
 #include <trace/events/fscache.h>
 
@@ -45,6 +46,7 @@ extern struct list_head netfs_io_requests;
 extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
+extern mempool_t netfs_bvecq_pool;
 extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
@@ -205,6 +207,7 @@ extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
 extern atomic_t netfs_n_folioq;
+extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
 
fs/netfs/main.c +7 -1
index 609e22e8f76a..3c796fe4280f 100644
@@ -7,7 +7,6 @@
 
 #include <linux/module.h>
 #include <linux/export.h>
-#include <linux/mempool.h>
 #include <linux/proc_fs.h>
 #include <linux/seq_file.h>
 #include "internal.h"
@@ -28,6 +27,7 @@ static struct kmem_cache *netfs_request_slab;
 static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
+mempool_t netfs_bvecq_pool;
 mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
@@ -111,6 +111,9 @@ static int __init netfs_init(void)
 	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
 		goto error_folioq_pool;
 
+	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_POOL_SIZE) < 0)
+		goto error_bvecq_pool;
+
 	netfs_request_slab = kmem_cache_create("netfs_request",
 					       sizeof(struct netfs_io_request), 0,
 					       SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT,
@@ -163,6 +166,8 @@ static int __init netfs_init(void)
 error_reqpool:
 	kmem_cache_destroy(netfs_request_slab);
 error_req:
+	mempool_exit(&netfs_bvecq_pool);
+error_bvecq_pool:
 	mempool_exit(&netfs_folioq_pool);
 error_folioq_pool:
 	return ret;
@@ -177,6 +182,7 @@ static void __exit netfs_exit(void)
 	kmem_cache_destroy(netfs_subrequest_slab);
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
+	mempool_exit(&netfs_bvecq_pool);
 	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
fs/netfs/objects.c +0 -1
index 4b8d20559b0e..6d7839c0498d 100644
@@ -6,7 +6,6 @@
  */
 
 #include <linux/slab.h>
-#include <linux/mempool.h>
 #include "internal.h"
 
 static void netfs_free_request(struct work_struct *work);
fs/netfs/rolling_buffer.c +0 -1
index d30d5ef6d86e..4347c99452d8 100644
@@ -6,7 +6,6 @@
  */
 
 #include <linux/bitops.h>
-#include <linux/mempool.h>
 #include <linux/pagemap.h>
 #include <linux/rolling_buffer.h>
 #include <linux/slab.h>
fs/netfs/stats.c +3 -1
index 9a607c4e62dd..a10d34f88597 100644
@@ -47,6 +47,7 @@ atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
 atomic_t netfs_n_folioq;
+atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
 {
@@ -88,9 +89,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
+		   atomic_read(&netfs_n_bvecq),
 		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
include/linux/bvecq.h +124 -0

132 lines not on this page · raw message

include/linux/netfs.h +1 -0
index e5961450f3f1..2cf1163aafa6 100644
@@ -17,6 +17,7 @@
 #include <linux/workqueue.h>
 #include <linux/fs.h>
 #include <linux/pagemap.h>
+#include <linux/bvecq.h>
 #include <linux/uio.h>
 #include <linux/rolling_buffer.h>
 
#8 lore raw

Re: [PATCH v17 04/17] netfs: Add some tools for managing bvecq chains

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [High] netfs: mempool deadlock in bvecq_alloc_chain() when for_writeback is true --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [High] netfs: mempool deadlock in bvecq_alloc_chain() when for_writeback is true
--

commit e5b9a085f1bcfd322d1044543745cbe0c1ae3076
Author: David Howells <[email protected]>

netfs: Add some tools for managing bvecq chains

This commit adds helpers for allocating, populating, refcounting, and freeing bvecq nodes and chains, along with an emergency mempool for writeback allocations.

Quoted 5 lines
fs/netfs/bvecq.c new file +0 -0
new file mode 100644
index 0000000000000..1d4da3de9b9d2

[ ... ]

Quoted 78 lines
fs/netfs/bvecq.c new file +78 -0
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+	struct bvecq *bq;
+	size_t size;
+	bool from_pool = false;
+
+	/* Don't overrun ->max_slots or ->nr_slots. */
+	if (nr_slots > BVECQ_MAX_INLINE_SLOTS)
+		nr_slots = BVECQ_MAX_INLINE_SLOTS;
+
+	size = kmalloc_size_roundup(struct_size(bq, __bv, nr_slots));
+	gfp &= ~(GFP_ZONEMASK | __GFP_THISNODE);
+
+	if (for_writeback) {
+		if (size != BVECQ_POOL_SIZE) {
+			gfp_t gfp_temp = gfp;
+
+			gfp_temp |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
+			gfp_temp &= ~(__GFP_DIRECT_RECLAIM | __GFP_IO);
+			bq = kmalloc(size, gfp_temp);
+			if (bq)
+				goto success;
+		}
+
+		bq = mempool_alloc(&netfs_bvecq_pool, gfp);
+		if (!bq)
+			return bq;
+		from_pool = true;
+		size = BVECQ_POOL_SIZE;
+	} else {
+		bq = kmalloc(size, gfp);
+		if (!bq)
+			return bq;
+	}
+
+success:
+	*bq = (struct bvecq) {
+		.ref		= REFCOUNT_INIT(1),
+		.bv		= bq->__bv,
+		.inline_bv	= true,
+		.max_slots	= min((size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+				      BVECQ_MAX_INLINE_SLOTS),
+		.from_pool	= from_pool,
+	};
+	netfs_stat(&netfs_n_bvecq);
+	return bq;
+}
+EXPORT_SYMBOL(bvecq_alloc_one);
+
+/**
+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes providing at least the requested cumulative
+ * number of slots.  Each node is a maximum of 4KiB in size.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+	struct bvecq *head = NULL, *tail = NULL;
+
+	_enter("%zu", nr_slots);
+
+	for (;;) {
+		struct bvecq *bq;
+
+		bq = bvecq_alloc_one(min(nr_slots, BVECQ_4KB_SLOTS), gfp, for_writeback);
+		if (!bq)
+			goto oom;
+
+		if (tail)
+			bvecq_append(tail, bq);
+		else
+			head = bq;
+		tail = bq;

fs/netfs/bvecq.c line 143

Quoted 10 lines
fs/netfs/bvecq.c new file +10 -0
+		if (tail->max_slots >= nr_slots)
+			break;
+		nr_slots -= tail->max_slots;
+	}
+
+	return head;
+oom:
+	bvecq_put(head);
+	return NULL;
+}

[Severity: High]
Can bvecq_alloc_chain() deadlock on netfs_bvecq_pool when for_writeback is true and gfp permits direct reclaim?

Under memory pressure, bvecq_alloc_one() falls back to mempool_alloc(&netfs_bvecq_pool, gfp), which yields a 512-byte node holding BVECQ_POOL_SLOTS (29 slots on 64-bit). bvecq_alloc_chain() loops calling bvecq_alloc_one() while holding previously allocated pool elements in head.

Because netfs_bvecq_pool has 100 elements, a single chain requesting more than 2900 slots (~11.3 MiB of 4 KiB pages) or concurrent writeback threads building chains at the same time can exhaust the pool midway through the loop. Each thread then sleeps in mempool_alloc() waiting for elements to be freed while pinning the elements already attached to head, before any I/O can be submitted to release them.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author maintainer
#9 lore raw

PATCH v17 5/17 iov_iter: Add an API document for the bvecq structure

In reply to David Howells

Add an API description document for the bvecq structure.

cc Paulo Alcantara (Red Hat) <[email protected]>
cc Matthew Wilcox <[email protected]>
cc Jens Axboe <[email protected]>
cc Christoph Hellwig <[email protected]>
---
 Documentation/core-api/bvecq.rst | 229 +++++++++++++++++++++++++++++++
 Documentation/core-api/index.rst |   1 +
 MAINTAINERS                      |   1 +
 fs/netfs/bvecq.c                 |   2 +
 include/linux/bvecq.h            |   2 +
 5 files changed, 235 insertions(+)
 create mode 100644 Documentation/core-api/bvecq.rst
Documentation/core-api/bvecq.rst new file +229 -0
new file mode 100644
index 000000000000..4a2b3e288982
@@ -0,0 +1,229 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================================
+bvecq - Chainable bio_vec queue structure
+=========================================
+
+Author: David Howells <[email protected]>
+
+.. Contents:
+
+ * Overview
+ * Allocating bvecqs
+ * Refcounting
+ * Expandable buffer
+ * Producer functions
+ * Consumer functions
+ * Utility functions
+ * I/O iterators
+ * API Function Reference
+
+
+Overview
+========
+
+The ``bvecq`` struct can be used to form chains of arrays of ``bio_vec``
+structures - kind of a single layer of indirection.  Each bvecq segment points
+to the previous and next bvecq segments in the chain, but is not intended to
+form a circular queue as there are applications where the producer is adding to
+one end whilst the consumer is taking off the other.
+
+To use this, the caller should do::
+
+	#include <linux/bvecq.h>
+
+The publicly accessible members of the structure are::
+
+	struct bvecq {
+		struct bvecq	*next;
+		struct bvecq	*prev;
+		u32		priv;
+		u16		nr_slots;
+		u16		max_slots;
+		enum bvecq_mem	mem_type:3;
+		struct bio_vec	*bv;
+		...
+	};
+
+A pair of pointers are provided, ``next`` and ``prev``, that point to the
+segments on either side of the segment being accessed.  Whilst this is a
+doubly-linked list, it is intentionally not a circular list; the outward
+sibling pointers in terminal segments should be NULL.
+
+The ``priv`` member is for the owner of the bvecq to use as they will.
+
+The ``bv`` pointer points at an array of bio_vecs that list a number of regions
+of memory.  ``max_slots`` indicates how large the array is and ``nr_slots``
+says how much of the array is being used.  Note that there are access functions
+for nr_slots that insert appropriate barriering in the case that there are
+separate producer and consumer threads.
+
+The ``mem_type`` member informs bvecq_put() of how to clean up the bio_vecs
+attached to this segment.  It can be one of:
+
+ * ``BVECQ_MEM_EXTERNAL`` - Externally managed; no cleanup.
+ * ``BVECQ_MEM_PAGECACHE`` - Part of pagecache; put_page() used.
+ * ``BVECQ_MEM_GUP`` - Pinned userspace memory; unpin_user_page() used.
+ * ``BVECQ_MEM_ALLOCED`` - Allocated page; __free_pages() used, may be pooled.
+
+
+Allocating bvecqs
+=================
+
+A number of functions are provided to allocate bvecqs::
+
+	struct bvecq *bvecq_alloc_one(size_t nr_slots,
+				      gfp_t gfp,
+				      bool for_writeback);
+	struct bvecq *bvecq_alloc_chain(size_t nr_slots,
+					gfp_t gfp,
+					bool for_writeback);
+	struct bvecq *bvecq_alloc_buffer(size_t size,
+					 gfp_t gfp,
+					 bool for_writeback);
+	struct bvecq *bvecq_alloc_buffer2(size_t size,
+					  unsigned int pre_slots,
+					  gfp_t gfp,
+					  bool for_writeback);
+
+The first of these, ``bvecq_alloc_one()`` allocates a single bvecq with at
+least the requested number of inline slots.  The size of the object is rounded
+up to the size of the slab granule that it will be allocated in and the number
+of slots increased appropriately to match.  If ``for_writeback`` is set, then
+the netfslib writeback bvecq pool will be used to backstop the allocation if
+the initial slab request fails; in such a case, the number of slots will get
+reduced to ``BVECQ_POOL_SLOTS``.
+
+Two convenience slot count constants are provided:
+
+ * ``BVECQ_POOL_SLOTS`` - The right number of slots to alloc from the mempool
+ * ``BVECQ_4KB_SLOTS`` - The right number of slots to get a 4K allocation
+
+``bvecq_alloc_chain()`` will allocate a sufficiently large chain of bvecqs to
+hold the requested number of slots, capping each allocation at 4K.
+
+``bvecq_alloc_buffer()`` will allocate a chain and populate it, allocating
+sufficient pages to provide a buffer of the specified size attached to the
+chain.  ``bvecq_alloc_buffer2()`` is similar, but it reserves some unused
+slots at the front of the chain.  Note that these functions may add multipage
+pages in the future.
+
+
+Refcounting
+===========
+
+Each bvecq has a refcount and two functions are provided to handle that::
+
+	struct bvecq *bvecq_get(struct bvecq *bq);
+	void bvecq_put(struct bvecq *bq);
+
+The first of these gets a new reference and the second drops a reference.  When
+the refcount hits zero, the second iteratively drops the refs along the chain
+until it hits a segment that has a refcount that doesn't reduce to zero.
+
+
+Expandable buffer
+=================
+
+This facility provides a function to manage an expandable buffer::
+
+	int bvecq_expand_buffer(struct bvecq **_buffer,
+				size_t *_cur_size,
+				size_t size,
+				gfp_t gfp);
+
+Pages are added to the bvecq chain and the chain expanded such that, on
+returning, it holds ``size`` amount of space (without losing any content).  The
+caller needs to retain ``*_cur_size`` between calls as that tracks the amount
+of space currently allocated.  Note that pages in the chain may be allocated as
+compound pages larger than PAGE_SIZE.
+
+
+Producer functions
+==================
+
+When building and filling a bvecq, there are a number of utility functions that
+can be used to provide the correct barriering and to query the fields in the
+bvecq currently being filled.
+
+Firstly, there's a function to check to see if a bvecq has any spare slots
+left::
+
+	bool bvecq_is_full(const struct bvecq *bvecq);
+
+If not, ``bvecq_alloc_one()`` can be called to allocate a new segment which can
+be initialised and then added to the chain with::
+
+	void bvecq_append(struct bvecq *to, struct bvecq *add);
+
+bio_vec slots in the segment can then be filled and the number of occupied
+slots updated by calling::
+
+	void bvecq_filled_to(struct bvecq *bvecq,
+			     unsigned int to);
+
+to insert the appropriate barrier.
+
+
+Consumer functions
+==================
+
+When consuming and destroying a bvecq, there are some utility functions for
+that too, providing the needed barriering.  Firstly, there's a function to find
+out how many slots have been used in the current bvecq::
+
+	unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq);
+
+Or, more conveniently, if the current slot position is valid within the bvecq::
+
+	bool bvecq_acquire_slot(const struct bvecq *bvecq,
+				unsigned int slot);
+
+If this returns true, the slot is valid and can be looked at; if it returns
+false, the bvecq may have been consumed.  If it has, the next bvecq can be
+obtained by calling::
+
+	struct bvecq *bvecq_next(const struct bvecq *bq);
+
+which inserts the appropriate barrier - though bvecq_acquire_slot should be
+rechecked after calling this in case the producer added more slots and then
+extended the list.
+
+
+Utility functions
+=================
+
+A utility function is provided to list the bio_vecs pointed to by a bvecq
+chain::
+
+	void bvecq_dump(const struct bvecq *bq);
+
+
+I/O iterators
+=============
+
+bvecq chains can be used with the ``iov_iter`` facility too.  An iterator can
+be initialised to point at a chain by calling::
+
+	void iov_iter_bvec_queue(struct iov_iter *i,
+				 unsigned int direction,
+				 const struct bvecq *bvecq,
+				 unsigned int first_slot,
+				 unsigned int offset,
+				 size_t count);
+
+In addition to setting the iterator, direction and count, the first bvecq in
+the chain to be iterated over is set, along with the starting slot in that
+iterator and the offset into the slot.  Once the iterator is set up, it can be
+treated like any other I/O iterator.
+
+In a producer/consumer environment, it is expected that when an iterator is set
+up, all the bvecqs and all the slots within the count of the iterator must
+exist and must be populated before this function is called.
+
+
+API Function Reference
+======================
+
+.. kernel-doc:: include/linux/bvecq.h
+.. kernel-doc:: fs/netfs/bvecq.c
Documentation/core-api/index.rst +1 -0
index 92f91c6a0d79..e39edef58ab6 100644
@@ -39,6 +39,7 @@ Library functionality that is used throughout the kernel.
    kref
    cleanup
    assoc_array
+   bvecq
    folio_queue
    xarray
    maple_tree
MAINTAINERS +1 -0
index 6fa777744163..9cb3785a5bd2 100644
@@ -10382,6 +10382,7 @@ M:	Paulo Alcantara <[email protected]>
 L:	[email protected]
 L:	[email protected]
 S:	Supported
+F:	Documentation/core-api/bvecq.rst
 F:	Documentation/filesystems/caching/
 F:	Documentation/filesystems/netfs_library.rst
 F:	fs/netfs/
fs/netfs/bvecq.c +2 -0
index 1d4da3de9b9d..f1264ae52033 100644
@@ -3,6 +3,8 @@
  *
  * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells ([email protected])
+ *
+ * See also Documentation/core-api/bvecq.rst
  */
 
 #include <linux/bvecq.h>
include/linux/bvecq.h +2 -0
index d9d3dd487a27..90e6eb9588bf 100644
@@ -3,6 +3,8 @@
  *
  * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells ([email protected])
+ *
+ * See also Documentation/core-api/bvecq.rst
  */
 
 #ifndef _LINUX_BVECQ_H
#10 lore raw

Re: [PATCH v17 05/17] iov_iter: Add an API document for the bvecq structure

Sashiko has reviewed this patch and found no issues.
Quoted 3 lines

Add an API description document for the bvecq structure.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author maintainer
#11 lore raw

PATCH v17 6/17 iov_iter: Add a bvecq kunit test

In reply to David Howells

Add a bvecq kunit test to exercise the bvecq iterator in a bunch of different ways. Each kunit test sets up a series of folios of different sizes (1, 2, 4 or 8 pages) and fills each subpage thereof with a distinct predictable pattern that it can then sample to check that the iterator appears to be in the right place.

The folios are then assembled into a bvecq chain depending on a particular recipe and partialness list and then advance, revert and copy-out is tested, along with checking iov_iter_npages() at each point.

Each test is a combination of a bvecq chain recipe:

- A single bvecq with 16 folios with no gaps (zero-len bio_vec).
- A chain of two bvecqs, each with 8 folios and no gaps.
- As the previous, but with a gap at the end of the first chain.
- A chain of four bvecqs, each with 4 folios and scattered gaps, including at the front and back.
- A chain of five bvecqs, 3 with folios and two with no folios, one empty and one with a couple of gaps.
- A chain of seven bvecqs, 3 with folios, 3 completely empty (including first and last bvecqs in the chain) and 1 with just a couple of gaps.

and a selection of folio parts:

- Whole folios only.
- Scattered parts of folios, with some contiguities.
- Every folio with the first and last bytes excluded.

and a test algorithm:

- For each possible starting position in the bvecq, do a small advance, check fill, a couple of small reverts, each checked and then advance to the next starting position.
- Starting at the beginning, advance in jumps by a largish step forward, checking after each jump; then when the end is hit revert in jumps by a slightly larger step back, checking each time; then double the step and repeat until the step exceeds the size of the iterator.

iov_iter_alignment() is compared at the beginning of one of the test sets to a compound of alignments from an ITER_BVEC. Note that the ITER_BVEC alignment check malfunctions if given a zero-length bio_vec array.

Exercising extraction, copy-from and copy-to is done in kunit_iov_iter.c, with those bits added in a different patch.

cc Paulo Alcantara (Red Hat) <[email protected]>
cc Matthew Wilcox <[email protected]>
cc Jens Axboe <[email protected]>
cc Christoph Hellwig <[email protected]>
---
 MAINTAINERS             |   1 +
 lib/Kconfig.debug       |  13 +
 lib/tests/Makefile      |   1 +
 lib/tests/kunit_bvecq.c | 599 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 614 insertions(+)
 create mode 100644 lib/tests/kunit_bvecq.c
MAINTAINERS +1 -0
index 9cb3785a5bd2..bdda4ad58ab4 100644
@@ -10391,6 +10391,7 @@ F:	include/linux/fscache*.h
 F:	include/linux/netfs.h
 F:	include/trace/events/fscache.h
 F:	include/trace/events/netfs.h
+F:	lib/tests/kunit_bvecq.c
 
 FILESYSTEMS [STACKABLE]
 M:	Miklos Szeredi <[email protected]>
lib/Kconfig.debug +13 -0
index 2d22638ec13a..06b4546421a1 100644
@@ -2385,6 +2385,19 @@ config TEST_IOV_ITER
 
 	  If unsure, say N.
 
+config TEST_BVECQ
+	tristate "Test bvecq operation" if !KUNIT_ALL_TESTS
+	depends on KUNIT
+	depends on MMU
+	depends on NETFS_SUPPORT
+	default KUNIT_ALL_TESTS
+	help
+	  Enable this to turn on testing of the operation of the bvecq
+	  structure.  This test is executed only once during system
+	  boot (so affects only boot time), or at module load time.
+
+	  If unsure, say N.
+
 config KPROBES_SANITY_TEST
 	tristate "Kprobes sanity tests" if !KUNIT_ALL_TESTS
 	depends on DEBUG_KERNEL
lib/tests/Makefile +1 -0
index df0537e1b91f..85e3c5e1f7d2 100644
@@ -27,6 +27,7 @@ obj-$(CONFIG_GLOB_KUNIT_TEST) += glob_kunit.o
 obj-$(CONFIG_HASHTABLE_KUNIT_TEST) += hashtable_test.o
 obj-$(CONFIG_HASH_KUNIT_TEST) += test_hash.o
 obj-$(CONFIG_TEST_IOV_ITER) += kunit_iov_iter.o
+obj-$(CONFIG_TEST_BVECQ) += kunit_bvecq.o
 obj-$(CONFIG_IS_SIGNED_TYPE_KUNIT_TEST) += is_signed_type_kunit.o
 obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o
 obj-$(CONFIG_LIST_KUNIT_TEST) += list-test.o
lib/tests/kunit_bvecq.c new file +599 -0

599 lines not on this page · raw message

#12 lore raw

Re: [PATCH v17 06/17] iov_iter: Add a bvecq kunit test

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Low] iov_iter: bvecq_kunit_advance() bounds revert steps by left instead of pos - [Low] iov_iter…

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Low] iov_iter: bvecq_kunit_advance() bounds revert steps by left instead of pos
- [Low] iov_iter: unchecked kunit_add_action_or_reset() in kunit_bvecq.c causes UAF
--

commit 395487bf3dfe00b3cc632478de3e9afe0e1c97ad
Author: David Howells <[email protected]>

iov_iter: Add a bvecq kunit test

This commit adds a KUnit test suite (CONFIG_TEST_BVECQ) to exercise ITER_BVECQ iterator operations including advance, revert, npages, and alignment across various bvecq chains and partial folio configurations.

Quoted 6 lines
lib/tests/kunit_bvecq.c new file +0 -0
new file mode 100644
index 0000000000000..19f3dc41eda78
@@ -0,0 +1,599 @@

[ ... ]

Quoted 13 lines
lib/tests/kunit_bvecq.c new file +13 -0
+static void iov_bvecq_alloc_folios(struct kunit *test)
+{
+	struct folio *folio, **folios = bvecq_kunit_folios;
+	const int nr = ARRAY_SIZE(bvecq_kunit_folios);
+
+	bvecq_kunit_pagemap_nr = 0;
+	memset(bvecq_kunit_pagemap, 0, sizeof(bvecq_kunit_pagemap));
+	memset(bvecq_kunit_folios, 0, sizeof(bvecq_kunit_folios));
+
+	kunit_add_action_or_reset(test, bvecq_kunit_release_folios, NULL);
+
+	for (int i = 0; i < nr; i++) {
+		folio = folio_alloc(GFP_KERNEL, bvecq_kunit_orders[i]);

[ ... ]

Quoted 22 lines
lib/tests/kunit_bvecq.c new file +22 -0
+static struct bvecq *bvecq_kunit_load_chain(struct kunit *test,
+					    const struct bvecq_test_recipe *rec,
+					    const struct bvecq_test_partials *parts,
+					    size_t *size)
+{
+	struct bvecq *head = NULL, *bq = NULL, *p;
+	size_t pos = 0;
+	int folio_index = 0;
+
+	*size = 0;
+
+	for (; rec->cmd != BVECQ_KUNIT_STOP; rec++) {
+		if (!head)
+			KUNIT_ASSERT_EQ(test, rec->cmd, BVECQ_KUNIT_ALLOC);
+
+		switch (rec->cmd) {
+		case BVECQ_KUNIT_STOP:
+			break;
+
+		case BVECQ_KUNIT_ALLOC:
+			p = bvecq_alloc_one(rec->count, GFP_KERNEL, false);
+			KUNIT_ASSERT_NOT_NULL(test, p);

lib/tests/kunit_bvecq.c line 309

Quoted 10 lines
lib/tests/kunit_bvecq.c new file +10 -0
+
+			if (!head) {
+				head = p;
+				kunit_add_action_or_reset(test, bvecq_kunit_bvecq_put, head);
+				bq = p;
+			} else {
+				bvecq_append(bq, p);
+				bq = p;
+			}
+			break;

[Severity: Low]
Does an allocation failure in kunit_add_action_or_reset() cause a use-after-free on head in bvecq_kunit_load_chain() and leak the folios in iov_bvecq_alloc_folios()?

When kunit_add_action() fails to allocate its action wrapper, kunit_add_action_or_reset() immediately runs the cleanup callback and returns an error without aborting the test. In bvecq_kunit_load_chain(), bvecq_kunit_bvecq_put() frees head via bvecq_put(), and the loop then continues using the freed bq pointer and leaks any subsequent segments appended to the chain. Similarly, in iov_bvecq_alloc_folios(), bvecq_kunit_release_folios() runs before the loop, leaving the 16 folios allocated afterward without a cleanup action. Asserting that kunit_add_action_or_reset() returns 0 (and resetting bvecq_kunit_folios_nr to 0 alongside the memset() in iov_bvecq_alloc_folios()) would avoid this.

[ ... ]

Quoted 4 lines
lib/tests/kunit_bvecq.c new file +4 -0
+static void bvecq_kunit_advance(struct kunit *test,
+				const struct bvecq_test_recipe *recipe,
+				const struct bvecq_test_partials *parts)
+{

[ ... ]

Quoted 18 lines
lib/tests/kunit_bvecq.c new file +18 -0
+	npages = iov_iter_npages(&iter, INT_MAX);
+	from = iov_iter_count(&iter);
+	do {
+		size_t left = from, part;
+
+		KUNIT_ASSERT_EQ(test, iov_iter_npages(&iter, INT_MAX),
+				npages - bvecq_kunit_pos2page(test, pos));
+
+		/* 3 steps forward. */
+		part = min(left, 3);
+		iov_iter_advance(&iter, 3); /* May be short. */
+		pos += part;
+		left -= part;
+		bvecq_kunit_check_next(test, &iter, &pos, &left, 3);
+
+		/* 2 steps back */
+		part = umin(2, left);
+		iov_iter_revert(&iter, part);

lib/tests/kunit_bvecq.c line 478

Quoted 10 lines
lib/tests/kunit_bvecq.c new file +10 -0
+		pos -= part;
+		left += part;
+		bvecq_kunit_check_next(test, &iter, &pos, &left, 2);
+
+		/* 1 more step back */
+		part = umin(1, left);
+		iov_iter_revert(&iter, part);
+		pos -= part;
+		left += part;
+		bvecq_kunit_check_next(test, &iter, &pos, &left, 1);

[Severity: Low]
Should the backward revert steps here be bounded by pos rather than left?

left is the number of unread bytes remaining ahead in the iterator, while pos is the number of bytes already advanced from the start. When the forward advance and check reach the end of the iterator (left == 0 on the last 6 iterations), umin(..., left) evaluates to 0, turning both iov_iter_revert() calls into no-ops instead of testing reverts at the tail of the bvecq chain.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author maintainer
#13 lore raw

PATCH v17 7/17 afs: Fix an inconsistency in the symlink code

In reply to David Howells

Fix the afs symlink code to set ->directory_size to the size of the buffer, not PAGE_SIZE-1 or i_size. This doesn't actually change the operation since the value is ignored by the symlink writeback path (which writes to the cache).

This was spotted by Claude.

---
 fs/afs/symlink.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/afs/symlink.c +1 -2
index 6b8c122877ca..a1dbd552ddff 100644
@@ -99,7 +99,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 		ret = netfs_alloc_folioq_buffer(NULL,
 						&vnode->directory, &cur_size, PAGE_SIZE,
 						mapping_gfp_mask(vnode->netfs.inode.i_mapping));
-		vnode->directory_size = PAGE_SIZE - 1;
+		vnode->directory_size = PAGE_SIZE;
 		if (ret < 0)
 			return ret;
 	}
@@ -116,7 +116,6 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
 			trace_afs_file_error(vnode, -EFBIG, afs_file_error_dir_big);
 			return -EFBIG;
 		}
-		vnode->directory_size = i_size;
 
 		/* Copy the symlink. */
 		symlink = kmalloc_flex(struct afs_symlink, content, i_size + 1);
#14 lore raw

Re: [PATCH v17 07/17] afs: Fix an inconsistency in the symlink code

Sashiko has reviewed this patch and found no issues.
Quoted 8 lines

Fix the afs symlink code to set ->directory_size to the size of the buffer, not PAGE_SIZE-1 or i_size. This doesn't actually change the operation since the value is ignored by the symlink writeback path (which writes to the cache).

This was spotted by Claude.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
author maintainer
#15 lore raw

PATCH v17 8/17 netfs: Explicitly pass tail padding to netfs_writeback_single()

In reply to David Howells

Change netfs_writeback_single() so that the function is now given an explicit content length in addition to the buffer iterator. The buffer must be rounded out to the cache DIO block size and netfslib will then draw on this excess space to write padding into the cache beyond the content length specified. The caller must also now clear the padding space.

AFS directory code is modified to comply with this, including allocating pre-zeroed buffer space and clearing excess buffer space if the content is shrunk by a third party. It's not done during writeback as that may happen many times for a directory.

AFS symlink code, however, is modified such that the clearing is done when the content is written to the cache (expected to be a one-off per symlink, if it happens).

A further patch will make better use of the two different lengths.

---
 Documentation/filesystems/netfs_library.rst | 47 +++++++++++++++------
 fs/afs/dir.c                                | 17 ++++++--
 fs/afs/dir_edit.c                           |  2 +-
 fs/afs/symlink.c                            | 16 ++++++-
 fs/netfs/write_issue.c                      | 13 ++++--
 include/linux/netfs.h                       |  2 +-
 6 files changed, 73 insertions(+), 24 deletions(-)
Documentation/filesystems/netfs_library.rst +34 -13
index 215a6621f3d3..1fe110ceb3f1 100644
@@ -437,22 +437,43 @@ Monolithic Files API
 
 There is also a special API set for files for which the content must be read in
 a single RPC (and not written back) and is maintained as a monolithic blob
-(e.g. an AFS directory), though it can be stored and updated in the local cache::
+(e.g. an AFS directory), though it can be stored and updated in the local
+cache.
+
+To start with, there's a function to synchronously read the contents of such a
+file into a buffer in a single operation::
+
+	ssize_t netfs_read_single(struct inode *inode,
+				  struct file *file,
+				  struct iov_iter *iter);
+
+If such a file is modified locally, it can be marked dirty with::
 
-	ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_iter *iter);
 	void netfs_single_mark_inode_dirty(struct inode *inode);
+
+which will schedule it for writeback later.  When the writeback eventually
+happens, the filesystem should call::
+
 	int netfs_writeback_single(struct address_space *mapping,
 				   struct writeback_control *wbc,
-				   struct iov_iter *iter);
+				   struct iov_iter *iter,
+				   size_t len);
+
+to write the data to the cache from its writepages method.
 
-The first function reads from a file into the given buffer, reading from the
-cache in preference if the data is cached there; the second function allows the
-inode to be marked dirty, causing a later writeback; and the third function can
-be called from the writeback code to write the data to the cache, if there is
-one.
+In both the read and writeback functions, the buffer needs to be rounded up to
+a multiple of the cache DIO size, with ``iter->count`` indicating the size of
+the buffer.  For the write function, ``len`` should indicate the size of the
+content which may be less than the buffer size.  The gap between ``len`` and the
+buffer size should be cleared by the filesystem as this will be used as padding
+if the write size has to be increased (e.g. for the cache DIO).
+
+Note that, for the moment, rounding the buffer up to a multiple of PAGE_SIZE
+instead of the cache DIO size should suffice.  DIO size > PAGE_SIZE is not
+currently supported.
 
 The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
-used.  The writeback function requires the buffer to be of ITER_FOLIOQ type.
+used.
 
 Clearing Stale Post-EOF Pagecache
 ---------------------------------
@@ -859,12 +880,12 @@ The methods defined in the table are:
    paths to ask the filesystem to update its idea of the file size.  If not
    given, netfslib will set i_size and i_blocks and update the local cache
    cookie.
-   
+
  * ``post_modify()``
 
    [Optional] This is called after netfslib writes to the pagecache or when it
    allows an mmap'd page to be marked as writable.
-   
+
  * ``begin_writeback()``
 
    [Optional] Netfslib calls this when processing a writeback request if it
@@ -872,7 +893,7 @@ The methods defined in the table are:
    indicating it must be written to the server.  This allows the filesystem to
    only set up writeback resources when it knows it's going to have to perform
    a write.
-   
+
  * ``prepare_write()``
 
    [Optional] This is called to allow the filesystem to limit the size of a
@@ -911,7 +932,7 @@ The methods defined in the table are:
    allows the filesystem to examine the state of the request, the subrequests
    in the indicated stream and of its own data and make adjustments or
    renegotiate resources.
-   
+
  * ``invalidate_cache()``
 
    [Optional] This is called by netfslib to invalidate data stored in the local
fs/afs/dir.c +13 -4
index 75f8f70cfeff..51c158d8f723 100644
@@ -252,7 +252,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 
 		ret = netfs_alloc_folioq_buffer(NULL,
 						&dvnode->directory, &cur_size, i_size,
-						mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+						mapping_gfp_mask(dvnode->netfs.inode.i_mapping) |
+						__GFP_ZERO);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
@@ -278,6 +279,13 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 			if (ret2 < 0)
 				ret = ret2;
 		}
+
+		if (ret >= 0 && ret < dvnode->directory_size) {
+			iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory,
+					     0, 0, dvnode->directory_size);
+			iov_iter_advance(&iter, ret);
+			iov_iter_zero(dvnode->directory_size - ret, &iter);
+		}
 	}
 
 	return ret;
@@ -2228,9 +2236,10 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				     i_size_read(&dvnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory,
+				     0, 0, dvnode->directory_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter,
+					     i_size_read(&dvnode->netfs.inode));
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
 	}
fs/afs/dir_edit.c +1 -1
index c31303059444..185e8819e424 100644
@@ -122,7 +122,7 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz
 
 		ret = netfs_alloc_folioq_buffer(
 			NULL, &dvnode->directory, &cur_size, blend,
-			mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+			mapping_gfp_mask(dvnode->netfs.inode.i_mapping) | __GFP_ZERO);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			goto fail;
fs/afs/symlink.c +14 -2
index a1dbd552ddff..c18e4f806423 100644
@@ -228,6 +228,10 @@ int afs_readlink(struct dentry *dentry, char __user *buffer, int buflen)
 /*
  * Write the symlink contents to the cache as a single blob.  We then throw
  * away the page we used to receive it.
+ *
+ * Note that the buffer tail is cleared under a shared lock on ->validate_lock
+ * - though hopefully there won't be racing writers since the only
+ * 'modification' is to be either read from the server or locally created.
  */
 int afs_symlink_writepages(struct address_space *mapping,
 			   struct writeback_control *wbc)
@@ -247,9 +251,17 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
+		size_t i_size = i_size_read(&vnode->netfs.inode);
+
+		if (i_size < vnode->directory_size) {
+			iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0,
+					     vnode->directory_size);
+			iov_iter_advance(&iter, i_size);
+			iov_iter_zero(vnode->directory_size - i_size, &iter);
+		}
 		iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				     i_size_read(&vnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+				     vnode->directory_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter, i_size);
 	}
 
 	if (ret == 0) {
fs/netfs/write_issue.c +10 -3
index b8237e1314d8..e041023811ed 100644
@@ -747,26 +747,33 @@ static int netfs_write_folio_single(struct netfs_io_request *wreq,
  * netfs_writeback_single - Write back a monolithic payload
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data to write, must be ITER_FOLIOQ.
+ * @iter: Data buffer to write from, must be ITER_FOLIOQ.
+ * @len: Amount of data to write.
  *
  * Write a monolithic, non-pagecache object back to the server and/or
  * the cache.
  *
+ * The data buffer may be bigger than @len (as set in @iter's count), and if
+ * so, this function will use the extra as needed to provide padding into the
+ * cache to handle DIO rounding, so the caller should clear it.
+ *
  * Return: 0 if successful; 1 if skipped due to lock conflict and WB_SYNC_NONE;
  * or a negative error code.
  */
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter)
+			   struct iov_iter *iter, size_t len)
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	struct folio_queue *fq;
-	size_t size = iov_iter_count(iter);
+	size_t size = len;
 	int ret;
 
 	if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
 		return -EIO;
+	if (WARN_ON_ONCE(iov_iter_count(iter) < len))
+		return -EIO;
 
 	if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
 		/* The VFS will have undirtied the inode. */
include/linux/netfs.h +1 -1
index 2cf1163aafa6..375fd415b4e5 100644
@@ -430,7 +430,7 @@ void netfs_single_mark_inode_dirty(struct inode *inode);
 ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_iter *iter);
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter);
+			   struct iov_iter *iter, size_t len);
 
 /* Address operations API */
 struct readahead_control;
#16 lore raw

Re: [PATCH v17 08/17] netfs: Explicitly pass tail padding to netfs_writeback_single()

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] afs: i_size exceeding directory_size in writepages triggers WARN_ON_ONCE() --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] afs: i_size exceeding directory_size in writepages triggers WARN_ON_ONCE()
--

commit e4f5c0122c57e02637340e7c5e9b182a6f565f23
Author: David Howells <[email protected]>

netfs: Explicitly pass tail padding to netfs_writeback_single()

This commit adds an explicit content length parameter to netfs_writeback_single() so the caller provides a padded buffer via the iterator while specifying the actual content size separately, and updates AFS directory and symlink writeback to zero tail padding and pass both sizes.

Quoted 4 lines
fs/afs/dir.c +0 -0
index 75f8f70cfeff0..51c158d8f723f 100644

[ ... ]

Quoted 18 lines
fs/afs/dir.c +4 -3
@@ -2228,9 +2236,10 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
-				     i_size_read(&dvnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory,
+				     0, 0, dvnode->directory_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter,
+					     i_size_read(&dvnode->netfs.inode));
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
 	}
fs/afs/symlink.c +0 -0
index a1dbd552ddff0..c18e4f8064234 100644

[ ... ]

Quoted 24 lines
fs/afs/symlink.c +10 -2
@@ -247,9 +251,17 @@ int afs_symlink_writepages(struct address_space *mapping,
 
 	if (vnode->directory &&
 	    atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
+		size_t i_size = i_size_read(&vnode->netfs.inode);
+
+		if (i_size < vnode->directory_size) {
+			iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0,
+					     vnode->directory_size);
+			iov_iter_advance(&iter, i_size);
+			iov_iter_zero(vnode->directory_size - i_size, &iter);
+		}
 		iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
-				     i_size_read(&vnode->netfs.inode));
-		ret = netfs_writeback_single(mapping, wbc, &iter);
+				     vnode->directory_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter, i_size);
 	}
 
 	if (ret == 0) {
fs/netfs/write_issue.c +0 -0
index b8237e1314d8a..e041023811ed0 100644

[ ... ]

Quoted 7 lines
fs/netfs/write_issue.c +1 -1
@@ -747,26 +747,33 @@ static int netfs_write_folio_single(struct netfs_io_request *wreq,
 int netfs_writeback_single(struct address_space *mapping,
 			   struct writeback_control *wbc,
-			   struct iov_iter *iter)
+			   struct iov_iter *iter, size_t len)
 {
 	struct netfs_io_request *wreq;

fs/netfs/write_issue.c static int netfs_write_folio_single(struct netfs_io_request *wreq, · line 760

Quoted 10 lines
fs/netfs/write_issue.c +3 -1
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
 	struct folio_queue *fq;
-	size_t size = iov_iter_count(iter);
+	size_t size = len;
 	int ret;
 
 	if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
 		return -EIO;
+	if (WARN_ON_ONCE(iov_iter_count(iter) < len))
+		return -EIO;

[Severity: Medium]
Can i_size exceed directory_size in afs_dir_writepages() or afs_symlink_writepages() and trigger the new WARN_ON_ONCE(iov_iter_count(iter) < len) in netfs_writeback_single()?

In afs_dir_writepages(), dvnode->validate_lock is held shared while checking AFS_VNODE_DIR_VALID and reading dvnode->directory_size and inode->i_size. A concurrent RPC completion such as afs_do_lookup_success() calls afs_vnode_commit_status() -> afs_apply_status() under cb_lock and i_lock without holding validate_lock. If a third party grew the directory on the server beyond dvnode->directory_size, afs_apply_status() clears AFS_VNODE_DIR_VALID and updates inode->i_size without updating dvnode->directory_size:

CPU0 CPU1
afs_dir_writepages()
down_read(&dvnode->validate_lock)
test_bit(AFS_VNODE_DIR_VALID, ...)

                                  afs_vnode_commit_status()
                                    afs_apply_status()
                                      afs_invalidate_dir()
                                        clear_bit(AFS_VNODE_DIR_VALID, ...)
                                      i_size_write(inode, status->size)
  iov_iter_folio_queue(...,
    dvnode->directory_size)

netfs_writeback_single(...,
i_size_read(...))
WARN_ON_ONCE(iov_iter_count(iter) < len) <-- triggers warning

Even after commit "afs: Use a bvecq to hold dir content rather than folioq" later in this series reads i_size and directory_size under dvnode->cb_lock, test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags) is still checked before acquiring cb_lock, so the same race window remains.

Similarly, afs_do_read_symlink() allocates vnode->directory to PAGE_SIZE. If the server returns a symlink status with size > PAGE_SIZE in netfs_read_single() (or a concurrent status update grows inode->i_size past PAGE_SIZE and sets AFS_VNODE_ZAP_DATA), netfs_rreq_assess_single() still marks the inode dirty before afs_do_read_symlink() returns -EFBIG. When afs_symlink_writepages() runs, it does not check AFS_VNODE_ZAP_DATA or whether i_size <= vnode->directory_size, hitting the same WARN_ON_ONCE() and returning -EIO (which also skips freeing vnode->directory until eviction).

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
author maintainer
#17 lore raw

PATCH v17 9/17 netfs: Change netfs_writeback_single() to not look inside the iterator

In reply to David Howells

Change netfs_writeback_single() so that it doesn't look around inside the data buffer iterator it is given and doesn't try to submit the folios from the list individually. Instead, it treats it as a DIO write and pushes the iterator directly, unsliced, to the cache (or the filesystem) to write from.

There are also some changes of behaviour:

(1) netfs_writeback_single() will make a single unbounded monolithic write to the cache.

(2) When writing, sreq_max_len/segs are ignored as the "single" type mandates all the data be transferred in a single op. Whilst this is not strictly necessary for the cache as the contract is really with the server, it simplifies things for now.

(3) An explicit wait is added to the function to make sure that the write is concluded before returning as there's currently no mechanism to check if the write is completed as page flags aren't used. This can be fixed later, but may require AFS's dir and symlink handling to check for write completion before allowing further changes be made.

---
 fs/netfs/write_issue.c | 172 +++++++----------------------------------
 1 file changed, 28 insertions(+), 144 deletions(-)
fs/netfs/write_issue.c +28 -144
index e041023811ed..f084a6796667 100644
@@ -625,133 +625,15 @@ int netfs_writepages(struct address_space *mapping,
 }
 EXPORT_SYMBOL(netfs_writepages);
 
-/*
- * Write some of a pending folio data back to the server and/or the cache.
- */
-static int netfs_write_folio_single(struct netfs_io_request *wreq,
-				    struct folio *folio)
-{
-	struct netfs_io_stream *upload = &wreq->io_streams[0];
-	struct netfs_io_stream *cache  = &wreq->io_streams[1];
-	struct netfs_io_stream *stream;
-	size_t iter_off = 0;
-	size_t fsize = folio_size(folio), flen;
-	uoff_t fpos = folio_pos(folio);
-	ssize_t ret;
-	bool to_eof = false;
-	bool no_debug = false;
-
-	_enter("");
-
-	flen = folio_size(folio);
-	if (flen > wreq->i_size - fpos) {
-		flen = wreq->i_size - fpos;
-		folio_zero_segment(folio, flen, fsize);
-		to_eof = true;
-	} else if (flen == wreq->i_size - fpos) {
-		to_eof = true;
-	}
-
-	_debug("folio %zx/%zx", flen, fsize);
-
-	if (!upload->avail && !cache->avail) {
-		trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
-		return 0;
-	}
-
-	if (!upload->construct)
-		trace_netfs_folio(folio, netfs_folio_trace_store);
-	else
-		trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-
-	/* Attach the folio to the rolling buffer. */
-	folio_get(folio);
-	ret = rolling_buffer_append(&wreq->buffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq->gfp);
-	if (ret < 0) {
-		folio_put(folio);
-		return ret;
-	}
-
-	/* Move the submission point forward to allow for write-streaming data
-	 * not starting at the front of the page.  We don't do write-streaming
-	 * with the cache as the cache requires DIO alignment.
-	 *
-	 * Also skip uploading for data that's been read and just needs copying
-	 * to the cache.
-	 */
-	for (int s = 0; s < NR_IO_STREAMS; s++) {
-		stream = &wreq->io_streams[s];
-		stream->submit_off = 0;
-		stream->submit_len = flen;
-		if (!stream->avail) {
-			stream->submit_off = UINT_MAX;
-			stream->submit_len = 0;
-		}
-	}
-
-	/* Attach the folio to one or more subrequests.  For a big folio, we
-	 * could end up with thousands of subrequests if the wsize is small -
-	 * but we might need to wait during the creation of subrequests for
-	 * network resources (eg. SMB credits).
-	 */
-	for (;;) {
-		ssize_t part;
-		size_t lowest_off = ULONG_MAX;
-		int choose_s = -1;
-
-		/* Always add to the lowest-submitted stream first. */
-		for (int s = 0; s < NR_IO_STREAMS; s++) {
-			stream = &wreq->io_streams[s];
-			if (stream->submit_len > 0 &&
-			    stream->submit_off < lowest_off) {
-				lowest_off = stream->submit_off;
-				choose_s = s;
-			}
-		}
-
-		if (choose_s < 0)
-			break;
-		stream = &wreq->io_streams[choose_s];
-
-		/* Advance the iterator(s). */
-		if (stream->submit_off > iter_off) {
-			rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
-			iter_off = stream->submit_off;
-		}
-
-		atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
-		stream->submit_extendable_to = fsize - stream->submit_off;
-		part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
-					   stream->submit_len, to_eof);
-		stream->submit_off += part;
-		if (part > stream->submit_len)
-			stream->submit_len = 0;
-		else
-			stream->submit_len -= part;
-		if (part > 0)
-			no_debug = true;
-	}
-
-	wreq->buffer.iter.iov_offset = 0;
-	if (fsize > iter_off)
-		rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
-	atomic64_set(&wreq->issued_to, fpos + fsize);
-
-	if (!no_debug)
-		kdebug("R=%x: No submit", wreq->debug_id);
-	_leave(" = 0");
-	return 0;
-}
-
 /**
- * netfs_writeback_single - Write back a monolithic payload
+ * netfs_writeback_single - Write back a monolithic payload to cache
  * @mapping: The mapping to write from
  * @wbc: Hints from the VM
- * @iter: Data buffer to write from, must be ITER_FOLIOQ.
+ * @iter: Data buffer to write from.
  * @len: Amount of data to write.
  *
- * Write a monolithic, non-pagecache object back to the server and/or
- * the cache.
+ * Write a monolithic, non-pagecache object back to the cache only.  The data
+ * is not written to the server.
  *
  * The data buffer may be bigger than @len (as set in @iter's count), and if
  * so, this function will use the extra as needed to provide padding into the
@@ -766,12 +648,8 @@ int netfs_writeback_single(struct address_space *mapping,
 {
 	struct netfs_io_request *wreq;
 	struct netfs_inode *ictx = netfs_inode(mapping->host);
-	struct folio_queue *fq;
-	size_t size = len;
 	int ret;
 
-	if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
-		return -EIO;
 	if (WARN_ON_ONCE(iov_iter_count(iter) < len))
 		return -EIO;
 
@@ -787,36 +665,42 @@ int netfs_writeback_single(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	wreq->buffer.iter = *iter;
+	wreq->len = len;
+	wreq->submitted = wreq->len;
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
 
-	if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
-		wreq->netfs_ops->begin_writeback(wreq);
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_subrequest *subreq;
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
+
+		if (!stream->avail)
+			continue;
 
-	for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-			size_t part = umin(folioq_folio_size(fq, slot), size);
+		netfs_prepare_write(wreq, stream, 0);
+		/* Allocation not allowed to fail in writeback. */
 
-			_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+		subreq = stream->construct;
+		subreq->len = wreq->len;
+		stream->submit_len = subreq->len;
+		stream->submit_extendable_to = iov_iter_count(iter) - len;
 
-			ret = netfs_write_folio_single(wreq, folio);
-			if (ret < 0)
-				goto stop;
-			size -= part;
-			if (size <= 0)
-				goto stop;
-		}
+		netfs_issue_write(wreq, stream);
 	}
 
-stop:
-	for (int s = 0; s < NR_IO_STREAMS; s++)
-		netfs_issue_write(wreq, &wreq->io_streams[s]);
 	netfs_all_subreqs_queued(wreq);
-
 	netfs_wake_collector(wreq);
 
+	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+	 * wait before modifying.
+	 */
+	ret = netfs_wait_for_write(wreq);
+	if (ret > 0)
+		ret = 0;
+
 	netfs_put_request(wreq, netfs_rreq_trace_put_return);
 	_leave(" = %d", ret);
 	return ret;
#18 lore raw

Re: [PATCH v17 09/17] netfs: Change netfs_writeback_single() to not look inside the iterator

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] netfs: netfs_write_collection() wakes waiter before netfs_wb_end() --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] netfs: netfs_write_collection() wakes waiter before netfs_wb_end()
--

commit 6578826ffb6d0041f876eafa90bcd2bd1b096881
Author: David Howells <[email protected]>

netfs: Change netfs_writeback_single() to not look inside the iterator

This commit changes netfs_writeback_single() to pass the caller's iterator directly to the write streams as a single monolithic write and wait synchronously for completion instead of iterating over the folios individually.

Quoted 4 lines
fs/netfs/write_issue.c +0 -0
index e041023811ed0..f084a6796667f 100644

[ ... ]

Quoted 47 lines
fs/netfs/write_issue.c +17 -17
@@ -787,36 +665,42 @@ int netfs_writeback_single(struct address_space *mapping,
 		goto couldnt_start;
 	}
 
+	wreq->buffer.iter = *iter;
+	wreq->len = len;
+	wreq->submitted = wreq->len;
+
 	__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
 	trace_netfs_write(wreq, netfs_write_trace_writeback_single);
 	netfs_stat(&netfs_n_wh_writepages);
 
-	if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
-		wreq->netfs_ops->begin_writeback(wreq);
+	for (int s = 0; s < NR_IO_STREAMS; s++) {
+		struct netfs_io_subrequest *subreq;
+		struct netfs_io_stream *stream = &wreq->io_streams[s];
+
+		if (!stream->avail)
+			continue;
 
-	for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
-		for (int slot = 0; slot < folioq_count(fq); slot++) {
-			struct folio *folio = folioq_folio(fq, slot);
-			size_t part = umin(folioq_folio_size(fq, slot), size);
+		netfs_prepare_write(wreq, stream, 0);
+		/* Allocation not allowed to fail in writeback. */
 
-			_debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+		subreq = stream->construct;
+		subreq->len = wreq->len;
+		stream->submit_len = subreq->len;
+		stream->submit_extendable_to = iov_iter_count(iter) - len;
 
-			ret = netfs_write_folio_single(wreq, folio);
-			if (ret < 0)
-				goto stop;
-			size -= part;
-			if (size <= 0)
-				goto stop;
-		}
+		netfs_issue_write(wreq, stream);
 	}
 
-stop:
-	for (int s = 0; s < NR_IO_STREAMS; s++)
-		netfs_issue_write(wreq, &wreq->io_streams[s]);

fs/netfs/write_issue.c int netfs_writeback_single(struct address_space *mapping, · line 702

Quoted 10 lines
fs/netfs/write_issue.c +6 -1
 	netfs_all_subreqs_queued(wreq);
-
 	netfs_wake_collector(wreq);
 
+	/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+	 * wait before modifying.
+	 */
+	ret = netfs_wait_for_write(wreq);
+	if (ret > 0)
+		ret = 0;

[Severity: Medium]
Can netfs_writeback_single() return while NETFS_ICTX_WB_LOCK is still held by the collector worker?

Because NETFS_RREQ_OFFLOAD_COLLECTION is set, netfs_wait_for_write() sleeps on wreq->waitq until NETFS_RREQ_IN_PROGRESS is cleared by the collector workqueue, which happens before netfs_wb_end() releases NETFS_ICTX_WB_LOCK:

fs/netfs/write_collect.c:netfs_write_collection() {
	...
	netfs_wake_rreq_flag(wreq, NETFS_RREQ_IN_PROGRESS, netfs_rreq_trace_wake_ip);
	/* As we cleared NETFS_RREQ_IN_PROGRESS, we acquired its ref. */

	switch (wreq->origin) {
	case NETFS_WRITEBACK:
	case NETFS_WRITEBACK_SINGLE:
		netfs_wb_end(ictx);
		break;
	...
}

CPU0 kworker
netfs_writeback_single()
netfs_wb_begin(ictx, ...)
...
netfs_wake_collector(wreq)
netfs_wait_for_write(wreq)

                                      netfs_write_collection()
                                        netfs_wake_rreq_flag(wreq,
                                          NETFS_RREQ_IN_PROGRESS, ...)

return 0;
netfs_writeback_single()
netfs_wb_begin(ictx, true) <-- sees NETFS_ICTX_WB_LOCK still set
netfs_wb_end(ictx)

An immediate subsequent WB_SYNC_NONE writeback on the same inode will see NETFS_ICTX_WB_LOCK still held, re-dirty the inode via netfs_single_mark_inode_dirty(), and abort the writeback pass.

Should netfs_write_collection() call netfs_wb_end() before waking NETFS_RREQ_IN_PROGRESS, or should netfs_writeback_single() omit NETFS_RREQ_OFFLOAD_COLLECTION while it waits synchronously?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
author maintainer
#19 lore raw

PATCH v17 10/17 netfs: Pass the write length to request creation where appropriate

In reply to David Howells

For unbuffered/DIO writes and writes to single-type objects, pass the length down into the request creation function so that wreq->len gets set correctly. For buffered writeback and pgpriv2 copying, this remains 0 as we don't know in advance how big the request will be.

A subsequent patch will use this to set as the i_size on single-type objects (e.g. afs directories) as we don't care what the old i_size was and we don't want to see inode->i_size in the netfs file changing due to third-party invalidation before we update fscache to say how big the object is going to be.

cc Paulo Alcantara (Red Hat) <[email protected]>
cc Matthew Wilcox <[email protected]>
cc Jens Axboe <[email protected]>
cc Christoph Hellwig <[email protected]>
---
 fs/netfs/direct_write.c | 4 ++--
 fs/netfs/internal.h     | 2 +-
 fs/netfs/read_pgpriv2.c | 2 +-
 fs/netfs/write_issue.c  | 9 +++++----
 4 files changed, 9 insertions(+), 8 deletions(-)
fs/netfs/direct_write.c +2 -2
index cc46b7d9321f..ff1795e1f0cb 100644
@@ -222,7 +222,7 @@ static void netfs_unbuffered_write_async(struct work_struct *work)
  * encrypted file.  This can also be used for direct I/O writes.
  */
 ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *iter,
-						  struct netfs_group *netfs_group)
+					   struct netfs_group *netfs_group)
 {
 	struct netfs_io_request *wreq;
 	ssize_t ret, n;
@@ -241,7 +241,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *
 
 	_debug("uw %llx-%llx", start, end);
 
-	wreq = netfs_create_write_req(iocb->ki_filp->f_mapping, iocb->ki_filp, start,
+	wreq = netfs_create_write_req(iocb->ki_filp->f_mapping, iocb->ki_filp, start, len,
 				      iocb->ki_flags & IOCB_DIRECT ?
 				      NETFS_DIO_WRITE : NETFS_UNBUFFERED_WRITE);
 	if (IS_ERR(wreq))
fs/netfs/internal.h +1 -1
index d554468bf694..6e30112c9112 100644
@@ -238,7 +238,7 @@ void netfs_write_collection_worker(struct work_struct *work);
  */
 struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 						struct file *file,
-						uoff_t start,
+						uoff_t start, size_t len,
 						enum netfs_io_origin origin);
 void netfs_prepare_write(struct netfs_io_request *wreq,
 			 struct netfs_io_stream *stream,
fs/netfs/read_pgpriv2.c +1 -1
index 5280b606fda4..85084aad9e31 100644
@@ -103,7 +103,7 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(
 	if (!fscache_resources_valid(&rreq->cache_resources))
 		goto cancel;
 
-	creq = netfs_create_write_req(rreq->mapping, NULL, folio_pos(folio),
+	creq = netfs_create_write_req(rreq->mapping, NULL, folio_pos(folio), 0,
 				      NETFS_PGPRIV2_COPY_TO_CACHE);
 	if (IS_ERR(creq))
 		goto cancel;
fs/netfs/write_issue.c +5 -4
index f084a6796667..4c1e597a6d03 100644
@@ -89,7 +89,7 @@ static void netfs_kill_dirty_pages(struct address_space *mapping,
  */
 struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 						struct file *file,
-						uoff_t start,
+						uoff_t start, size_t len,
 						enum netfs_io_origin origin)
 {
 	struct netfs_io_request *wreq;
@@ -98,7 +98,7 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 			     origin == NETFS_WRITEBACK_SINGLE ||
 			     origin == NETFS_PGPRIV2_COPY_TO_CACHE);
 
-	wreq = netfs_alloc_request(mapping, file, start, 0, origin);
+	wreq = netfs_alloc_request(mapping, file, start, len, origin);
 	if (IS_ERR(wreq))
 		return wreq;
 
@@ -573,7 +573,8 @@ int netfs_writepages(struct address_space *mapping,
 	if (!folio)
 		goto out;
 
-	wreq = netfs_create_write_req(mapping, NULL, folio_pos(folio), NETFS_WRITEBACK);
+	wreq = netfs_create_write_req(mapping, NULL, folio_pos(folio), 0,
+				      NETFS_WRITEBACK);
 	if (IS_ERR(wreq)) {
 		error = PTR_ERR(wreq);
 		goto couldnt_start;
@@ -659,7 +660,7 @@ int netfs_writeback_single(struct address_space *mapping,
 		return 1;
 	}
 
-	wreq = netfs_create_write_req(mapping, NULL, 0, NETFS_WRITEBACK_SINGLE);
+	wreq = netfs_create_write_req(mapping, NULL, 0, len, NETFS_WRITEBACK_SINGLE);
 	if (IS_ERR(wreq)) {
 		ret = PTR_ERR(wreq);
 		goto couldnt_start;
#20 lore raw

Re: [PATCH v17 10/17] netfs: Pass the write length to request creation where appropriate

Sashiko has reviewed this patch and found no issues.
Quoted 5 lines

For unbuffered/DIO writes and writes to single-type objects, pass the length down into the request creation function so that wreq->len gets set correctly. For buffered writeback and pgpriv2 copying, this remains 0 as we don't know in advance how big the request will be.

Quoted 7 lines

A subsequent patch will use this to set as the i_size on single-type objects (e.g. afs directories) as we don't care what the old i_size was and we don't want to see inode->i_size in the netfs file changing due to third-party invalidation before we update fscache to say how big the object is going to be.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
author maintainer
#21 lore raw

PATCH v17 11/17 afs: Use a bvecq to hold dir content rather than folioq

In reply to David Howells

Use a bvecq to hold the contents of a directory rather than the folioq so that the latter can be phased out. This is also applied to the symlink I/O buffer.

Also use the bvecq chain being refcounted to fix an issue in afs directory handling and netfs_writeback_single() whereby the application of third-party induced invalidation can race with the writeback code because afs doesn't hold the vnode validate_lock when updating the inode info.

Now, AFS FS RPC calls typically return a status update any time they access a vnode (file) on the server and the Linux afs fs driver applies it directly in the postprocessing of the RPC and, ideally, it would take the vnode's validate_lock whilst doing so - unfortunately, however, the caller may already be holding the validate_lock, as does afs_dir_writepages().

Worse, afs_dir_writepages() takes a shared lock so the inode info can't be updated exclusively under that lock.

Fix this by getting the directory contents, dir buffer size and inode size inside the cb_lock so that the set is at least consistent (if not currently valid). Invalidation will replace the cachefile anyway to discard in-progress old writebacks.

Further, in netfs_alloc_request() called from netfs_writeback_single(), use the content length parameter rather than i_size (we don't care about the cachefile size) on behalf of single-writeback.

---
 fs/afs/dir.c        | 54 ++++++++++++++++++++++++++-------------------
 fs/afs/dir_edit.c   | 44 +++++++++++++++++-------------------
 fs/afs/dir_search.c | 33 +++++++++++++--------------
 fs/afs/inode.c      |  2 +-
 fs/afs/internal.h   |  6 ++---
 fs/afs/symlink.c    | 42 +++++++++++++++++------------------
 fs/netfs/objects.c  |  4 ++++
 7 files changed, 95 insertions(+), 90 deletions(-)
fs/afs/dir.c +31 -23
index 51c158d8f723..9cefd90fd4f8 100644
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)
 	pr_warn("DIR %llx:%llx is=%llx\n",
 		dvnode->fid.vid, dvnode->fid.vnode, i_size);
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	iterate_folioq(&iter, iov_iter_count(&iter), NULL, NULL,
-		       afs_dir_dump_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iterate_bvecq(&iter, iov_iter_count(&iter), NULL, NULL,
+		      afs_dir_dump_step);
 }
 
 /*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)
 	if (unlikely(!i_size))
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
-	checked = iterate_folioq(&iter, iov_iter_count(&iter), dvnode, NULL,
-				 afs_dir_check_step);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	checked = iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, NULL,
+				afs_dir_check_step);
 	if (checked != i_size) {
 		afs_dir_dump(dvnode);
 		return -EIO;
@@ -250,16 +250,16 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 	if (dvnode->directory_size < i_size) {
 		size_t cur_size = dvnode->directory_size;
 
-		ret = netfs_alloc_folioq_buffer(NULL,
-						&dvnode->directory, &cur_size, i_size,
-						mapping_gfp_mask(dvnode->netfs.inode.i_mapping) |
-						__GFP_ZERO);
+		ret = bvecq_expand_buffer(&dvnode->directory, &cur_size,
+					  round_up(i_size, PAGE_SIZE),
+					  mapping_gfp_mask(dvnode->netfs.inode.i_mapping) |
+					  __GFP_ZERO);
 		dvnode->directory_size = cur_size;
 		if (ret < 0)
 			return ret;
 	}
 
-	iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
+	iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
 
 	/* AFS requires us to perform the read of a directory synchronously as
 	 * a single unit to avoid issues with the directory contents being
@@ -281,8 +281,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
 		}
 
 		if (ret >= 0 && ret < dvnode->directory_size) {
-			iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory,
-					     0, 0, dvnode->directory_size);
+			iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory,
+					    0, 0, dvnode->directory_size);
 			iov_iter_advance(&iter, ret);
 			iov_iter_zero(dvnode->directory_size - ret, &iter);
 		}
@@ -302,8 +302,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)
 }
 
 /*
- * Read the directory into a folio_queue buffer in one go, scrubbing the
- * previous contents.  We return -ESTALE if the caller needs to call us again.
+ * Read the directory into the buffer in one go, scrubbing the previous
+ * contents.  We return -ESTALE if the caller needs to call us again.
  */
 ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)
 	__acquires(&dvnode->validate_lock)
@@ -491,7 +491,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,
 }
 
 /*
- * Iterate through the directory folios.
+ * Iterate through the directory content.
  */
 static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)
 {
@@ -506,11 +506,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c
 	if (i_size <= 0 || dir_ctx->pos >= i_size)
 		return 0;
 
-	iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+	iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
 	iov_iter_advance(&iter, round_down(dir_ctx->pos, AFS_DIR_BLOCK_SIZE));
 
-	iterate_folioq(&iter, iov_iter_count(&iter), dvnode, &ctx,
-		       afs_dir_iterate_step);
+	iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, &ctx,
+		      afs_dir_iterate_step);
 
 	if (ctx.error == -ESTALE)
 		afs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);
@@ -2220,6 +2220,7 @@ static int afs_dir_writepages(struct address_space *mapping,
 			      struct writeback_control *wbc)
 {
 	struct afs_vnode *dvnode = AFS_FS_I(mapping->host);
+	struct bvecq *directory = NULL;
 	struct iov_iter iter;
 	int ret = 0;
 
@@ -2236,14 +2237,21 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory,
-				     0, 0, dvnode->directory_size);
-		ret = netfs_writeback_single(mapping, wbc, &iter,
-					     i_size_read(&dvnode->netfs.inode));
+		size_t i_size, d_size;
+
+		read_seqlock_excl(&dvnode->cb_lock);
+		i_size = i_size_read(&dvnode->netfs.inode);
+		d_size = dvnode->directory_size;
+		directory = bvecq_get(dvnode->directory);
+		read_sequnlock_excl(&dvnode->cb_lock);
+
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, directory, 0, 0, d_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter, i_size);
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
 	}
 
 	up_read(&dvnode->validate_lock);
+	bvecq_put(directory);
 	return ret;
 }
fs/afs/dir_edit.c +20 -24

77 lines not on this page · raw message

fs/afs/dir_search.c +15 -18
index 11ebdfffcb1d..e1f91b3ac6e2 100644
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)
  */
 union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)
 {
-	struct folio_queue *fq = iter->fq;
 	struct afs_vnode *dvnode = iter->dvnode;
-	struct folio *folio;
+	struct bvecq *bq = iter->bq;
 	size_t blpos = block * AFS_DIR_BLOCK_SIZE;
 	size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
-	int slot = iter->fq_slot;
+	int slot = iter->bq_slot;
 
 	_enter("%zx,%d", block, slot);
 
@@ -80,36 +79,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
 	if (dvnode->directory_size < blend)
 		goto fail;
 
-	if (!fq || blpos < fpos) {
-		fq = dvnode->directory;
+	if (!bq || blpos < fpos) {
+		bq = dvnode->directory;
 		slot = 0;
 		fpos = 0;
 	}
 
 	/* Search the folio queue for the folio containing the block... */
-	for (; fq; fq = fq->next) {
-		for (; slot < folioq_count(fq); slot++) {
-			size_t fsize = folioq_folio_size(fq, slot);
+	for (; bq; bq = bq->next) {
+		for (; slot < bq->nr_slots; slot++) {
+			struct bio_vec *bv = &bq->bv[slot];
+			size_t bsize = bv->bv_len;
 
-			if (blend <= fpos + fsize) {
+			if (blend <= fpos + bsize) {
 				/* ... and then return the mapped block. */
-				folio = folioq_folio(fq, slot);
-				if (WARN_ON_ONCE(folio_pos(folio) != fpos))
-					goto fail;
-				iter->fq = fq;
-				iter->fq_slot = slot;
+				iter->bq = bq;
+				iter->bq_slot = slot;
 				iter->fpos = fpos;
-				iter->block = kmap_local_folio(folio, blpos - fpos);
+				iter->block = bvec_kmap_partial(bv, blpos - fpos);
 				return iter->block;
 			}
-			fpos += fsize;
+			fpos += bsize;
 		}
 		slot = 0;
 	}
 
 fail:
-	iter->fq = NULL;
-	iter->fq_slot = 0;
+	iter->bq = NULL;
+	iter->bq_slot = 0;
 	afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
 	return NULL;
 }
fs/afs/inode.c +1 -1
index 0733abff77a5..af95640da03f 100644
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)
 	flush_delayed_work(&vnode->lock_work);
 	netfs_wait_for_outstanding_io(inode);
 	truncate_inode_pages_final(&inode->i_data);
-	netfs_free_folioq_buffer(vnode->directory);
+	bvecq_put(vnode->directory);
 	if (vnode->symlink)
 		afs_evict_symlink(vnode);
 
fs/afs/internal.h +3 -3
index 7cea5ba0d20d..2a8b029b4d69 100644
@@ -712,7 +712,7 @@ struct afs_vnode {
 #define AFS_VNODE_MODIFYING	10		/* Set if we're performing a modification op */
 #define AFS_VNODE_DIR_READ	11		/* Set if we've read a dir's contents */
 
-	struct folio_queue	*directory;	/* Directory contents */
+	struct bvecq		*directory;	/* Directory contents */
 	struct afs_symlink __rcu *symlink;	/* Symlink content */
 	struct list_head	wb_keys;	/* List of keys available for writeback */
 	struct list_head	pending_locks;	/* locks waiting to be granted */
@@ -993,9 +993,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl
 struct afs_dir_iter {
 	struct afs_vnode	*dvnode;
 	union afs_xdr_dir_block *block;
-	struct folio_queue	*fq;
+	struct bvecq		*bq;
 	unsigned int		fpos;
-	int			fq_slot;
+	int			bq_slot;
 	unsigned int		loop_check;
 	u8			nr_slots;
 	u8			bucket;
fs/afs/symlink.c +21 -21

97 lines not on this page · raw message

fs/netfs/objects.c +4 -0
index 6d7839c0498d..9895cf662bca 100644
@@ -78,6 +78,10 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
 		rreq->io_streams[0].avail = true;
 	} else {
 		INIT_WORK(&rreq->work, netfs_write_collection_worker);
+
+		/* For single-type, we disregard the previous content. */
+		if (origin == NETFS_WRITEBACK_SINGLE)
+			rreq->i_size = rreq->len;
 	}
 
 	__set_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
#22 lore raw

Re: [PATCH v17 11/17] afs: Use a bvecq to hold dir content rather than folioq

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] afs: TOCTOU race checking AFS_VNODE_DIR_VALID in afs_dir_writepages() --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] afs: TOCTOU race checking AFS_VNODE_DIR_VALID in afs_dir_writepages()
--

commit 6db299c57b12dc739490b1b98b5287a7b4f00825
Author: David Howells <[email protected]>

afs: Use a bvecq to hold dir content rather than folioq

This commit replaces folio_queue buffers with bvecq buffers for AFS directories and symlinks, and snapshots the directory size, inode size, and bvecq buffer under cb_lock during directory writeback.

Quoted 4 lines
fs/afs/dir.c +0 -0
index 51c158d8f723f..9cefd90fd4f8b 100644

[ ... ]

Quoted 19 lines
fs/afs/dir.c +4 -4
@@ -2220,6 +2220,7 @@ static int afs_dir_writepages(struct address_space *mapping,
 			      struct writeback_control *wbc)
 {
 	struct afs_vnode *dvnode = AFS_FS_I(mapping->host);
+	struct bvecq *directory = NULL;
 	struct iov_iter iter;
 	int ret = 0;
 
@@ -2236,14 +2237,21 @@ static int afs_dir_writepages(struct address_space *mapping,
 	}
 
 	if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
-		iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory,
-				     0, 0, dvnode->directory_size);
-		ret = netfs_writeback_single(mapping, wbc, &iter,
-					     i_size_read(&dvnode->netfs.inode));
+		size_t i_size, d_size;
+
+		read_seqlock_excl(&dvnode->cb_lock);

fs/afs/dir.c static int afs_dir_writepages(struct address_space *mapping, · line 2252

Quoted 10 lines
fs/afs/dir.c +7 -0
+		i_size = i_size_read(&dvnode->netfs.inode);
+		d_size = dvnode->directory_size;
+		directory = bvecq_get(dvnode->directory);
+		read_sequnlock_excl(&dvnode->cb_lock);
+
+		iov_iter_bvec_queue(&iter, ITER_SOURCE, directory, 0, 0, d_size);
+		ret = netfs_writeback_single(mapping, wbc, &iter, i_size);
 		if (ret == 1)
 			ret = 0; /* Skipped write due to lock conflict. */
 	}

[Severity: Medium]
Can afs_dir_writepages() still read a new i_size paired with an old directory_size if a third-party status update races between the AFS_VNODE_DIR_VALID check and read_seqlock_excl(&dvnode->cb_lock)?

CPU0 CPU1
afs_dir_writepages()
down_read(&dvnode->validate_lock)
test_bit(AFS_VNODE_DIR_VALID, ...)

                                  afs_vnode_commit_status()
                                    write_seqlock(&dvnode->cb_lock)
                                    afs_apply_status()
                                      afs_invalidate_dir()
                                        clear_bit(AFS_VNODE_DIR_VALID, ...)
                                      netfs_write_sizes()
                                        i_size_write(..., status->size)
                                    write_sequnlock(&dvnode->cb_lock)

read_seqlock_excl(&dvnode->cb_lock)
i_size = i_size_read(...)
d_size = dvnode->directory_size read_sequnlock_excl(&dvnode->cb_lock) netfs_writeback_single(..., i_size)
WARN_ON_ONCE(iov_iter_count(iter) < len) <-- i_size > d_size

When afs_apply_status() processes a third-party directory change under cb_lock without holding validate_lock, it clears AFS_VNODE_DIR_VALID and updates inode->i_size to the new server size, while leaving dvnode->directory_size and dvnode->directory unchanged until the directory is later re-read. If the directory grew on the server beyond d_size, netfs_writeback_single() hits WARN_ON_ONCE(iov_iter_count(iter) < len) and fails with -EIO; otherwise it writes the invalidated buffer with the new i_size to the cache.

Re-checking AFS_VNODE_DIR_VALID inside the cb_lock critical section would ensure i_size and directory_size come from a valid, consistent state.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
author maintainer
#23 lore raw

PATCH v17 12/17 cifs: Use a bvecq for buffering instead of a folioq

In reply to David Howells

Use a bvecq for internal buffering for crypto purposes instead of a folioq so that the latter can be phased out.

---
 fs/smb/client/cifsglob.h |  2 +-
 fs/smb/client/smb2ops.c  | 78 +++++++++++++++++++---------------------
 2 files changed, 38 insertions(+), 42 deletions(-)
fs/smb/client/cifsglob.h +1 -1
index d3ca688b6aba..806242d9cce0 100644
@@ -290,7 +290,7 @@ struct smb_rqst {
 	struct kvec	*rq_iov;	/* array of kvecs */
 	unsigned int	rq_nvec;	/* number of kvecs in array */
 	struct iov_iter	rq_iter;	/* Data iterator */
-	struct folio_queue *rq_buffer;	/* Buffer for encryption */
+	struct bvecq	*rq_buffer;	/* Buffer for encryption */
 };
 
 struct mid_q_entry;
fs/smb/client/smb2ops.c +37 -41

171 lines not on this page · raw message

#24 lore raw

Re: [PATCH v17 12/17] cifs: Use a bvecq for buffering instead of a folioq

Sashiko has reviewed this patch and found no issues.
Quoted 4 lines

Use a bvecq for internal buffering for crypto purposes instead of a folioq so that the latter can be phased out.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
author maintainer
#25 lore raw

PATCH v17 13/17 smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()

In reply to David Howells

Add support for ITER_BVECQ to smbdirect_map_sges_from_iter().

---
 fs/smb/smbdirect/connection.c | 67 +++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)
fs/smb/smbdirect/connection.c +67 -0
index e6c7b343fe87..b13bd07ffb8c 100644
@@ -5,6 +5,7 @@
  */
 
 #include "internal.h"
+#include <linux/bvecq.h>
 #include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
@@ -2015,6 +2016,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,
 	return ret;
 }
 
+/*
+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA
+ * list.  The fragments are not pinned.
+ */
+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,
+					     struct smbdirect_map_sges *state,
+					     ssize_t maxsize)
+{
+	const struct bvecq *bq = iter->bvecq, *next;
+	unsigned int slot = iter->bvecq_slot;
+	ssize_t extracted = 0;
+	size_t offset = iter->iov_offset;
+
+	maxsize = umin(maxsize, iov_iter_count(iter));
+
+	do {
+		struct bio_vec *bv;
+		size_t bsize;
+
+		while (slot >= bq->nr_slots) {
+			next = bvecq_next(bq);
+			if (!next) {
+				if (WARN_ON_ONCE(maxsize > 0))
+					return -EIO;
+				goto out;
+			}
+			bq = next;
+			slot = 0;
+		}
+
+		bv = &bq->bv[slot];
+		bsize = bv->bv_len;
+
+		if (offset < bsize) {
+			size_t part = umin(maxsize, bsize - offset);
+			bool ok;
+
+			ok = smbdirect_map_sges_single_page(state,
+							    bv->bv_page,
+							    bv->bv_offset + offset,
+							    part);
+			if (!ok)
+				return -EIO;
+
+			offset += part;
+			extracted += part;
+			maxsize -= part;
+		}
+
+		if (offset >= bsize) {
+			offset = 0;
+			slot++;
+		}
+	} while (state->num_sge < state->max_sge && maxsize > 0);
+
+out:
+	iter->bvecq = bq;
+	iter->bvecq_slot = slot;
+	iter->iov_offset = offset;
+	iter->count -= extracted;
+	return extracted;
+}
+
 /*
  * Extract fragments from a KVEC-class iterator and add them to an ib_sge list.
  * This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2164,6 +2228,9 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_BVEC:
 		ret = smbdirect_map_sges_from_bvec(iter, state, len);
 		break;
+	case ITER_BVECQ:
+		ret = smbdirect_map_sges_from_bvecq(iter, state, len);
+		break;
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;
#26 lore raw

Re: [PATCH v17 13/17] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()

Sashiko has reviewed this patch and found no issues.
Quoted 3 lines

Add support for ITER_BVECQ to smbdirect_map_sges_from_iter().

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
author maintainer
#27 lore raw

PATCH v17 14/17 netfs: Switch folioq to bvecq

In reply to David Howells

In netfslib, perform more or less a straight switch from using folio_queue to using bvecq to hold the lists of folios involved in various sorts of buffered read and in buffered writeback.

It's not quite a straight swap, however, because:

(1) The bveq_alloc_*() routines want to know if the callers is doing writeback for emergency pool use.

(2) folio_queue includes a fixed capacity folio_batch, but bvecq has a variable size bio_vec array.

(3) folio_queue has a set of per-folio marks that are mostly unused; the one exception is for netfs_prefetch_for_write() - but the marked folio is then ignored because no_unlock_folio points to it (and the bvecq mem cleanup type can handle that anyway). This will become an issue if/when netfs_prefetch_for_write() expands the read to cope with large cache granularity.

(4) The bvecq API doesn't have functions to get a folio's length or to clear a folio pointer by slot, instead accessing the bio_vec array directly. Also bvecq stores the folio length in bv_len instead of storing the folio orders.

(5) rolling_buffer_bulk_load_from_ra() needs to work a bit differently as the folio_queue contains a folio_batch and bvecq doesn't.

(6) rolling_buffer_delete_spent() has to clear the bvecq->next pointer before putting the bvecq to avoid rolling up the entire list. On the other hand, rolling_buffer_clear() just needs to put the tail.

(7) The bvecq chain needs to be pinned by the subrequests to prevent the retry reverting the iterator and hitting a spent link in the chain that has been removed. (Note that the intention is to remove the iterator from the netfs_io_subrequest struct and push the use of iterators down into the filesystem in a later patchset).

---
 fs/netfs/buffered_read.c       |  36 ++++-----
 fs/netfs/iterator.c            |  42 +++++-----
 fs/netfs/objects.c             |   1 +
 fs/netfs/read_collect.c        |  48 +++++------
 fs/netfs/read_pgpriv2.c        |  26 +++---
 fs/netfs/read_retry.c          |  53 ++++++++----
 fs/netfs/rolling_buffer.c      | 144 ++++++++++++++-------------------
 fs/netfs/write_collect.c       |  26 +++---
 fs/netfs/write_issue.c         |  26 ++++--
 fs/netfs/write_retry.c         |  15 +++-
 include/linux/netfs.h          |   1 +
 include/linux/rolling_buffer.h |  42 ++++------
 include/trace/events/netfs.h   |   2 +-
 13 files changed, 239 insertions(+), 223 deletions(-)
fs/netfs/buffered_read.c +18 -18
index 69e68ef0652a..c6c850ffd56e 100644
@@ -131,6 +131,8 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 	}
 
 	subreq->io_iter	= rreq->buffer.iter;
+	if (iov_iter_is_bvecq(&subreq->io_iter))
+		subreq->bvecq_pin = bvecq_get((struct bvecq *)subreq->io_iter.bvecq);
 
 	iov_iter_truncate(&subreq->io_iter, subreq->len);
 	rolling_buffer_advance(&rreq->buffer, subreq->len);
@@ -215,7 +217,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  * otherwise we set the deprecated PG_private_2.
  */
 static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
-				     struct folio_queue **fq,
+				     struct bvecq **bq,
 				     unsigned int *offset,
 				     int *slot,
 				     size_t len,
@@ -225,21 +227,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
 		struct folio *folio;
 		size_t fsize, overlap;
 
-		if (!*fq)
+		if (!*bq)
 			break;
-		if (*slot >= folioq_count(*fq)) {
-			*fq = (*fq)->next;
+		if (!bvecq_acquire_slot(*bq, *slot)) {
+			*bq = bvecq_next(*bq);
 			*slot = 0;
 			*offset = 0;
 			continue;
 		}
 
 		/* Determine how much the subreq overlaps the folio, if at all. */
-		fsize = folioq_folio_size(*fq, *slot);
+		fsize = (*bq)->bv[*slot].bv_len;
 		overlap = min(len, fsize - *offset);
 
 		if (overlap > 0 && copy) {
-			folio = folioq_folio(*fq, *slot);
+			folio = bvec_folio(&(*bq)->bv[*slot]);
 			if (netfs_using_pgpriv2(rreq)) {
 				if (!folio_test_private_2(folio))
 					folio_start_private_2(folio);
@@ -275,7 +277,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		.cached_to[1]	= ULLONG_MAX,
 	};
 	struct fscache_occupancy *occ = &_occ;
-	struct folio_queue *fq = rreq->buffer.tail;
+	struct bvecq *bq = rreq->buffer.tail;
 	unsigned int offset = 0;
 	ssize_t size = rreq->len;
 	uoff_t start = rreq->start;
@@ -408,10 +410,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		if (size <= 0)
 			netfs_all_subreqs_queued(rreq);
 
-		if (fq) {
+		if (bq) {
 			/* See if the cache indicated this should be cached. */
 			copy = test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
-			netfs_mark_copy_to_cache(rreq, &fq, &slot, &offset, slice, copy);
+			netfs_mark_copy_to_cache(rreq, &bq, &slot, &offset, slice, copy);
 		}
 
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
@@ -479,8 +481,7 @@ void netfs_readahead(struct readahead_control *ractl)
 	 * acquires a ref on each folio that we will need to release later -
 	 * but we don't want to do that until after we've started the I/O.
 	 */
-	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl,
-						 rreq->debug_id, rreq->gfp);
+	added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->gfp);
 	if (added < 0) {
 		ret = added;
 		goto cleanup_free;
@@ -503,15 +504,14 @@ EXPORT_SYMBOL(netfs_readahead);
 /*
  * Create a rolling buffer with a single occupying folio.
  */
-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,
-					unsigned int rollbuf_flags)
+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
 {
 	ssize_t added;
 
-	if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+	if (rolling_buffer_init(&rreq->buffer, ITER_DEST, rreq->gfp, false) < 0)
 		return -ENOMEM;
 
-	added = rolling_buffer_append(&rreq->buffer, folio, rollbuf_flags, rreq->gfp);
+	added = rolling_buffer_append(&rreq->buffer, folio, rreq->gfp);
 	if (added < 0)
 		return added;
 	rreq->submitted = rreq->start + added;
@@ -668,7 +668,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
 	trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto discard;
 
@@ -825,7 +825,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
 	trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, 0);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
@@ -890,7 +890,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
 	trace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);
 
 	/* Set up the output buffer */
-	ret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);
+	ret = netfs_create_singular_buffer(rreq, folio);
 	if (ret < 0)
 		goto error_put;
 
fs/netfs/iterator.c +21 -21

73 lines not on this page · raw message

fs/netfs/objects.c +1 -0
index 9895cf662bca..0ce5d528b7fe 100644
@@ -254,6 +254,7 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)
 	trace_netfs_sreq(subreq, netfs_sreq_trace_free);
 	if (rreq->netfs_ops->free_subrequest)
 		rreq->netfs_ops->free_subrequest(subreq);
+	bvecq_put(subreq->bvecq_pin);
 	mempool_free(subreq, rreq->netfs_ops->subrequest_pool ?: &netfs_subrequest_pool);
 	netfs_stat_d(&netfs_n_rh_sreq);
 	netfs_put_request(rreq, netfs_rreq_trace_put_subreq);
fs/netfs/read_collect.c +24 -24

128 lines not on this page · raw message

fs/netfs/read_pgpriv2.c +14 -12
index 85084aad9e31..037fa80037c7 100644
@@ -53,7 +53,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
 	trace_netfs_folio(folio, netfs_folio_trace_store_copy);
 
 	/* Attach the folio to the rolling buffer. */
-	if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
+	if (rolling_buffer_append(&creq->buffer, folio, creq->gfp) < 0) {
 		set_bit(NETFS_RREQ_CANCEL_CACHING, &creq->flags);
 		folio_end_private_2(folio);
 		return;
@@ -173,13 +173,15 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
  */
 bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 {
-	struct folio_queue *folioq = creq->buffer.tail;
+	struct bvecq *bq = creq->buffer.tail;
 	unsigned int slot = creq->buffer.first_tail_slot;
 	uoff_t collected_to = creq->collected_to;
 	bool made_progress = false;
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&creq->buffer);
+	while (!bvecq_acquire_slot(bq, slot)) {
+		bq = rolling_buffer_delete_spent(&creq->buffer);
+		if (!bq)
+			return false;
 		slot = 0;
 	}
 
@@ -188,7 +190,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 		uoff_t fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		folio = bvec_folio(&bq->bv[slot]);
 		if (WARN_ONCE(!folio_test_private_2(folio),
 			      "R=%08x: folio %lx is not marked private_2\n",
 			      creq->debug_id, folio->index))
@@ -211,15 +213,15 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 		creq->cleaned_to = fpos + fsize;
 		made_progress = true;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
+		/* Clean up the head bq.  If we clear an entire bq, then
+		 * we can get rid of it provided it's not also the tail bq
 		 * being filled by the issuer.
 		 */
-		folioq_clear(folioq, slot);
+		bq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&creq->buffer);
-			if (!folioq)
+		while (!bvecq_acquire_slot(bq, slot)) {
+			bq = rolling_buffer_delete_spent(&creq->buffer);
+			if (!bq)
 				goto done;
 			slot = 0;
 		}
@@ -228,7 +230,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
 			break;
 	}
 
-	creq->buffer.tail = folioq;
+	creq->buffer.tail = bq;
 done:
 	creq->buffer.first_tail_slot = slot;
 	return made_progress;
fs/netfs/read_retry.c +38 -15

111 lines not on this page · raw message

fs/netfs/rolling_buffer.c +61 -83

235 lines not on this page · raw message

fs/netfs/write_collect.c +13 -13
index 6e8ea534230d..91b42820c892 100644
@@ -114,11 +114,11 @@ int netfs_folio_written_back(struct folio *folio)
 static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 					  unsigned int *notes)
 {
-	struct folio_queue *folioq = wreq->buffer.tail;
+	struct bvecq *bq = wreq->buffer.tail;
 	unsigned int slot = wreq->buffer.first_tail_slot;
 	uoff_t collected_to = wreq->collected_to;
 
-	if (WARN_ON_ONCE(!folioq)) {
+	if (WARN_ON_ONCE(!bq)) {
 		pr_err("[!] Writeback unlock found empty rolling buffer!\n");
 		netfs_dump_request(wreq);
 		return;
@@ -130,9 +130,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		return;
 	}
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&wreq->buffer);
-		if (!folioq)
+	while (!bvecq_acquire_slot(bq, slot)) {
+		bq = rolling_buffer_delete_spent(&wreq->buffer);
+		if (!bq)
 			return;
 		slot = 0;
 	}
@@ -143,7 +143,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		uoff_t fpos, fend;
 		size_t fsize, flen;
 
-		folio = folioq_folio(folioq, slot);
+		folio = bvec_folio(&bq->bv[slot]);
 		if (WARN_ONCE(!folio_test_writeback(folio),
 			      "R=%08x: folio %lx is not under writeback\n",
 			      wreq->debug_id, folio->index))
@@ -166,15 +166,15 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		wreq->cleaned_to = fpos + fsize;
 		*notes |= MADE_PROGRESS;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
+		/* Clean up the head bq.  If we clear an entire bq, then
+		 * we can get rid of it provided it's not also the tail bq
 		 * being filled by the issuer.
 		 */
-		folioq_clear(folioq, slot);
+		bq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&wreq->buffer);
-			if (!folioq)
+		while (!bvecq_acquire_slot(bq, slot)) {
+			bq = rolling_buffer_delete_spent(&wreq->buffer);
+			if (!bq)
 				goto done;
 			slot = 0;
 		}
@@ -183,7 +183,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 			break;
 	}
 
-	wreq->buffer.tail = folioq;
+	wreq->buffer.tail = bq;
 done:
 	wreq->buffer.first_tail_slot = slot;
 }
fs/netfs/write_issue.c +19 -7
index 4c1e597a6d03..e7b1d5c3baa6 100644
@@ -107,7 +107,9 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
 	ictx = netfs_inode(wreq->inode);
 	if (is_cacheable)
 		fscache_begin_write_operation(&wreq->cache_resources, netfs_i_cookie(ictx));
-	if (rolling_buffer_init(&wreq->buffer, wreq->debug_id, ITER_SOURCE, wreq->gfp) < 0)
+	if (rolling_buffer_init(&wreq->buffer, ITER_SOURCE, wreq->gfp,
+				(origin == NETFS_WRITEBACK ||
+				 origin == NETFS_WRITEBACK_SINGLE)) < 0)
 		goto nomem;
 
 	wreq->cleaned_to = wreq->start;
@@ -163,12 +165,12 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 	struct netfs_io_subrequest *subreq;
 	struct iov_iter *wreq_iter = &wreq->buffer.iter;
 
-	/* Make sure we don't point the iterator at a used-up folio_queue
-	 * struct being used as a placeholder to prevent the queue from
-	 * collapsing.  In such a case, extend the queue.
+	/* Make sure we don't point the iterator at a used-up bvecq struct
+	 * being used as a placeholder to prevent the queue from collapsing.
+	 * In such a case, extend the queue.
 	 */
-	if (iov_iter_is_folioq(wreq_iter) &&
-	    wreq_iter->folioq_slot >= folioq_nr_slots(wreq_iter->folioq))
+	if (iov_iter_is_bvecq(wreq_iter) &&
+	    !bvecq_acquire_slot(wreq_iter->bvecq, wreq_iter->bvecq_slot))
 		rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
 
 	subreq = netfs_alloc_subrequest(wreq, stream->source);
@@ -177,6 +179,8 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
 	subreq->start		= start;
 	subreq->stream_nr	= stream->stream_nr;
 	subreq->io_iter		= *wreq_iter;
+	if (iov_iter_is_bvecq(wreq_iter))
+		subreq->bvecq_pin = bvecq_get((struct bvecq *)wreq->buffer.iter.bvecq);
 
 	_enter("R=%x[%x]", wreq->debug_id, subreq->debug_index);
 
@@ -267,6 +271,14 @@ void netfs_reissue_write(struct netfs_io_stream *stream,
 	iov_iter_advance(source, size);
 	iov_iter_truncate(&subreq->io_iter, size);
 
+	if (iov_iter_is_bvecq(&subreq->io_iter) &&
+	    subreq->bvecq_pin != subreq->io_iter.bvecq) {
+		struct bvecq *old = subreq->bvecq_pin;
+
+		subreq->bvecq_pin = bvecq_get((struct bvecq *)subreq->io_iter.bvecq);
+		bvecq_put(old);
+	}
+
 	subreq->retry_count++;
 	subreq->error = 0;
 	__clear_bit(NETFS_SREQ_MADE_PROGRESS, &subreq->flags);
@@ -451,7 +463,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
 	}
 
 	/* Attach the folio to the rolling buffer. */
-	rolling_buffer_append(&wreq->buffer, folio, 0, wreq->gfp);
+	rolling_buffer_append(&wreq->buffer, folio, wreq->gfp);
 
 	/* Move the submission point forward to allow for write-streaming data
 	 * not starting at the front of the page.  We don't do write-streaming
fs/netfs/write_retry.c +14 -1
index 2f20577563e1..81c106ef37f7 100644
@@ -18,6 +18,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 				     struct netfs_io_stream *stream)
 {
 	struct list_head *next;
+	struct bvecq *pin = NULL;
 
 	_enter("R=%x[%x:]", wreq->debug_id, stream->stream_nr);
 
@@ -59,6 +60,9 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		size_t part;
 		bool boundary = false;
 
+		bvecq_put(pin);
+		pin = NULL;
+
 		/* Go through the stream and find the next span of contiguous
 		 * data that we then rejig (cifs, for example, needs the wsize
 		 * renegotiating) and reissue.
@@ -70,7 +74,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 
 		if (test_bit(NETFS_SREQ_FAILED, &from->flags) ||
 		    !test_bit(NETFS_SREQ_NEED_RETRY, &from->flags))
-			return;
+			break;
 
 		for (;;) {
 			/* Read pointer to subreq before reading subreq state. */
@@ -87,6 +91,13 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 			len += to->len;
 		}
 
+		/* Pin start of the bvecq from where the first subreq in the
+		 * sequence started.  We will advance the subreqs' pins to
+		 * their new starting points and then drop this pin.
+		 */
+		if (from->bvecq_pin)
+			pin = bvecq_get(from->bvecq_pin);
+
 		/* Determine the set of buffers we're going to use.  Each
 		 * subreq gets a subset of a single overall contiguous buffer.
 		 */
@@ -197,6 +208,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,
 		} while (len);
 
 	} while (!list_is_head(next, &stream->subrequests));
+
+	bvecq_put(pin);
 }
 
 /*
include/linux/netfs.h +1 -0
index 375fd415b4e5..6f3276e8eeb0 100644
@@ -188,6 +188,7 @@ struct netfs_io_subrequest {
 	struct netfs_io_request *rreq;		/* Supervising I/O request */
 	struct work_struct	work;
 	struct list_head	rreq_link;	/* Link in rreq->subrequests */
+	struct bvecq		*bvecq_pin;	/* Pin in request's bvecq chain (holds ref) */
 	struct iov_iter		io_iter;	/* Iterator for this subrequest */
 	uoff_t			start;		/* Where to start the I/O */
 	size_t			len;		/* Size of the I/O */
include/linux/rolling_buffer.h +14 -28

63 lines not on this page · raw message

include/trace/events/netfs.h +1 -1
index bf1e1f185b05..41f7f0733bfa 100644
@@ -398,7 +398,7 @@ TRACE_EVENT(netfs_sreq,
 		    __entry->len	= sreq->len;
 		    __entry->transferred = sreq->transferred;
 		    __entry->start	= sreq->start;
-		    __entry->slot	= sreq->io_iter.folioq_slot;
+		    __entry->slot	= sreq->io_iter.bvecq_slot;
 			   ),
 
 	    TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
#28 lore raw

Re: [PATCH v17 14/17] netfs: Switch folioq to bvecq

Thank you for your contribution! Sashiko AI review found 3 potential issues to consider: - [Critical] netfs: use-after-free on spent bvecq in netfs_read_to_pagecache() - [High] netfs: premature…

Thank you for your contribution! Sashiko AI review found 3 potential issues to consider:
- [Critical] netfs: use-after-free on spent bvecq in netfs_read_to_pagecache()
- [High] netfs: premature bvecq deletion race in folio unlock functions
- [Low] netfs: commit message mismatch for rolling_buffer_delete_spent() --

commit e56d7d03f56d46856c2af8166a341d7fbaef3849
Author: David Howells <[email protected]>

netfs: Switch folioq to bvecq

This commit switches netfslib's rolling buffer and buffered read/writeback paths from folio_queue to bvecq and pins bvecq segments on subrequests across retries.

Quoted 3 lines

(6) rolling_buffer_delete_spent() has to clear the bvecq->next pointer before putting the bvecq to avoid rolling up the entire list. On the other hand, rolling_buffer_clear() just needs to put the tail.

[Severity: Low]
Does this description match rolling_buffer_delete_spent()? Rather than clearing spent->next, rolling_buffer_delete_spent() clears next->prev and takes a reference on next via bvecq_get(next) before calling bvecq_put(spent), leaving spent->next intact for subrequests pinning spent.

Quoted 46 lines
fs/netfs/buffered_read.c +8 -6
index 69e68ef0652a7..c6c850ffd56ea 100644
@@ -131,6 +131,8 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq)
 	}
 
 	subreq->io_iter	= rreq->buffer.iter;
+	if (iov_iter_is_bvecq(&subreq->io_iter))
+		subreq->bvecq_pin = bvecq_get((struct bvecq *)subreq->io_iter.bvecq);
 
 	iov_iter_truncate(&subreq->io_iter, subreq->len);
 	rolling_buffer_advance(&rreq->buffer, subreq->len);
@@ -215,7 +217,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
  * otherwise we set the deprecated PG_private_2.
  */
 static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
-				     struct folio_queue **fq,
+				     struct bvecq **bq,
 				     unsigned int *offset,
 				     int *slot,
 				     size_t len,
@@ -225,21 +227,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
 		struct folio *folio;
 		size_t fsize, overlap;
 
-		if (!*fq)
+		if (!*bq)
 			break;
-		if (*slot >= folioq_count(*fq)) {
-			*fq = (*fq)->next;
+		if (!bvecq_acquire_slot(*bq, *slot)) {
+			*bq = bvecq_next(*bq);
 			*slot = 0;
 			*offset = 0;
 			continue;
 		}
 
 		/* Determine how much the subreq overlaps the folio, if at all. */
-		fsize = folioq_folio_size(*fq, *slot);
+		fsize = (*bq)->bv[*slot].bv_len;
 		overlap = min(len, fsize - *offset);
 
 		if (overlap > 0 && copy) {
-			folio = folioq_folio(*fq, *slot);
+			folio = bvec_folio(&(*bq)->bv[*slot]);

[ ... ]

fs/netfs/buffered_read.c static void netfs_read_to_pagecache(struct netfs_io_request *rreq) · line 419

Quoted 13 lines
fs/netfs/buffered_read.c +2 -2
@@ -408,10 +410,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
 		if (size <= 0)
 			netfs_all_subreqs_queued(rreq);
 
-		if (fq) {
+		if (bq) {
 			/* See if the cache indicated this should be cached. */
 			copy = test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
-			netfs_mark_copy_to_cache(rreq, &fq, &slot, &offset, slice, copy);
+			netfs_mark_copy_to_cache(rreq, &bq, &slot, &offset, slice, copy);
 		}
 
 		trace_netfs_sreq(subreq, netfs_sreq_trace_submit);

[Severity: Critical]
Can rreq->buffer.iter.bvecq and the local bq pointer in netfs_read_to_pagecache() be used after free when a read subrequest ends at a bvecq segment boundary?

During readahead, rolling_buffer_bulk_load_from_ra() pre-populates the chain of bvecq segments (bq0 -> bq1 -> ...), where rreq->buffer.tail holds a reference on bq0 and bq0->next holds a reference on bq1, while neither rreq->buffer.iter.bvecq nor bq in netfs_read_to_pagecache() holds a reference.

Unlike iov_iter_folioq_advance(), iov_iter_bvecq_advance() only advances to bq->next at the top of its loop:

lib/iov_iter.c:iov_iter_bvecq_advance() {
    ...
		if (slot >= bq->nr_slots) {
			if (!bq->next)
				break;
			bq = bq->next;
			slot = 0;
			continue;
		}

		len = bq->bv[slot].bv_len;

		if (likely(by < len))
			break;
		by -= len;
		slot++;
	} while (by);
    ...
}

When a subrequest ends at the end of the last slot in bq0, both rolling_buffer_advance() and netfs_mark_copy_to_cache() stop as soon as the remaining length reaches 0, leaving rreq->buffer.iter.bvecq and bq pointing to the exhausted bq0 at slot == bq0->nr_slots:

CPU0 (issuer) CPU1 (collector)
netfs_read_to_pagecache()
netfs_prepare_read_iterator(sreq0)

    sreq0->bvecq_pin = bvecq_get(bq0)
    rolling_buffer_advance()
      /* leaves iter.bvecq == bq0,
         bvecq_slot == bq0->nr_slots */
  netfs_mark_copy_to_cache(&bq, ...)
    /* leaves bq == bq0,
       slot == bq0->nr_slots */
  netfs_issue_read(rreq, sreq0)
  netfs_maybe_bulk_drop_ra_refs()
                                  netfs_collect_read_results()
                                    netfs_read_unlock_folios()
                                      rolling_buffer_delete_spent()
                                        bvecq_put(bq0)
                                    netfs_put_subrequest(sreq0)
                                      bvecq_put(bq0)  <-- frees bq0
  /* next subrequest sreq1 */
  netfs_prepare_read_iterator(sreq1)
    bvecq_get(iter.bvecq)         <-- use-after-free on bq0
    rolling_buffer_advance()      <-- use-after-free on bq0
  netfs_mark_copy_to_cache(&bq)   <-- use-after-free on bq0

[ ... ]

Quoted 17 lines
fs/netfs/write_collect.c +3 -3
index 6e8ea534230df..91b42820c8925 100644
@@ -130,9 +130,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		return;
 	}
 
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = rolling_buffer_delete_spent(&wreq->buffer);
-		if (!folioq)
+	while (!bvecq_acquire_slot(bq, slot)) {
+		bq = rolling_buffer_delete_spent(&wreq->buffer);
+		if (!bq)
 			return;
 		slot = 0;
 	}

[ ... ]

Quoted 23 lines
fs/netfs/write_collect.c +6 -6
@@ -166,15 +166,15 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 		wreq->cleaned_to = fpos + fsize;
 		*notes |= MADE_PROGRESS;
 
-		/* Clean up the head folioq.  If we clear an entire folioq, then
-		 * we can get rid of it provided it's not also the tail folioq
+		/* Clean up the head bq.  If we clear an entire bq, then
+		 * we can get rid of it provided it's not also the tail bq
 		 * being filled by the issuer.
 		 */
-		folioq_clear(folioq, slot);
+		bq->bv[slot].bv_page = NULL;
 		slot++;
-		if (slot >= folioq_nr_slots(folioq)) {
-			folioq = rolling_buffer_delete_spent(&wreq->buffer);
-			if (!folioq)
+		while (!bvecq_acquire_slot(bq, slot)) {
+			bq = rolling_buffer_delete_spent(&wreq->buffer);
+			if (!bq)
 				goto done;
 			slot = 0;
 		}

fs/netfs/write_collect.c

Quoted 3 lines
fs/netfs/write_collect.c +0 -0
 		if (fpos + fsize >= collected_to)
 			break;
 	}

[Severity: High]
Can this race with the producer and delete a partially-consumed bvecq segment, leaving the remaining folios in that segment permanently locked or under writeback?

Previously, rolling_buffer_delete_spent() was only called when slot >= folioq_nr_slots(folioq), which tested the fixed capacity of the segment. Here, bvecq_acquire_slot(bq, slot) tests slot < bq->nr_slots, and it runs immediately after slot++ before checking fpos + fsize >= collected_to. Because the producer updates bq->nr_slots before setting bq->next, while the collector reads bq->nr_slots before reading bq->next in rolling_buffer_delete_spent(), the collector can delete bq0 while uncollected folios remain in slots k..max_slots - 1:

CPU0 (producer) CPU1 (collector)
netfs_write_folio() netfs_writeback_unlock_folios()
/* bq0 has k slots filled */ /* unlocks slot k - 1 */

                                    slot++ /* slot == k */
                                    bvecq_acquire_slot(bq0, k)
                                      /* sees nr_slots == k, false */
  /* fills slots k..max_slots-1
     of bq0 */
  rolling_buffer_make_space()
    bvecq_append(bq0, bq1)
      bq0->next = bq1
                                    rolling_buffer_delete_spent()
                                      /* sees bq0->next == bq1,
                                         deletes bq0! */
                                    slot = 0 /* on bq1 */
                                    /* slots k..max_slots-1 of bq0
                                       are skipped forever */

Those skipped folios in bq0 never have folio_end_writeback() called, hanging writeback on the inode. The same pattern is present in netfs_pgpriv2_unlock_copied_folios() and netfs_read_unlock_folios().

Checking slot >= bq->max_slots (or checking slot >= bq->nr_slots only after verifying bvecq_next(bq) is non-NULL with an acquire load) before calling rolling_buffer_delete_spent() would avoid dropping a bvecq that still has unprocessed slots.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
author maintainer
#29 lore raw

PATCH v17 15/17 smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()

In reply to David Howells

netfslib now only presents an bvecq queue and an associated ITER_BVECQ iterator to the filesystem, so it isn't going to see the ITER_FOLIOQ iterator. So remove that code.

Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect might; further in future, it won't supply iterators at all, but rather a bvecq slice (that can be used to construct an iterator).

---
 fs/smb/smbdirect/connection.c | 68 -----------------------------------
 1 file changed, 68 deletions(-)
fs/smb/smbdirect/connection.c +0 -68
index b13bd07ffb8c..210e53495fe8 100644
@@ -6,7 +6,6 @@
 
 #include "internal.h"
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 struct smbdirect_map_sges {
 	struct ib_sge *sge;
@@ -2140,70 +2139,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,
 	return ret;
 }
 
-/*
- * Extract folio fragments from a FOLIOQ-class iterator and add them to an
- * ib_sge list.  The folios are not pinned.
- */
-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,
-					      struct smbdirect_map_sges *state,
-					      ssize_t maxsize)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	ssize_t ret = 0;
-	size_t offset = iter->iov_offset;
-
-	if (WARN_ON_ONCE(!folioq))
-		return -EIO;
-
-	if (slot >= folioq_nr_slots(folioq)) {
-		folioq = folioq->next;
-		if (WARN_ON_ONCE(!folioq))
-			return -EIO;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t fsize = folioq_folio_size(folioq, slot);
-
-		if (offset < fsize) {
-			size_t part = umin(maxsize, fsize - offset);
-			bool ok;
-
-			ok = smbdirect_map_sges_single_page(state,
-							    folio_page(folio, 0),
-							    offset,
-							    part);
-			if (!ok)
-				return -EIO;
-
-			offset += part;
-			ret += part;
-			maxsize -= part;
-		}
-
-		if (offset >= fsize) {
-			offset = 0;
-			slot++;
-			if (slot >= folioq_nr_slots(folioq)) {
-				if (!folioq->next) {
-					WARN_ON_ONCE(ret < iter->count);
-					break;
-				}
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-	} while (state->num_sge < state->max_sge && maxsize > 0);
-
-	iter->folioq = folioq;
-	iter->folioq_slot = slot;
-	iter->iov_offset = offset;
-	iter->count -= ret;
-	return ret;
-}
-
 /*
  * Extract page fragments from up to the given amount of the source iterator
  * and build up an ib_sge list that refers to all of those bits.  The ib_sge list
@@ -2234,9 +2169,6 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
 	case ITER_KVEC:
 		ret = smbdirect_map_sges_from_kvec(iter, state, len);
 		break;
-	case ITER_FOLIOQ:
-		ret = smbdirect_map_sges_from_folioq(iter, state, len);
-		break;
 	default:
 		WARN_ONCE(1, "iov_iter_type[%u]\n", iov_iter_type(iter));
 		return -EIO;
#30 lore raw

Re: [PATCH v17 15/17] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()

Sashiko has reviewed this patch and found no issues.
Quoted 9 lines

netfslib now only presents an bvecq queue and an associated ITER_BVECQ iterator to the filesystem, so it isn't going to see the ITER_FOLIOQ iterator. So remove that code.

Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect might; further in future, it won't supply iterators at all, but rather a bvecq slice (that can be used to construct an iterator).

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
author maintainer
#31 lore raw

PATCH v17 16/17 iov_iter: Remove ITER_FOLIOQ

In reply to David Howells

Remove ITER_FOLIOQ as it's no longer used.

---
 include/linux/iov_iter.h   |  65 +--------
 include/linux/uio.h        |  12 --
 lib/iov_iter.c             | 236 +-------------------------------
 lib/scatterlist.c          |  69 +---------
 lib/tests/kunit_iov_iter.c | 271 -------------------------------------
 5 files changed, 8 insertions(+), 645 deletions(-)
include/linux/iov_iter.h +2 -63
index 79825cf84e58..e88930af7f36 100644
@@ -11,7 +11,6 @@
 #include <linux/uio.h>
 #include <linux/bvec.h>
 #include <linux/bvecq.h>
-#include <linux/folio_queue.h>
 
 typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
 			     void *priv, void *priv2);
@@ -207,62 +206,6 @@ size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
 	return progress;
 }
 
-/*
- * Handle ITER_FOLIOQ.
- */
-static __always_inline
-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
-		      iov_step_f step)
-{
-	const struct folio_queue *folioq = iter->folioq;
-	unsigned int slot = iter->folioq_slot;
-	size_t progress = 0, skip = iter->iov_offset;
-
-	if (slot == folioq_nr_slots(folioq)) {
-		/* The iterator may have been extended. */
-		folioq = folioq->next;
-		slot = 0;
-	}
-
-	do {
-		struct folio *folio = folioq_folio(folioq, slot);
-		size_t part, remain = 0, consumed;
-		size_t fsize;
-		void *base;
-
-		if (!folio)
-			break;
-
-		fsize = folioq_folio_size(folioq, slot);
-		if (skip < fsize) {
-			base = kmap_local_folio(folio, skip);
-			part = umin(len, PAGE_SIZE - skip % PAGE_SIZE);
-			remain = step(base, progress, part, priv, priv2);
-			kunmap_local(base);
-			consumed = part - remain;
-			len -= consumed;
-			progress += consumed;
-			skip += consumed;
-		}
-		if (skip >= fsize) {
-			skip = 0;
-			slot++;
-			if (slot == folioq_nr_slots(folioq) && folioq->next) {
-				folioq = folioq->next;
-				slot = 0;
-			}
-		}
-		if (remain)
-			break;
-	} while (len);
-
-	iter->folioq_slot = slot;
-	iter->folioq = folioq;
-	iter->iov_offset = skip;
-	iter->count -= progress;
-	return progress;
-}
-
 /*
  * Handle ITER_XARRAY.
  */
@@ -374,8 +317,6 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	if (WARN_ON_ONCE(iov_iter_is_dmabuf_map(iter)))
@@ -412,8 +353,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
  * buffer is presented in segments, which for kernel iteration are broken up by
  * physical pages and mapped, with the mapped address being presented.
  *
- * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
- * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type
+ * iterators; it will not handle UBUF or IOVEC-type iterators.
  *
  * A step functions, @step, must be provided, one for handling mapped kernel
  * addresses and the other is given user addresses which have the potential to
@@ -442,8 +383,6 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
 		return iterate_kvec(iter, len, priv, priv2, step);
 	if (iov_iter_is_bvecq(iter))
 		return iterate_bvecq(iter, len, priv, priv2, step);
-	if (iov_iter_is_folioq(iter))
-		return iterate_folioq(iter, len, priv, priv2, step);
 	if (iov_iter_is_xarray(iter))
 		return iterate_xarray(iter, len, priv, priv2, step);
 	return iterate_discard(iter, len, priv, priv2, step);
include/linux/uio.h +0 -12
index f87e05033eac..3fc98542922e 100644
@@ -11,7 +11,6 @@
 #include <uapi/linux/uio.h>
 
 struct page;
-struct folio_queue;
 struct dma_buf_io_map;
 
 typedef unsigned int __bitwise iov_iter_extraction_t;
@@ -28,7 +27,6 @@ enum iter_type {
 	ITER_BVEC,
 	ITER_KVEC,
 	ITER_BVECQ,
-	ITER_FOLIOQ,
 	ITER_XARRAY,
 	ITER_DISCARD,
 	ITER_DMABUF_MAP,
@@ -72,7 +70,6 @@ struct iov_iter {
 				const struct kvec *kvec;
 				const struct bio_vec *bvec;
 				const struct bvecq *bvecq;
-				const struct folio_queue *folioq;
 				struct xarray *xarray;
 				void __user *ubuf;
 				struct dma_buf_io_map *dmabuf_map;
@@ -83,7 +80,6 @@ struct iov_iter {
 	union {
 		unsigned long nr_segs;
 		u16 bvecq_slot;
-		u8 folioq_slot;
 		loff_t xarray_start;
 	};
 };
@@ -156,11 +152,6 @@ static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
 	return iov_iter_type(i) == ITER_BVECQ;
 }
 
-static inline bool iov_iter_is_folioq(const struct iov_iter *i)
-{
-	return iov_iter_type(i) == ITER_FOLIOQ;
-}
-
 static inline bool iov_iter_is_xarray(const struct iov_iter *i)
 {
 	return iov_iter_type(i) == ITER_XARRAY;
@@ -314,9 +305,6 @@ void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
 void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
 			 const struct bvecq *bvecq,
 			 unsigned int first_slot, unsigned int offset, size_t count);
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-			  const struct folio_queue *folioq,
-			  unsigned int first_slot, unsigned int offset, size_t count);
 void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,
 		     loff_t start, size_t count);
 void iov_iter_dmabuf_map(struct iov_iter *i, unsigned int direction,
lib/iov_iter.c +4 -232

314 lines not on this page · raw message

lib/scatterlist.c +2 -67

93 lines not on this page · raw message

lib/tests/kunit_iov_iter.c +0 -271

310 lines not on this page · raw message

#32 lore raw

Re: [PATCH v17 16/17] iov_iter: Remove ITER_FOLIOQ

Sashiko has reviewed this patch and found no issues.
Quoted 3 lines

Remove ITER_FOLIOQ as it's no longer used.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
author maintainer
#33 lore raw

PATCH v17 17/17 netfs: Remove folio_queue

In reply to David Howells

Remove folio_queue as it's no longer used.

---
 Documentation/core-api/folio_queue.rst | 209 ------------------
 Documentation/core-api/index.rst       |   1 -
 fs/netfs/internal.h                    |   5 -
 fs/netfs/main.c                        |   7 -
 fs/netfs/misc.c                        |  96 ---------
 fs/netfs/rolling_buffer.c              |  46 ----
 fs/netfs/stats.c                       |   4 +-
 include/linux/folio_queue.h            | 282 -------------------------
 include/linux/netfs.h                  |  15 --
 include/trace/events/netfs.h           |  35 ---
 kernel/bpf/btf.c                       |   2 -
 11 files changed, 1 insertion(+), 701 deletions(-)
 delete mode 100644 Documentation/core-api/folio_queue.rst
 delete mode 100644 include/linux/folio_queue.h
Documentation/core-api/folio_queue.rst deleted +0 -209
deleted file mode 100644
index b7628896d2b6..000000000000
@@ -1,209 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0+
-
-===========
-Folio Queue
-===========
-
-:Author: David Howells <[email protected]>
-
-.. Contents:
-
- * Overview
- * Initialisation
- * Adding and removing folios
- * Querying information about a folio
- * Querying information about a folio_queue
- * Folio queue iteration
- * Folio marks
- * Lockless simultaneous production/consumption issues
-
-
-Overview
-========
-
-The folio_queue struct forms a single segment in a segmented list of folios
-that can be used to form an I/O buffer.  As such, the list can be iterated over
-using the ITER_FOLIOQ iov_iter type.
-
-The publicly accessible members of the structure are::
-
-	struct folio_queue {
-		struct folio_queue *next;
-		struct folio_queue *prev;
-		...
-	};
-
-A pair of pointers are provided, ``next`` and ``prev``, that point to the
-segments on either side of the segment being accessed.  Whilst this is a
-doubly-linked list, it is intentionally not a circular list; the outward
-sibling pointers in terminal segments should be NULL.
-
-Each segment in the list also stores:
-
- * an ordered sequence of folio pointers,
- * the size of each folio and
- * three 1-bit marks per folio,
-
-but these should not be accessed directly as the underlying data structure may
-change, but rather the access functions outlined below should be used.
-
-The facility can be made accessible by::
-
-	#include <linux/folio_queue.h>
-
-and to use the iterator::
-
-	#include <linux/uio.h>
-
-
-Initialisation
-==============
-
-A segment should be initialised by calling::
-
-	void folioq_init(struct folio_queue *folioq);
-
-with a pointer to the segment to be initialised.  Note that this will not
-necessarily initialise all the folio pointers, so care must be taken to check
-the number of folios added.
-
-
-Adding and removing folios
-==========================
-
-Folios can be set in the next unused slot in a segment struct by calling one
-of::
-
-	unsigned int folioq_append(struct folio_queue *folioq,
-				   struct folio *folio);
-
-	unsigned int folioq_append_mark(struct folio_queue *folioq,
-					struct folio *folio);
-
-Both functions update the stored folio count, store the folio and note its
-size.  The second function also sets the first mark for the folio added.  Both
-functions return the number of the slot used.  [!] Note that no attempt is made
-to check that the capacity wasn't overrun and the list will not be extended
-automatically.
-
-A folio can be excised by calling::
-
-	void folioq_clear(struct folio_queue *folioq, unsigned int slot);
-
-This clears the slot in the array and also clears all the marks for that folio,
-but doesn't change the folio count - so future accesses of that slot must check
-if the slot is occupied.
-
-
-Querying information about a folio
-==================================
-
-Information about the folio in a particular slot may be queried by the
-following function::
-
-	struct folio *folioq_folio(const struct folio_queue *folioq,
-				   unsigned int slot);
-
-If a folio has not yet been set in that slot, this may yield an undefined
-pointer.  The size of the folio in a slot may be queried with either of::
-
-	unsigned int folioq_folio_order(const struct folio_queue *folioq,
-					unsigned int slot);
-
-	size_t folioq_folio_size(const struct folio_queue *folioq,
-				 unsigned int slot);
-
-The first function returns the size as an order and the second as a number of
-bytes.
-
-
-Querying information about a folio_queue
-========================================
-
-Information may be retrieved about a particular segment with the following
-functions::
-
-	unsigned int folioq_nr_slots(const struct folio_queue *folioq);
-
-	unsigned int folioq_count(struct folio_queue *folioq);
-
-	bool folioq_full(struct folio_queue *folioq);
-
-The first function returns the maximum capacity of a segment.  It must not be
-assumed that this won't vary between segments.  The second returns the number
-of folios added to a segments and the third is a shorthand to indicate if the
-segment has been filled to capacity.
-
-Not that the count and fullness are not affected by clearing folios from the
-segment.  These are more about indicating how many slots in the array have been
-initialised, and it assumed that slots won't get reused, but rather the segment
-will get discarded as the queue is consumed.
-
-
-Folio marks
-===========
-
-Folios within a queue can also have marks assigned to them.  These marks can be
-used to note information such as if a folio needs folio_put() calling upon it.
-There are three marks available to be set for each folio.
-
-The marks can be set by::
-
-	void folioq_mark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_mark2(struct folio_queue *folioq, unsigned int slot);
-
-Cleared by::
-
-	void folioq_unmark(struct folio_queue *folioq, unsigned int slot);
-	void folioq_unmark2(struct folio_queue *folioq, unsigned int slot);
-
-And the marks can be queried by::
-
-	bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);
-	bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);
-
-The marks can be used for any purpose and are not interpreted by this API.
-
-
-Folio queue iteration
-=====================
-
-A list of segments may be iterated over using the I/O iterator facility using
-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type.  The iterator may be
-initialised with::
-
-	void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
-				  const struct folio_queue *folioq,
-				  unsigned int first_slot, unsigned int offset,
-				  size_t count);
-
-This may be told to start at a particular segment, slot and offset within a
-queue.  The iov iterator functions will follow the next pointers when advancing
-and prev pointers when reverting when needed.
-
-
-Lockless simultaneous production/consumption issues
-===================================================
-
-If properly managed, the list can be extended by the producer at the head end
-and shortened by the consumer at the tail end simultaneously without the need
-to take locks.  The ITER_FOLIOQ iterator inserts appropriate barriers to aid
-with this.
-
-Care must be taken when simultaneously producing and consuming a list.  If the
-last segment is reached and the folios it refers to are entirely consumed by
-the IOV iterators, an iov_iter struct will be left pointing to the last segment
-with a slot number equal to the capacity of that segment.  The iterator will
-try to continue on from this if there's another segment available when it is
-used again, but care must be taken lest the segment got removed and freed by
-the consumer before the iterator was advanced.
-
-It is recommended that the queue always contain at least one segment, even if
-that segment has never been filled or is entirely spent.  This prevents the
-head and tail pointers from collapsing.
-
-
-API Function Reference
-======================
-
-.. kernel-doc:: include/linux/folio_queue.h
Documentation/core-api/index.rst +0 -1
index e39edef58ab6..bfc2f1afc285 100644
@@ -40,7 +40,6 @@ Library functionality that is used throughout the kernel.
    cleanup
    assoc_array
    bvecq
-   folio_queue
    xarray
    maple_tree
    idr
fs/netfs/internal.h +0 -5
index 6e30112c9112..1f40217ae446 100644
@@ -7,7 +7,6 @@
 
 #include <linux/slab.h>
 #include <linux/seq_file.h>
-#include <linux/folio_queue.h>
 #include <linux/netfs.h>
 #include <linux/fscache.h>
 #include <linux/fscache-cache.h>
@@ -47,7 +46,6 @@ extern spinlock_t netfs_proc_lock;
 extern mempool_t netfs_request_pool;
 extern mempool_t netfs_subrequest_pool;
 extern mempool_t netfs_bvecq_pool;
-extern mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)
@@ -72,8 +70,6 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
 /*
  * misc.c
  */
-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,
-					    enum netfs_folioq_trace trace);
 void netfs_reset_iter(struct netfs_io_subrequest *subreq);
 void netfs_wake_collector(struct netfs_io_request *rreq);
 void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
@@ -206,7 +202,6 @@ extern atomic_t netfs_n_wh_retry_write_req;
 extern atomic_t netfs_n_wh_retry_write_subreq;
 extern atomic_t netfs_n_wb_lock_skip;
 extern atomic_t netfs_n_wb_lock_wait;
-extern atomic_t netfs_n_folioq;
 extern atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v);
fs/netfs/main.c +0 -7
index 3c796fe4280f..b1a7521b7210 100644
@@ -28,7 +28,6 @@ static struct kmem_cache *netfs_subrequest_slab;
 mempool_t netfs_request_pool;
 mempool_t netfs_subrequest_pool;
 mempool_t netfs_bvecq_pool;
-mempool_t netfs_folioq_pool;
 
 #ifdef CONFIG_PROC_FS
 LIST_HEAD(netfs_io_requests);
@@ -108,9 +107,6 @@ static int __init netfs_init(void)
 {
 	int ret = -ENOMEM;
 
-	if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
-		goto error_folioq_pool;
-
 	if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100, BVECQ_POOL_SIZE) < 0)
 		goto error_bvecq_pool;
 
@@ -168,8 +164,6 @@ static int __init netfs_init(void)
 error_req:
 	mempool_exit(&netfs_bvecq_pool);
 error_bvecq_pool:
-	mempool_exit(&netfs_folioq_pool);
-error_folioq_pool:
 	return ret;
 }
 fs_initcall(netfs_init);
@@ -183,6 +177,5 @@ static void __exit netfs_exit(void)
 	mempool_exit(&netfs_request_pool);
 	kmem_cache_destroy(netfs_request_slab);
 	mempool_exit(&netfs_bvecq_pool);
-	mempool_exit(&netfs_folioq_pool);
 }
 module_exit(netfs_exit);
fs/netfs/misc.c +0 -96

102 lines not on this page · raw message

fs/netfs/rolling_buffer.c +0 -46

52 lines not on this page · raw message

fs/netfs/stats.c +1 -3
index a10d34f88597..0ba6ce9295b2 100644
@@ -46,7 +46,6 @@ atomic_t netfs_n_wh_retry_write_req;
 atomic_t netfs_n_wh_retry_write_subreq;
 atomic_t netfs_n_wb_lock_skip;
 atomic_t netfs_n_wb_lock_wait;
-atomic_t netfs_n_folioq;
 atomic_t netfs_n_bvecq;
 
 int netfs_stats_show(struct seq_file *m, void *v)
@@ -89,11 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
 		   atomic_read(&netfs_n_rh_retry_read_subreq),
 		   atomic_read(&netfs_n_wh_retry_write_req),
 		   atomic_read(&netfs_n_wh_retry_write_subreq));
-	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u foq=%u wsc=%u\n",
+	seq_printf(m, "Objs   : rr=%u sr=%u bq=%u wsc=%u\n",
 		   atomic_read(&netfs_n_rh_rreq),
 		   atomic_read(&netfs_n_rh_sreq),
 		   atomic_read(&netfs_n_bvecq),
-		   atomic_read(&netfs_n_folioq),
 		   atomic_read(&netfs_n_wh_wstream_conflict));
 	seq_printf(m, "WbLock : skip=%u wait=%u\n",
 		   atomic_read(&netfs_n_wb_lock_skip),
include/linux/folio_queue.h deleted +0 -282

282 lines not on this page · raw message

include/linux/netfs.h +0 -15
index 6f3276e8eeb0..5d0015bc9db7 100644
@@ -24,7 +24,6 @@
 enum netfs_sreq_ref_trace;
 typedef struct mempool mempool_t;
 struct fscache_occupancy;
-struct folio_queue;
 
 /**
  * folio_start_private_2 - Start an fscache write on a folio.  [DEPRECATED]
@@ -250,8 +249,6 @@ struct netfs_io_request {
 #define NR_IO_STREAMS 2 //wreq->nr_io_streams
 	struct netfs_group	*group;		/* Writeback group being written back */
 	struct rolling_buffer	buffer;		/* Unencrypted buffer */
-#define NETFS_ROLLBUF_PUT_MARK		ROLLBUF_MARK_1
-#define NETFS_ROLLBUF_PAGECACHE_MARK	ROLLBUF_MARK_2
 	wait_queue_head_t	waitq;		/* Processor waiter */
 	void			*netfs_priv;	/* Private data for the netfs */
 	void			*netfs_priv2;	/* Private data for the netfs */
@@ -473,18 +470,6 @@ void netfs_end_io_write(struct inode *inode);
 int netfs_start_io_direct(struct inode *inode);
 void netfs_end_io_direct(struct inode *inode);
 
-/* Miscellaneous APIs. */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
-				       unsigned int trace /*enum netfs_folioq_trace*/);
-void netfs_folioq_free(struct folio_queue *folioq,
-		       unsigned int trace /*enum netfs_trace_folioq*/);
-
-/* Buffer wrangling helpers API. */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
-			      struct folio_queue **_buffer,
-			      size_t *_cur_size, ssize_t size, gfp_t gfp);
-void netfs_free_folioq_buffer(struct folio_queue *fq);
-
 /* Writeback exclusion API. */
 bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);
 void netfs_wb_end(struct netfs_inode *ictx);
include/trace/events/netfs.h +0 -35
index 41f7f0733bfa..e80c27c49ce2 100644
@@ -199,7 +199,6 @@
 	EM(netfs_streaming_filled_page,		"mod-streamw-f") \
 	EM(netfs_streaming_cont_filled_page,	"mod-streamw-f+") \
 	EM(netfs_folio_trace_abandon,		"abandon")	\
-	EM(netfs_folio_trace_alloc_buffer,	"alloc-buf")	\
 	EM(netfs_folio_trace_cancel_copy,	"cancel-copy")	\
 	EM(netfs_folio_trace_cancel_store,	"cancel-store")	\
 	EM(netfs_folio_trace_end_copy,		"end-copy")	\
@@ -222,7 +221,6 @@
 	EM(netfs_folio_trace_not_locked,	"!locked")	\
 	EM(netfs_folio_trace_not_under_wback,	"!wback")	\
 	EM(netfs_folio_trace_pgpriv2_copy,	"pgpriv2-copy")	\
-	EM(netfs_folio_trace_put,		"put")		\
 	EM(netfs_folio_trace_read,		"read")		\
 	EM(netfs_folio_trace_read_done,		"read-done")	\
 	EM(netfs_folio_trace_read_gaps,		"read-gaps")	\
@@ -244,14 +242,6 @@
 	EM(netfs_trace_donate_to_next,		"to-next")	\
 	E_(netfs_trace_donate_to_deferred_next,	"defer-next")
 
-#define netfs_folioq_traces					\
-	EM(netfs_trace_folioq_alloc_buffer,	"alloc-buf")	\
-	EM(netfs_trace_folioq_clear,		"clear")	\
-	EM(netfs_trace_folioq_delete,		"delete")	\
-	EM(netfs_trace_folioq_make_space,	"make-space")	\
-	EM(netfs_trace_folioq_rollbuf_init,	"roll-init")	\
-	E_(netfs_trace_folioq_read_progress,	"r-progress")
-
 #ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
 #define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
 
@@ -270,7 +260,6 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
 enum netfs_folio_trace { netfs_folio_traces } __mode(byte);
 enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);
 enum netfs_donate_trace { netfs_donate_traces } __mode(byte);
-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);
 
 #endif
 
@@ -294,7 +283,6 @@ netfs_sreq_ref_traces;
 netfs_folio_traces;
 netfs_collect_contig_traces;
 netfs_donate_traces;
-netfs_folioq_traces;
 
 /*
  * Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -773,29 +761,6 @@ TRACE_EVENT(netfs_collect_stream,
 		      __entry->collected_to, __entry->issued_to)
 	    );
 
-TRACE_EVENT(netfs_folioq,
-	    TP_PROTO(const struct folio_queue *fq,
-		     enum netfs_folioq_trace trace),
-
-	    TP_ARGS(fq, trace),
-
-	    TP_STRUCT__entry(
-		    __field(unsigned int,		rreq)
-		    __field(unsigned int,		id)
-		    __field(enum netfs_folioq_trace,	trace)
-			     ),
-
-	    TP_fast_assign(
-		    __entry->rreq	= fq ? fq->rreq_id : 0;
-		    __entry->id		= fq ? fq->debug_id : 0;
-		    __entry->trace	= trace;
-			   ),
-
-	    TP_printk("R=%08x fq=%x %s",
-		      __entry->rreq, __entry->id,
-		      __print_symbolic(__entry->trace, netfs_folioq_traces))
-	    );
-
 TRACE_EVENT(netfs_read_progress_at,
 	    TP_PROTO(const struct netfs_io_request *rreq),
 
kernel/bpf/btf.c +0 -2
index 1be30487a2fa..0fca0c3132fe 100644
@@ -7344,8 +7344,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
 	/* amdgpu */
 	{ "amdgpu_vm_bo_map", 0x1 },
 	{ "amdgpu_vm_bo_unmap", 0x1 },
-	/* netfs */
-	{ "netfs_folioq", 0x1 },
 	/* xfs from xfs_defer_pending_class */
 	{ "xfs_defer_create_intent", 0x1 },
 	{ "xfs_defer_cancel_list", 0x1 },
#34 lore raw

Re: [PATCH v17 17/17] netfs: Remove folio_queue

Sashiko has reviewed this patch and found no issues.
Quoted 3 lines

Remove folio_queue as it's no longer used.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=17