netfs, iov_iter: Use a chain of bio_vec arrays instead of folio_queue
Subsystems (15)
David Howells
maintainerv19 cover netfs, iov_iter: Use a chain of bio_vec arrays instead of folio_queue
Hi Christian,
Could you pull these patches please into your vfs-7.5.netfs branch? This is the third of five batches (two already having gone upstream), in this case replacing the folio_queue struct with a bvecq struct. It has been rebased on v7.4-rc3. This was split from v11 of a larger netfslib series[1].
This series adds a new type, struct bvecq. This points to a bounded array of bio_vecs with information about how to clean up the memory they point to. A number of bvecq structs can be linked together to form a chain and an iterator type, ITER_BVECQ, is provided that can be pointed at the chain and can walk it. Netfslib is then switched to use the new type and folio_queue is removed along with ITER_FOLIOQ.
This is intended as a more flexible replacement for struct folio_queue as the latter can only point to whole folios and cannot point to partial pages that might be extracted from an ITER_IOVEC/ITER_UBUF for direct or unbuffered I/O purposes.
These patches make a more or less straight swap between struct folio_queue and struct bvecq. The change is not quite trivial as the folio_queue struct contains a folio_batch struct and is basically a list of whole folios only, whereas the new bvecq struct is a chain of bio_vec arrays.
The patches also add a pair of gated memory pools for allocating bvecqs in writeback under low memory conditions. One is for use by requests and the other by subrequests. The two pools have different lifetime and typical allocation amount expectations and avoid deadlocking between the two levels. To avoid overrunning pool capacity, access is controlled by semaphore for each pool.
A further patchset will switch the netfslib unbuffered I/O code to using bvecqs and then the bvecqs will be passed down into the filesystem rather than passing an iterator.
The primary reason behind making this change is so that bvecq chains can be used in the assembly of network filesystem messages. Netfslib can furnish the filesystem with bvecq chains representing the data buffers that need to be read or written and then, for example, for a write RPC, the filesystem can add protocol headers and trailers onto the chain without the need to copy it and can also glue multiple buffers together to form compound operations and/or perform sparse operations.
One option here is to add two more fields to the bvecq struct, one to increase the offset of the first bio_vec and one to decrease the length of the last, thereby allowing a bvecq struct to point at a slice of another bio_vec array without having to copy the array - but at the cost of adding one or two extra conditional ops when getting the offset or length of a segment.
I have in-progress patch sets to rewrite the cifs transport[2] and the ceph and rbd transport[3] to make use of bvecq chains. For the cifs transport, the idea is to extend the SMB message concept further up the stack, where the PDU-marshalling routines will create individual RPC request blobs that the transport can then automatically chain together when it is assembling a compound SMB request for transmission.
With the ceph transport, the idea is to convert all the different data containers it has into just passing around bvecqs. These can then be chained together in order to transmit them.
This improves efficiency in the TCP stack as we no longer need to cork the TCP socket, call sendmsg() multiple times and then uncork; rather we can preassemble the message in a bvecq chain and just send the entire message in one shot with a single sendmsg() and reduce the number of places doing loops. Using a single sendmsg got an improvement in nfsd performance[4].
I also have some changes on the TCP receive side for the cifs transport (which is also likely applicable to the ceph TCP transport) whereby the receive buffers are 'spliced' out into a bvecq in the cifs I/O thread rather than being copied. Using a bvecq chain here is advantageous as we don't know in advance how many segments we're going to have. This allows us firstly to avoid copying data with the socket lock held (thus holding up sendmsg) and secondly to avoid copying data in the I/O thread (copying can be offloaded to the app thread). The last time I benchmarked this, it appeared to get fio reading tests on cifs a 5% speedup.
Unfortunately, this doesn't help AFS much as that uses a UDP transport, but it might also help 9P, at least with its TCP transport.
The patches can also be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next-3
Thanks,
David
Changes
=======
ver #19)
- Fixed issues found by Sashiko[10] and by running Gemini locally:
- Removed some unnecessary braces.
- Changed some missed "folio queue" to "bvecq".
- In netfs_read_to_pagecache(), pin the main bvecq around the loop so that netfs_mark_copy_to_cache() can make use of it.
- In netfs_mark_copy_to_cache(), switch the slot and offset args to match the caller (and rename the mark tracking vars in the caller as mark_*).
- Added comments on the emergency pool unlocking to indicate that it's fine to do this before releasing the objects back to the pool.
- Fixed writeback subreq accelerated issue under OOM to issue when the head bvecq is full rather than any time there's more than one bvecq in the chain to avoid issuing a subreq for every single folio.
- In netfs_writeback_unlock_folios(), fix producer/consumer race as the break directive only exits one loop, not two.
- Don't init the rolling buffer for single-type or unbuffered/DIO writes.
- In afs_symlink_writepages(), clear vnode->directory{,_size} whilst
holding the vnode->cb_lock to group them matching the read earlier in
that function. - In bvecq_alloc_buffer2(), trim empty tail off a chain if OOM occurs.
- In iov_iter_bvecq_advance(), remove check of '!by' so that zero-length slots will be immediately advanced over.
- In a number of iterator functions, make sure the loop advances beyond a spent bvecq before returning.
- Fixed miscellaneous typos.
ver #18)
- Rebase on v7.4-rc3.
- Fixed issues found by Sashiko[9]:
- Fixed the bvecq mempool handling:
- Changed the bvecq_alloc_*() functions to take a set of updateable flags instead of a true/false writeback indicator. This allows the caller to be notified that an OOM occurred and to keep persistent state.
- The allocation funcs set BVECQ_HIT_OOM if an out of memory condition occurred, even if it is dealt with through a mempool.
- The alloc funcs set BVECQ_HOLD_POOL_SEM if they return holding a count on the semaphore.
- Added a function to clean up the state (including releasing the semaphore).
- Split the bvecq mempool so that there's one pool for request-level and one for subrequest-level. These allocations have different characteristics and lifetimes and a process might deadlock against itself if using a single pool for both.
- The req-level pool allows up to 50 procs to share a pool of 100.
- The subreq-level pool allows 2 procs to have enough bvecqs each on average to track 16MiB worth of pages.
- Gate the pools with semaphores to control access to the pools, thereby making sure that at least some reqs/subreqs can make progress by having sufficient available. The semaphore count must be significantly less than the capacity of the pool.
- Added a flag to allow the caller to say that they are fine with a buffer or chain allocation being shortened if OOM occurs.
- Made cifs use the subreq pool for its encryption buffer. This really needs to be restricted to just writeback and reconnect during writeback, but that's not easy without restructuring the cifs transport (I have patches for that). Further, we really want to shorten the subrequest - but cifs can't do that at the point it currently allocates its enc buffer.
- Fixed kunit_bvecq release on abort as kunit_add_action_or_reset() just returns an error on OOM rather than aborting.
- Fixed the 'can I revert?' bounds checking in kunit_bvecq.
- Made afs dir and symlink writeback check the inode validity after taking the callback lock (cb_lock).
- Fixed issues found by local review with claude-opus-5-5:
- Fixed some issues in the above changes for sashiko.
- Made the bvecq chain advance functions advance to the next occupied slot in the next occupied bvecq before returning rather than leaving the iterator on a non-terminal spent bvecq (where "occupied" means a slot numbered lower than nr_slots; it could have bv_len==0).
- Fixed the bvecq test to abort rather than fail and continue.
- Increased the subreq-level bvecq pool to handle 16MiB-worth of pages to support CIFS_MAX_WSIZE.
- Recheck ->nr_slots and ->next before deleting spent bvecq.
ver #17)
- Fixed issues found by Sashiko[8]:
- Fix bvecq_alloc_buffer2() to set mem_type if zero slots requested.
- Fix folio leak in the kunit tests.
- Don't mark kunit functions as __init.
- Make subreqs pin the bvecq chain to avoid retry of a partially complete subreq seeing released memory.
- Fix handling of concurrent writeback of a single-type object and afs vnode status update.
- Fixed issues found by local review with claude-opus-5-5:
- Pass len from netfs_create_write_req() to netfs_alloc_request().
- Added a patch to pass the request length in for unbuffered/DIO writes
and single-type object writeback to request creation, to set wreq->len.
- In netfs_writeback_single() set wreq->len to the specified length, not padded buffer size.
- Added a fix patch to remove __init and __refdata marks from kunit_iov_iter.c.
- Removed __refdata from kunit_bvecq.c too.
- Extended the bvecq chain pinning to the retry functions too.
ver #16)
- Rebase on v7.4-rc1.
- Add docs for the bvecq tools.
- Add some kunit tests for the bvecq iterator.
- Fixed issues found by local review with claude-opus-5-5:
- Added missing linux/mempool.h inclusion.
- Fix bvecq_alloc_one() to check the rounded-up allocation size against the pool allocation size, not the struct+array size which may have a gap.
- Fix bvecq_alloc_one() to limit the max slots to USHRT_MAX to not overrun the slot counters.
- Make bvec_kmap_partial()'s bvec pointer const.
- Added an iter_to_sg kunit test for bvecq.
- Added a patch to make afs symlink's use of ->directory_size consistent.
- In netfs_writeback_single(), fix the value ->submit_extendable_to is set to (though a future patch series will remove this).
- Split the afs dir bvecq patch into three:
- A patch to pass the content length to netfs_writeback_single(), separately from the buffer size in the iov_iter. The filesystem is then responsible for providing zeroed padding.
- A check is added to make sure the buffer is not smaller than the content.
- The API doc is updated.
- A patch to simplify netfs_writeback_single():
- No looking inside the iov_iter.
- Effectively do DIO rather than buffered I/O (pagecache not used).
- Pass a single op to the cache.
- Wait synchronously for op completion for now as the calling filesystem should otherwise do a wait.
- Removed the bits to set up a write to the server since it should only write to the cache.
- A patch to switch afs dir (and symlink) buffering to use bvecq instead of folio_queue.
- Clear new pages as added and clear the tail if shrunk by a third party for AFS directories.
- Clear the buffer tail before calling netfs_writeback_single() for AFS symlinks.
- Fix netfs_limit_bvecq() to handle empty bvecqs and short chains.
- Added missing SPDX to new doc.
- Added new files to MAINTAINERS.
ver #15)
- Fix issues from Sashiko[7]:
- Don't need to set page counts after calling alloc_pages_bulk().
- Adjust netfs_writeback_single() to return 0 on success, not the amount written.
- Remove a couple of unused constants.
ver #14)
- Fix issues from Sashiko[6]:
- In iter_bvecq_get_pages(), use folio_get() not get_pages.
- In netfs_read_set_unlock_at(), netfs_read_unlock_folios(), netfs_pgpriv2_unlock_copied_folios() and netfs_writeback_unlock_folios(), skip wholly unoccupied bvecqs.
- Remove some now-unused trace bits.
ver #13)
- Fix issues from Sashiko[5]:
- In iter_bvecq_get_pages(), need to add bv_offset when calling
want_pages_array().
- In iter_bvecq_get_pages(), make the getting of a ref on the page dependent on the page being non-slab.
- In bvecq_alloc_one(), use mempool_alloc(), not mempool_alloc_noreserve().
- In bvecq_expand_buffer(), don't add __GFP_NORETRY|__GFP_WARN for order 0 allocations.
- Drop the netfs_bv_slot tracepoint as it's not used in these patches.
- In afs_do_read_single(), afs_dir_get_block() and afs_do_read_symlink(), use mapping_gfp_mask() rather than GFP_KERNEL.
- In netfs_writeback_single(), set ->submitted so that netfs_wait_for_write() doesn't then just return -EIO.
- In netfs_unlock_abandoned_read_pages(), reset ->first_tail_slot when moving on to next node.
- Remove netfs_folioq_trace(s) trace info enum and string set as their tracepoint got removed.
ver #12)
- Split from v11 of "netfs: Keep track of folios in a segmented bio_vec[] chain"[1]
[1] https://lore.kernel.org/r/[email protected]/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=cifs-experimental
[3] https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=ceph-iter
[4] https://lore.kernel.org/netdev/168979108540.1905271.9720708849149797793.stgit@morisot.1015granger.net/
[5] https://sashiko.dev/#/patchset/20260929075913.2740968-1-dhowells%40redhat.com
[6] https://sashiko.dev/#/patchset/20260929103412.2807494-1-dhowells%40redhat.com
[7] https://sashiko.dev/#/patchset/20260930125247.3232130-1-dhowells%40redhat.com
[8] https://sashiko.dev/#/patchset/20261107160823.1465553-1-dhowells%40redhat.com
[9] https://sashiko.dev/#/patchset/20261111203034.2126314-1-dhowells%40redhat.com
[10] https://sashiko.dev/#/patchset/20261117171958.2982801-1-dhowells%40redhat.com
David Howells (17):
iov_iter: Fix kunit test to remove __init and __refdata marks
block: Add a function to kmap one page of a multipage bio_vec
iov_iter: Add a segmented queue of bio_vec[]
netfs: Add some tools for managing bvecq chains
iov_iter: Add an API document for the bvecq structure
iov_iter: Add a bvecq kunit test
afs: Fix an inconsistency in the symlink code
netfs: Explicitly pass tail padding to netfs_writeback_single()
netfs: Change netfs_writeback_single() to not look inside the iterator
netfs: Pass the write length to request creation where appropriate
afs: Use a bvecq to hold dir content rather than folioq
cifs: Use a bvecq for buffering instead of a folioq
smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
netfs: Switch folioq to bvecq
smbdirect: Remove support for ITER_FOLIOQ from
smbdirect_map_sges_from_iter()
iov_iter: Remove ITER_FOLIOQ
netfs: Remove folio_queue
Documentation/core-api/bvecq.rst | 259 +++++++++ Documentation/core-api/folio_queue.rst | 209 ------- Documentation/core-api/index.rst | 2 +- Documentation/filesystems/netfs_library.rst | 47 +- MAINTAINERS | 3 + fs/afs/dir.c | 60 +- fs/afs/dir_edit.c | 44 +- fs/afs/dir_search.c | 35 +- fs/afs/inode.c | 2 +- fs/afs/internal.h | 6 +- fs/afs/symlink.c | 69 ++- fs/netfs/Makefile | 1 + fs/netfs/buffered_read.c | 48 +- fs/netfs/bvecq.c | 431 ++++++++++++++ fs/netfs/direct_write.c | 4 +- fs/netfs/internal.h | 12 +- fs/netfs/iterator.c | 42 +- fs/netfs/main.c | 50 +- fs/netfs/misc.c | 96 ---- fs/netfs/objects.c | 6 +- fs/netfs/read_collect.c | 48 +- fs/netfs/read_pgpriv2.c | 45 +- fs/netfs/read_retry.c | 53 +- fs/netfs/rolling_buffer.c | 198 +++---- fs/netfs/stats.c | 8 +- fs/netfs/write_collect.c | 44 +- fs/netfs/write_issue.c | 228 +++----- fs/netfs/write_retry.c | 15 +- fs/smb/client/cifsglob.h | 2 +- fs/smb/client/smb2ops.c | 102 ++-- fs/smb/smbdirect/connection.c | 138 ++--- include/linux/bvec.h | 18 + include/linux/bvecq.h | 197 +++++++ include/linux/folio_queue.h | 282 --------- include/linux/iov_iter.h | 94 +-- include/linux/netfs.h | 19 +- include/linux/rolling_buffer.h | 42 +- include/linux/uio.h | 17 +- include/trace/events/netfs.h | 37 +- kernel/bpf/btf.c | 2 - lib/Kconfig.debug | 13 + lib/iov_iter.c | 510 ++++++++++++----- lib/scatterlist.c | 72 +-- lib/tests/Makefile | 1 + lib/tests/kunit_bvecq.c | 603 ++++++++++++++++++++ lib/tests/kunit_iov_iter.c | 208 +++---- 46 files changed, 2782 insertions(+), 1640 deletions(-) create mode 100644 Documentation/core-api/bvecq.rst delete mode 100644 Documentation/core-api/folio_queue.rst create mode 100644 fs/netfs/bvecq.c create mode 100644 include/linux/bvecq.h delete mode 100644 include/linux/folio_queue.h create mode 100644 lib/tests/kunit_bvecq.c
David Howells
maintainerPATCH v19 1/17 iov_iter: Fix kunit test to remove __init and __refdata marks
Remove the __init marks from the functions in the iov_iter kunit test so that they don't get removed after the module is loaded or if the test is built in as the tests can be triggered through debugfs too.
Also remove the __refdata mark which is unnecessary if the table so marked doesn't point at any __init funcs.
This was found by claude-opus-5-5 run locally.
--- lib/tests/kunit_iov_iter.c | 96 +++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 49 deletions(-)
lib/tests/kunit_iov_iter.c +47 -49
@@ -48,9 +48,9 @@ static void iov_kunit_unmap(void *data)vfree (data); } -static void *__init iov_kunit_create_buffer (struct kunit *test, -struct page ***ppages, -size_t npages) +static void *iov_kunit_create_buffer (struct kunit *test, +struct page ***ppages, +size_t npages) {struct page **pages;unsigned long got, last;
@@ -90,11 +90,11 @@ static void *__init iov_kunit_create_buffer(struct kunit *test,return buffer; } -static void __init iov_kunit_load_kvec (struct kunit *test, -struct iov_iter *iter,int dir, -struct kvec *kvec,unsigned int kvmax, -void *buffer,size_t bufsize, -const struct kvec_test_range *pr) +static void iov_kunit_load_kvec (struct kunit *test, +struct iov_iter *iter,int dir, +struct kvec *kvec,unsigned int kvmax, +void *buffer,size_t bufsize, +const struct kvec_test_range *pr) {size_t size =0 ;int i;
@@ -116,7 +116,7 @@ static void __init iov_kunit_load_kvec(struct kunit *test,/* * Test copying to a ITER_KVEC-type iterator. */ -static void __init iov_kunit_copy_to_kvec (struct kunit *test) +static void iov_kunit_copy_to_kvec (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -166,7 +166,7 @@ static void __init iov_kunit_copy_to_kvec(struct kunit *test)/* * Test copying from a ITER_KVEC-type iterator. */ -static void __init iov_kunit_copy_from_kvec (struct kunit *test) +static void iov_kunit_copy_from_kvec (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -234,12 +234,12 @@ static const struct bvec_test_range bvec_test_ranges[] = { { -1, -1, -1 } }; -static void __init iov_kunit_load_bvec (struct kunit *test, -struct iov_iter *iter,int dir, -struct bio_vec *bvec,unsigned int bvmax, -struct page **pages,size_t npages, -size_t bufsize, -const struct bvec_test_range *pr) +static void iov_kunit_load_bvec (struct kunit *test, +struct iov_iter *iter,int dir, +struct bio_vec *bvec,unsigned int bvmax, +struct page **pages,size_t npages, +size_t bufsize, +const struct bvec_test_range *pr) {struct page *can_merge =NULL , *page;size_t size =0 ;
@@ -275,7 +275,7 @@ static void __init iov_kunit_load_bvec(struct kunit *test,/* * Test copying to a ITER_BVEC-type iterator. */ -static void __init iov_kunit_copy_to_bvec (struct kunit *test) +static void iov_kunit_copy_to_bvec (struct kunit *test) {const struct bvec_test_range *pr;struct iov_iter iter;
@@ -328,7 +328,7 @@ static void __init iov_kunit_copy_to_bvec(struct kunit *test)/* * Test copying from a ITER_BVEC-type iterator. */ -static void __init iov_kunit_copy_from_bvec (struct kunit *test) +static void iov_kunit_copy_from_bvec (struct kunit *test) {const struct bvec_test_range *pr;struct iov_iter iter;
@@ -392,10 +392,10 @@ static void iov_kunit_destroy_folioq(void *data) } } -static void __init iov_kunit_load_folioq (struct kunit *test, -struct iov_iter *iter,int dir, -struct folio_queue *folioq, -struct page **pages,size_t npages) +static void iov_kunit_load_folioq (struct kunit *test, +struct iov_iter *iter,int dir, +struct folio_queue *folioq, +struct page **pages,size_t npages) {struct folio_queue *p = folioq;size_t size =0 ;
@@ -429,7 +429,7 @@ static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)/* * Test copying to a ITER_FOLIOQ-type iterator. */ -static void __init iov_kunit_copy_to_folioq (struct kunit *test) +static void iov_kunit_copy_to_folioq (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -491,7 +491,7 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)/* * Test copying from a ITER_FOLIOQ-type iterator. */ -static void __init iov_kunit_copy_from_folioq (struct kunit *test) +static void iov_kunit_copy_from_folioq (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -560,10 +560,10 @@ static void iov_kunit_destroy_xarray(void *data)kfree (xarray); } -static void __init iov_kunit_load_xarray (struct kunit *test, -struct iov_iter *iter,int dir, -struct xarray *xarray, -struct page **pages,size_t npages) +static void iov_kunit_load_xarray (struct kunit *test, +struct iov_iter *iter,int dir, +struct xarray *xarray, +struct page **pages,size_t npages) {size_t size =0 ;int i;
@@ -591,7 +591,7 @@ static struct xarray *iov_kunit_create_xarray(struct kunit *test)/* * Test copying to a ITER_XARRAY-type iterator. */ -static void __init iov_kunit_copy_to_xarray (struct kunit *test) +static void iov_kunit_copy_to_xarray (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -649,7 +649,7 @@ static void __init iov_kunit_copy_to_xarray(struct kunit *test)/* * Test copying from a ITER_XARRAY-type iterator. */ -static void __init iov_kunit_copy_from_xarray (struct kunit *test) +static void iov_kunit_copy_from_xarray (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -712,7 +712,7 @@ static void __init iov_kunit_copy_from_xarray(struct kunit *test)/* * Test the extraction of ITER_KVEC-type iterators. */ -static void __init iov_kunit_extract_pages_kvec (struct kunit *test) +static void iov_kunit_extract_pages_kvec (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -792,7 +792,7 @@ static void __init iov_kunit_extract_pages_kvec(struct kunit *test)/* * Test the extraction of ITER_BVEC-type iterators. */ -static void __init iov_kunit_extract_pages_bvec (struct kunit *test) +static void iov_kunit_extract_pages_bvec (struct kunit *test) {const struct bvec_test_range *pr;struct iov_iter iter;
@@ -870,7 +870,7 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)/* * Test the extraction of ITER_FOLIOQ-type iterators. */ -static void __init iov_kunit_extract_pages_folioq (struct kunit *test) +static void iov_kunit_extract_pages_folioq (struct kunit *test) {const struct kvec_test_range *pr;struct folio_queue *folioq;
@@ -949,7 +949,7 @@ static void __init iov_kunit_extract_pages_folioq(struct kunit *test)/* * Test the extraction of ITER_XARRAY-type iterators. */ -static void __init iov_kunit_extract_pages_xarray (struct kunit *test) +static void iov_kunit_extract_pages_xarray (struct kunit *test) {const struct kvec_test_range *pr;struct iov_iter iter;
@@ -1033,8 +1033,7 @@ struct iov_kunit_iter_to_sg_data {size_t npages; }; -static void __init -iov_kunit_iter_unpin_sgt (void *data) +static void iov_kunit_iter_unpin_sgt (void *data) {struct sg_table *sgt = data;
@@ -1042,9 +1041,9 @@ iov_kunit_iter_unpin_sgt(void *data)unpin_user_page (sg_page (&sgt->sgl[i])); } -static void __init -iov_kunit_iter_to_sg_init (struct kunit *test,size_t bufsize,bool user, -struct iov_kunit_iter_to_sg_data *data) +static void iov_kunit_iter_to_sg_init (struct kunit *test,size_t bufsize, +bool user, +struct iov_kunit_iter_to_sg_data *data) {struct page **spages;struct scatterlist *sg;
@@ -1080,10 +1079,9 @@ iov_kunit_iter_to_sg_init(struct kunit *test, size_t bufsize, bool user,memset (data->scratch,0 , bufsize); } -static void __init -iov_kunit_iter_to_sg_check (struct kunit *test,struct iov_iter *iter, -size_t bufsize, -struct iov_kunit_iter_to_sg_data *data) +static void iov_kunit_iter_to_sg_check (struct kunit *test,struct iov_iter *iter, +size_t bufsize, +struct iov_kunit_iter_to_sg_data *data) {static const size_t tail =16 *PAGE_SIZE ;size_t i;
@@ -1128,7 +1126,7 @@ iov_kunit_iter_to_sg_check(struct kunit *test, struct iov_iter *iter,KUNIT_EXPECT_EQ (test, i, bufsize); } -static void __init iov_kunit_iter_to_sg_kvec (struct kunit *test) +static void iov_kunit_iter_to_sg_kvec (struct kunit *test) {struct iov_kunit_iter_to_sg_data data;struct iov_iter iter;
@@ -1145,7 +1143,7 @@ static void __init iov_kunit_iter_to_sg_kvec(struct kunit *test)iov_kunit_iter_to_sg_check (test, &iter, bufsize, &data); } -static void __init iov_kunit_iter_to_sg_bvec (struct kunit *test) +static void iov_kunit_iter_to_sg_bvec (struct kunit *test) {struct iov_kunit_iter_to_sg_data data;struct page *p, *can_merge =NULL ;
@@ -1173,7 +1171,7 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)iov_kunit_iter_to_sg_check (test, &iter, bufsize, &data); } -static void __init iov_kunit_iter_to_sg_folioq (struct kunit *test) +static void iov_kunit_iter_to_sg_folioq (struct kunit *test) {struct iov_kunit_iter_to_sg_data data;struct folio_queue *folioq;
@@ -1190,7 +1188,7 @@ static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)iov_kunit_iter_to_sg_check (test, &iter, bufsize, &data); } -static void __init iov_kunit_iter_to_sg_xarray (struct kunit *test) +static void iov_kunit_iter_to_sg_xarray (struct kunit *test) {struct iov_kunit_iter_to_sg_data data;struct xarray *xarray;
@@ -1207,7 +1205,7 @@ static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)iov_kunit_iter_to_sg_check (test, &iter, bufsize, &data); } -static void __init iov_kunit_iter_to_sg_ubuf (struct kunit *test) +static void iov_kunit_iter_to_sg_ubuf (struct kunit *test) {struct iov_kunit_iter_to_sg_data data;struct iov_iter iter;
@@ -1221,7 +1219,7 @@ static void __init iov_kunit_iter_to_sg_ubuf(struct kunit *test)iov_kunit_iter_to_sg_check (test, &iter, bufsize, &data); } -static struct kunit_case __refdata iov_kunit_cases[] = { +static struct kunit_case iov_kunit_cases[] = {KUNIT_CASE (iov_kunit_copy_to_kvec),KUNIT_CASE (iov_kunit_copy_from_kvec),KUNIT_CASE (iov_kunit_copy_to_bvec),
[email protected]
AI reviewRe: [PATCH v19 01/17] iov_iter: Fix kunit test to remove __init and __refdata marks
Sashiko has reviewed this patch and found no issues.
Quoted 11 lines
Remove the
__initmarks from the functions in theiov_iterkunit test so that they don't get removed after the module is loaded or if the test is built in as the tests can be triggered through debugfs too.Also remove the
__refdatamark which is unnecessary if the table so marked doesn't point at any__initfuncs.This was found by claude-opus-5-5 run locally.
Fixes 2d71340ff1d4 ("iov_iter: Kunit tests for copying to/from an iterator")Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH v19 01/17] iov_iter: Fix kunit test to remove __init and __refdata marks
In reply to David HowellsHi David,
Quoted 3 lines
Remove the
__initmarks from the functions in theiov_iterkunit test so that they don't get removed after the module is loaded or if the test is built in as the tests can be triggered through debugfs too.
The reasoning looks right to me. The suite is registered with kunit_test_suites(), so it gets a debugfs "run" file and can be re-run after the init sections have been freed; calling into an __init function from there is a use-after-free, and __refdata on the case array keeps modpost from flagging the reference to init text.
There is one more file in the tree with the same problem:
lib/tests/bitfield_kunit.c:60 __init test_bitfields_constants() lib/tests/bitfield_kunit.c:103 __init test_bitfields_variables() lib/tests/bitfield_kunit.c:140 __refdata bitfields_test_cases[] lib/tests/bitfield_kunit.c:151 kunit_test_suites(&bitfields_test_suite)
(line numbers from v7.4-rc1; test_bitfields_compile() is also marked __init but is only built under TEST_BITFIELD_COMPILE and is not in the case array)
I went through the other suites that still have __init case functions -- drivers/char/misc_minor_kunit.c, init/initramfs_test.c, security/landlock/id.c and lib/kunit/kunit-example-test.c -- and they are fine: they are all registered with kunit_test_init_section_suites(), which does not create a "run" file, so keeping __init there is correct.
Do you want the bitfields one folded into this series, or would you rather I send it separately?
Thanks,
Wang Yibo
David Howells
maintainerPATCH v19 2/17 block: Add a function to kmap one page of a multipage bio_vec
In reply to David HowellsAdd a function to kmap one page of a multipage bio_vec by offset (which is added to the offset in the bio_vec internally). The caller is responsible for calculating how much of the page is then available.
--- include/linux/bvec.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
include/linux/bvec.h +18 -0
@@ -344,4 +344,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec) return page_to_phys(bvec->bv_page) + bvec->bv_offset; } +/** +* bvec_kmap_partial - Map part of a bvec into the kernel virtual address space +* @bvec: bvec to map +* @offset: Offset into bvec +* +* Map the page containing the byte at @offset into the kernel virtual address +* space. The caller is responsible for making sure this doesn't overrun. +* +* Call kunmap_local on the returned address to unmap. +*/ +static inline void *bvec_kmap_partial (const struct bio_vec *bvec,size_t offset) +{ + offset += bvec->bv_offset; + +return kmap_local_page (bvec->bv_page + (offset >>PAGE_SHIFT )) + + (offset & ~PAGE_MASK ); +} +#endif /* __LINUX_BVEC_H */
[email protected]
AI reviewRe: [PATCH v19 02/17] block: Add a function to kmap one page of a multipage bio_vec
Sashiko has reviewed this patch and found no issues.
Quoted 5 lines
Add a function to kmap one page of a multipage
bio_vecby offset (which is added to the offset in thebio_vecinternally). The caller is responsible for calculating how much of the page is then available.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
David Howells
maintainerPATCH v19 3/17 iov_iter: Add a segmented queue of bio_vec[]
In reply to David HowellsAdd the concept of a segmented queue of bio_vec[] arrays. This allows an indefinite quantity of elements to be handled and allows things like network filesystems and crypto drivers to glue bits on the ends without having to reallocate the array.
The bvecq struct that defines each segment also carries capacity/usage information along with flags indicating whether the constituent memory regions need freeing or unpinning. The bvecq structs are refcounted to allow a queue to be extracted in batches and split between a number of subrequests.
The bvecq can have the bio_vec[] it manages allocated in with it, but this is not required. A flag is provided for if this is the case as comparing ->bv to ->__bv is not sufficient to detect this case.
Add an iterator type ITER_BVECQ for it. This is intended to replace ITER_FOLIOQ (and ITER_XARRAY).
Note that the prev pointer is only really needed for iov_iter_revert() and could be dispensed with if struct iov_iter contained the head information as well as the current point.
--- MAINTAINERS | 1 + include/linux/bvecq.h | 50 +++++ include/linux/iov_iter.h | 77 ++++++- include/linux/uio.h | 11 + lib/iov_iter.c | 443 ++++++++++++++++++++++++++++++++++++- lib/scatterlist.c | 68 +++++- lib/tests/kunit_iov_iter.c | 277 +++++++++++++++++++++++ 7 files changed, 918 insertions(+), 9 deletions(-) create mode 100644 include/linux/bvecq.h
MAINTAINERS +1 -0
@@ -10389,6 +10389,7 @@ S: Supported F: Documentation/filesystems/caching/ F: Documentation/filesystems/netfs_library.rst F: fs/netfs/ +F: include/linux/bvecq.h F: include/linux/fscache*.h F: include/linux/netfs.h F: include/trace/events/fscache.h
include/linux/bvecq.h new file +50 -0
50 lines not on this page · raw message
include/linux/iov_iter.h +75 -2
107 lines not on this page · raw message
include/linux/uio.h +11 -0
@@ -27,6 +27,7 @@ enum iter_type {ITER_IOVEC ,ITER_BVEC ,ITER_KVEC , +ITER_BVECQ ,ITER_FOLIOQ ,ITER_XARRAY , ITER_DISCARD,
@@ -70,6 +71,7 @@ struct iov_iter {const struct iovec *__iov;const struct kvec *kvec;const struct bio_vec *bvec; +const struct bvecq *bvecq;const struct folio_queue *folioq;struct xarray *xarray;void __user *ubuf;
@@ -80,6 +82,7 @@ struct iov_iter { };union {unsigned long nr_segs; +u16 bvecq_slot;u8 folioq_slot;loff_t xarray_start; };
@@ -148,6 +151,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)return iov_iter_type (i) ==ITER_DISCARD ; } +static inline bool iov_iter_is_bvecq (const struct iov_iter *i) +{ +return iov_iter_type (i) ==ITER_BVECQ ; +} +static inline bool iov_iter_is_folioq (const struct iov_iter *i) {return iov_iter_type (i) ==ITER_FOLIOQ ;
@@ -303,6 +311,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvecvoid iov_iter_bvec (struct iov_iter *i,unsigned int direction,const struct bio_vec *bvec,unsigned long nr_segs,size_t count);void iov_iter_discard (struct iov_iter *i,unsigned int direction,size_t count); +void iov_iter_bvec_queue (struct iov_iter *i,unsigned int direction, +const struct bvecq *bvecq, +unsigned int first_slot,unsigned int offset,size_t count);void iov_iter_folio_queue (struct iov_iter *i,unsigned int direction,const struct folio_queue *folioq,unsigned int first_slot,unsigned int offset,size_t count);
lib/iov_iter.c +438 -5
534 lines not on this page · raw message
lib/scatterlist.c +66 -2
92 lines not on this page · raw message
lib/tests/kunit_iov_iter.c +277 -0
317 lines not on this page · raw message
[email protected]
AI reviewRe: [PATCH v19 03/17] iov_iter: Add a segmented queue of bio_vec[]
Sashiko has reviewed this patch and found no issues.
Quoted 15 lines
Add the concept of a segmented queue of bio_vec[] arrays. This allows an indefinite quantity of elements to be handled and allows things like network filesystems and crypto drivers to glue bits on the ends without having to reallocate the array.
The bvecq struct that defines each segment also carries capacity/usage information along with flags indicating whether the constituent memory regions need freeing or unpinning. The bvecq structs are refcounted to allow a queue to be extracted in batches and split between a number of subrequests.
The bvecq can have the bio_vec[] it manages allocated in with it, but this is not required. A flag is provided for if this is the case as comparing ->bv to ->__bv is not sufficient to detect this case.
Quoted 8 lines
Add an iterator type
ITER_BVECQfor it. This is intended to replaceITER_FOLIOQ(andITER_XARRAY).Note that the prev pointer is only really needed for
iov_iter_revert()and could be dispensed with if structiov_itercontained the head information as well as the current point.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
David Howells
maintainerPATCH v19 4/17 netfs: Add some tools for managing bvecq chains
In reply to David HowellsProvide a selection of tools for managing bvec queue chains and a memory pool for writeback to draw upon if needed. The tools include the allocation, prepopulation and refcounting of bvecqs and bvecq chains. When a chain is being cleaned up, the memory segments will be appropriately disposed of according to the flags on each bvecq in that chain, allowing for mixed memory cleanup types.
A pair of memory pools is provided for emergency allocation under low memory conditions. The two pools are intended for use in different circumstances with different usage characteristics. The request-level pool will be used by netfslib requests to build a rolling buffer as it goes along, whereas the subrequest-level pool will be used by subrequests (e.g. cifs can use it to allocate an encryption buffer). Separate pools are provided to avoid deadlocking between levels.
The main use for this is to replace the use of both folio_queues and kvmalloc'd bio_vec arrays in netfslib and creating a structure that allows network filesystems to easily build an entire message by gluing bits of protocol on either side of the bvecq chains and also glue chains together for sparse writes and compound operations. This allows TCP-based transports to pass a message in a single sendmsg() call and avoid corking as this makes things more efficient.
This will also be used to do things like creating an encryption buffer in cifs or a directory content buffer in afs.
Note that these tools are placed in netfslib for the moment, but can be moved to lib/ later if any non-netfslib code starts to use them. The iov_iter bits don't depend on these tools and must live in the core.
--- fs/netfs/Makefile | 1 + fs/netfs/bvecq.c | 429 ++++++++++++++++++++++++++++++++++++++ fs/netfs/internal.h | 5 + fs/netfs/main.c | 46 +++- fs/netfs/objects.c | 1 - fs/netfs/rolling_buffer.c | 1 - fs/netfs/stats.c | 6 +- include/linux/bvecq.h | 145 +++++++++++++ include/linux/netfs.h | 1 + lib/iov_iter.c | 17 +- lib/scatterlist.c | 5 +- 11 files changed, 645 insertions(+), 12 deletions(-) create mode 100644 fs/netfs/bvecq.c
fs/netfs/Makefile +1 -0
@@ -3,6 +3,7 @@ netfs-y := \ buffered_read.o \ buffered_write.o \ + bvecq.o \ direct_read.o \ direct_write.o \ iterator.o \
fs/netfs/bvecq.c new file +429 -0
429 lines not on this page · raw message
fs/netfs/internal.h +5 -0
@@ -11,6 +11,8 @@#include <linux/netfs.h> #include <linux/fscache.h> #include <linux/fscache-cache.h> +#include <linux/mempool.h> +#include <linux/semaphore.h> #include <trace/events/netfs.h> #include <trace/events/fscache.h>
@@ -46,6 +48,8 @@ extern spinlock_t netfs_proc_lock;extern mempool_t netfs_request_pool;extern mempool_t netfs_subrequest_pool;extern mempool_t netfs_folioq_pool; +extern mempool_t netfs_bvecq_pool[4 ]; +extern struct semaphore netfs_bvecq_pool_sem[4 ];#ifdef CONFIG_PROC_FS static inline void netfs_proc_add_rreq (struct netfs_io_request *rreq)
@@ -205,6 +209,7 @@ extern atomic_t netfs_n_wh_retry_write_subreq;extern atomic_t netfs_n_wb_lock_skip;extern atomic_t netfs_n_wb_lock_wait;extern atomic_t netfs_n_folioq; +extern atomic_t netfs_n_bvecq;int netfs_stats_show (struct seq_file *m,void *v);
fs/netfs/main.c +45 -1
76 lines not on this page · raw message
fs/netfs/objects.c +0 -1
@@ -6,7 +6,6 @@*/ #include <linux/slab.h> -#include <linux/mempool.h> #include "internal.h" static void netfs_free_request (struct work_struct *work);
fs/netfs/rolling_buffer.c +0 -1
@@ -6,7 +6,6 @@*/ #include <linux/bitops.h> -#include <linux/mempool.h> #include <linux/pagemap.h> #include <linux/rolling_buffer.h> #include <linux/slab.h>
fs/netfs/stats.c +5 -1
@@ -47,6 +47,7 @@ atomic_t netfs_n_wh_retry_write_subreq;atomic_t netfs_n_wb_lock_skip;atomic_t netfs_n_wb_lock_wait;atomic_t netfs_n_folioq; +atomic_t netfs_n_bvecq;int netfs_stats_show (struct seq_file *m,void *v) {
@@ -88,10 +89,13 @@ int netfs_stats_show(struct seq_file *m, void *v)atomic_read (&netfs_n_rh_retry_read_subreq),atomic_read (&netfs_n_wh_retry_write_req), atomic_read(&netfs_n_wh_retry_write_subreq)); -seq_printf (m,"Objs : rr=%u sr=%u foq=%u wsc=%u\n" , +seq_printf (m,"Objs : rr=%u sr=%u bq=%u foq=%u bqs=%d,%d wsc=%u\n" ,atomic_read (&netfs_n_rh_rreq),atomic_read (&netfs_n_rh_sreq), +atomic_read (&netfs_n_bvecq),atomic_read (&netfs_n_folioq), +READ_ONCE (netfs_bvecq_pool_sem[1 ].count), +READ_ONCE (netfs_bvecq_pool_sem[2 ].count),atomic_read (&netfs_n_wh_wstream_conflict)); seq_printf(m, "WbLock : skip=%u wait=%u\n", atomic_read(&netfs_n_wb_lock_skip),
include/linux/bvecq.h +145 -0
159 lines not on this page · raw message
include/linux/netfs.h +1 -0
@@ -17,6 +17,7 @@#include <linux/workqueue.h> #include <linux/fs.h> #include <linux/pagemap.h> +#include <linux/bvecq.h> #include <linux/uio.h> #include <linux/rolling_buffer.h>
lib/iov_iter.c +10 -7
@@ -558,11 +558,8 @@ static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by) slot =0 ;continue ; } -if (!by) -break ; len = bq->bv[slot].bv_len; -if (likely (by < len))break ; by -= len;
@@ -1183,7 +1180,7 @@ static ssize_t iter_bvecq_get_pages(struct iov_iter *iter, pages = *ppages;/* Now transcribe the page pointers. */ - do { +for (;;) {const struct bio_vec *bv;size_t boff, blen;
@@ -1198,6 +1195,9 @@ static ssize_t iter_bvecq_get_pages(struct iov_iter *iter,continue ; } +if (nr >= maxpages || extracted >= maxsize) +break ; + bv = &bvecq->bv[slot]; boff = bv->bv_offset; blen = bv->bv_len;
@@ -1230,7 +1230,7 @@ static ssize_t iter_bvecq_get_pages(struct iov_iter *iter, offset =0 ; slot++; } - }while (nr < maxpages && extracted < maxsize); + } iter->bvecq = bvecq; iter->bvecq_slot = slot;
@@ -1893,7 +1893,7 @@ static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter, offset = iter->iov_offset; slot = iter->bvecq_slot; - do { +for (;;) {const struct bio_vec *bv;size_t boff, blen;
@@ -1908,6 +1908,9 @@ static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,continue ; } +if (nr >= maxpages || extracted >= maxsize) +break ; + bv = &bvecq->bv[slot]; boff = bv->bv_offset; blen = bv->bv_len;
@@ -1934,7 +1937,7 @@ static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter, offset =0 ; slot++; } - }while (nr < maxpages && extracted < maxsize); + } iter->bvecq = bvecq; iter->bvecq_slot = slot;
lib/scatterlist.c +4 -1
@@ -1286,7 +1286,7 @@ static ssize_t extract_bvecq_to_sg(struct iov_iter *iter, maxsize =umin (maxsize,iov_iter_count (iter)); -while (sg_max >0 && ret < maxsize) { +for (;;) {const struct bio_vec *bv;size_t blen, part;
@@ -1301,6 +1301,9 @@ static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,continue ; } +if (sg_max ==0 || ret >= maxsize) +break ; + bv = &bvecq->bv[slot]; blen = bv->bv_len;
[email protected]
AI reviewRe: [PATCH v19 04/17] netfs: Add some tools for managing bvecq chains
Thank you for your contribution! Sashiko AI review found 4 potential issues to consider: - [High] netfs: dangling prev pointer in bvecq_put() when next node survives - [Medium] netfs: false OOM in…
Thank you for your contribution! Sashiko AI review found 4 potential issues to consider:
- [High] netfs: dangling prev pointer in bvecq_put() when next node survives
- [Medium] netfs: false OOM in bvecq_alloc_buffer2() when pre_slots fills first node
- [Medium] netfs: bvecq_alloc_buffer2() stops page fill early on BVECQ_HIT_OOM
- [Low] netfs: direct access to private semaphore count in netfs_stats_show()
--
commit abff4f06be20b79f4ce9a5e857e386c2486d016a
Author: David Howells <[email protected]>
netfs: Add some tools for managing bvecq chains
This commit adds helpers for allocating, populating, refcounting, and freeing bvecq chains along with emergency mempools for request and subrequest allocations.
Quoted 5 lines
fs/netfs/bvecq.c new file +0
-0
[ ... ]
Quoted 56 lines
fs/netfs/bvecq.c new file +56
-0+struct bvecq *bvecq_alloc_buffer2 (size_t size,unsigned int pre_slots,gfp_t gfp, +unsigned int *flags) +{ +struct bvecq *head =NULL , *tail =NULL ; +size_t count = pre_slots +DIV_ROUND_UP (size,PAGE_SIZE ); + +_enter ("%zx,%zx,%u" , size, count, pre_slots); + + *flags &= ~BVECQ_HIT_OOM ; +if (WARN_ON_ONCE (pre_slots >BVECQ_POOL_SLOTS )) +return NULL ; + + do { + struct page **pages; + size_t unused, want, got, slot; + + if (!tail || bvecq_is_full(tail)) { + struct bvecq *bq; + + bq = bvecq_alloc_one(min(count, BVECQ_4KB_SLOTS), gfp, flags); + if (!bq) + goto oom; + + if (tail) + bvecq_append(tail, bq); + else + head = bq; + tail = bq; + + /* Need to clear pre slots and pages[], so just clear all. */ + memset(tail->bv, 0, tail->max_slots * sizeof(tail->bv[0])); + tail->mem_type = BVECQ_MEM_ALLOCED; + if (tail == head) { + tail->nr_slots = pre_slots; + count -= pre_slots; + } + } + + if (!count) + break; + + unused = tail->max_slots - tail->nr_slots; + + /* We can put pages[] at the tail of bv[] as the elements are + * smaller and we won't overrun if the slots are filled in + * ascending order. + */ + pages = (struct page **)&tail->bv[tail->max_slots]; + pages -= unused; + + want = min(count, unused); + got = alloc_pages_bulk(gfp, want, pages); + if (!got) { + *flags |= BVECQ_HIT_OOM; + goto pages_oom; + }
fs/netfs/bvecq.c line 263
Quoted 10 lines
fs/netfs/bvecq.c new file +10
-0+ + slot = tail->nr_slots; +for (int i =0 ; i < got; i++) +bvec_set_page (&tail->bv[slot++], pages[i],PAGE_SIZE ,0 ); + +bvecq_filled_to (tail, slot); + count -= got; + +if (unlikely (*flags &BVECQ_HIT_OOM ) && (*flags &BVECQ_SHORT_OKAY )) +break ;
[Severity: Medium]
Does checking BVECQ_HIT_OOM after every alloc_pages_bulk() call stop populating tail prematurely?
When bvecq_alloc_one() falls back to an emergency mempool, it sets BVECQ_HIT_OOM in *flags. Under low-memory conditions where zone watermarks are not met, alloc_pages_bulk() falls back to allocating a single page per call and relies on the caller's loop to fill the remaining slots, so breaking here before tail is full leaves a 29-slot pool bvecq with only one page populated.
Quoted 4 lines
fs/netfs/bvecq.c new file +4
-0+ }while (count >0 ); + +out: +return head;
fs/netfs/bvecq.c line 277
Quoted 10 lines
fs/netfs/bvecq.c new file +10
-0+ +pages_oom: +/* If tail is empty, discard it. */ +if (tail != head && tail->nr_slots ==0 ) { + tail->prev->next =NULL ; +bvecq_put (tail); + } +oom: +if (!head || (tail == head && head->nr_slots == pre_slots)) + goto discard;/* Allocated no pages */
[Severity: Medium]
Can bvecq_alloc_buffer2() fail with a false OOM when pre_slots fills the first bvecq node?
pre_slots is allowed to be up to BVECQ_POOL_SLOTS. If the first bvecq is allocated from an emergency mempool (max_slots == BVECQ_POOL_SLOTS) with pre_slots == BVECQ_POOL_SLOTS and size > 0, setting tail->nr_slots = pre_slots leaves unused == 0 and want == 0, so alloc_pages_bulk(gfp, 0, pages) returns 0 and triggers pages_oom instead of allocating a second bvecq node.
Also, when pages_oom discards an empty non-head tail node via bvecq_put(tail), tail is not reset to tail->prev before the oom check tests tail == head && head->nr_slots == pre_slots. If the second node fails its first page allocation while head only contains pre_slots, bvecq_alloc_buffer2() with BVECQ_SHORT_OKAY will return a buffer with zero allocated pages.
[ ... ]
Quoted 8 lines
fs/netfs/bvecq.c new file +8
-0+void bvecq_put (struct bvecq *bq) +{ +struct bvecq *next; + +for (; bq; bq = next) { +if (!refcount_dec_and_test (&bq->ref)) +break ; +for (int slot =0 ; slot < bq->nr_slots; slot++)
fs/netfs/bvecq.c line 338
Quoted 10 lines
fs/netfs/bvecq.c new file +10
-0+bvecq_free_slot (bq, slot); + next = bq->next; +netfs_stat_d (&netfs_n_bvecq); + +if (bq->from_pool !=BVECQ_USE_NO_POOL ) +mempool_free (bq, &netfs_bvecq_pool[bq->from_pool]); +else +kfree (bq); + } +}
[Severity: High]
Does bvecq_put() leave a dangling prev pointer on a surviving successor node?
When an interior node in a chain has an extra reference held (for example, via subrequest or retry pinning) and bvecq_put() frees the preceding node, the loop breaks when refcount_dec_and_test(&bq->ref) returns false on the surviving node without clearing its prev pointer. A subsequent backward walk such as iov_iter_revert() -> iov_iter_bvecq_revert() will then dereference the freed predecessor node:
lib/iov_iter.c:iov_iter_bvecq_revert() {
...
if (slot == 0) {
bq = bq->prev;
slot = bq->nr_slots;
continue;
}
...
}Quoted 7 lines
fs/netfs/stats.c +0
-0@@ -88,10 +89,13 @@ int netfs_stats_show(struct seq_file *m, void *v)atomic_read (&netfs_n_rh_retry_read_subreq), atomic_read(&netfs_n_wh_retry_write_req),
fs/netfs/stats.c int netfs_stats_show(struct seq_file *m, void *v) · line 99
Quoted 10 lines
fs/netfs/stats.c +4
-1atomic_read(&netfs_n_wh_retry_write_subreq)); -seq_printf (m,"Objs : rr=%u sr=%u foq=%u wsc=%u\n" , +seq_printf (m,"Objs : rr=%u sr=%u bq=%u foq=%u bqs=%d,%d wsc=%u\n" ,atomic_read (&netfs_n_rh_rreq),atomic_read (&netfs_n_rh_sreq), +atomic_read (&netfs_n_bvecq),atomic_read (&netfs_n_folioq), +READ_ONCE (netfs_bvecq_pool_sem[1 ].count), +READ_ONCE (netfs_bvecq_pool_sem[2 ].count),atomic_read (&netfs_n_wh_wstream_conflict));
[Severity: Low]
struct semaphore in include/linux/semaphore.h is marked "Please don't access any members of this structure directly", and its count field is an unsigned int protected by sem->lock (formatted here with %d).
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
David Howells
maintainerRe: [PATCH v19 04/17] netfs: Add some tools for managing bvecq chains
In reply to David HowellsHi Christian,
If you pull this patch, can you apply the attached diff to it to fix some issues that Sashiko brought up[1]?
- In bvecq_alloc_buffer2(), fix OOM discard where the requested number of pre-slots fills up a entire bvecq.
- In bvecq_alloc_buffer2(), add comment on why we want to exit the buffer alloc loop immediately.
- In bvecq_put(), clear the prev pointer of the next bvecq if it points back to the one being released.
- Fix bvecq_append() kdoc.
- Change %flags to @flags in function kdoc when referring to func arg.
Thanks,
David
[1] https://sashiko.dev/#/patchset/20261119083851.3306418-1-dhowells%40redhat.com
---
fs/netfs/bvecq.c +15 -5
@@ -61,7 +61,7 @@ EXPORT_SYMBOL(bvecq_dump);* that at least some progress can be made. * * Return: The node pointer or NULL on allocation failure. BVECQ_HIT_OOM is -* set in *flags if an allocation failure occurs. +* set in *@flags if an allocation failure occurs. */ struct bvecq *bvecq_alloc_one (size_t nr_slots,gfp_t gfp,unsigned int *flags) {
@@ -136,10 +136,10 @@ EXPORT_SYMBOL(bvecq_alloc_one);* * Allocate a chain of bvecq nodes providing at least the requested cumulative * number of slots. Each node is a maximum of 4KiB in size. If -* %BVECQ_SHORT_OKAY is set in *%flags, then the chain may be shortened on OOM. +* %BVECQ_SHORT_OKAY is set in *@flags, then the chain may be shortened on OOM. * * Return: The first node pointer or NULL on allocation failure. BVECQ_HIT_OOM -* is set in *flags if an allocation failure occurs. +* is set in *@flags if an allocation failure occurs. */ struct bvecq *bvecq_alloc_chain (size_t nr_slots,gfp_t gfp,unsigned int *flags) {
@@ -191,7 +191,7 @@ EXPORT_SYMBOL(bvecq_alloc_chain);* refcounts set to 1 and can be passed to MSG_SPLICE_PAGES. * * Return: The first node pointer or NULL on allocation failure. BVECQ_HIT_OOM -* is set in *flags if a bvecq chain allocation failure occurs (though if +* is set in *@flags if a bvecq chain allocation failure occurs (though if * %BVECQ_SHORT_OKAY is set, the func may return early but will have at least * one bvecq in the allocation). */
@@ -200,6 +200,7 @@ struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp {struct bvecq *head =NULL , *tail =NULL ;size_t count = pre_slots +DIV_ROUND_UP (size,PAGE_SIZE ); +bool got_page = false;_enter ("%zx,%zx,%u" , size, count, pre_slots);
@@ -258,7 +259,12 @@ struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfpbvecq_filled_to (tail, slot); count -= got; + got_page = true; +/* TODO: If we got an OOM trying to allocate a bvecq, we should +* probably stop allocating pages unless/until we have an +* emergency pool of those also. +*/ if (unlikely (*flags &BVECQ_HIT_OOM ) && (*flags &BVECQ_SHORT_OKAY ))break ; }while (count >0 );
@@ -273,7 +279,9 @@ struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfpbvecq_put (tail); } oom: -if (!head || (tail == head && head->nr_slots == pre_slots)) +if (!head) + goto out; +if (!got_page && size >0 ) goto discard;/* Allocated no pages */ if (*flags &BVECQ_SHORT_OKAY ) goto out;
@@ -328,6 +336,8 @@ void bvecq_put(struct bvecq *bq)for (int slot =0 ; slot < bq->nr_slots; slot++)bvecq_free_slot (bq, slot); next = bq->next; +if (next && next->prev == bq) + next->prev =NULL ;netfs_stat_d (&netfs_n_bvecq);if (bq->from_pool !=BVECQ_USE_NO_POOL )
include/linux/bvecq.h +1 -1
@@ -162,7 +162,7 @@ static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int sl }/** -* bvecq_append - Get the next bvecq with appropriate barrier +* bvecq_append - Attach new bvecq to chain with appropriate barrier * @to: The bvecq to append to * @add: The bvecq to append *
David Howells
maintainerPATCH v19 5/17 iov_iter: Add an API document for the bvecq structure
In reply to David HowellsAdd an API description document for the bvecq structure.
--- Documentation/core-api/bvecq.rst | 259 +++++++++++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + MAINTAINERS | 1 + fs/netfs/bvecq.c | 2 + include/linux/bvecq.h | 2 + 5 files changed, 265 insertions(+) create mode 100644 Documentation/core-api/bvecq.rst
Documentation/core-api/bvecq.rst new file +259 -0
@@ -0,0 +1,259 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +bvecq - Chainable bio_vec queue structure +========================================= + +Author: David Howells <[email protected]> + +.. Contents: + + * Overview + * Allocating bvecqs + * Refcounting + * Expandable buffer + * Producer functions + * Consumer functions + * Utility functions + * I/O iterators + * API Function Reference + + +Overview +======== + +The ``bvecq`` struct can be used to form chains of arrays of ``bio_vec`` +structures - kind of a single layer of indirection. Each bvecq segment points +to the previous and next bvecq segments in the chain, but is not intended to +form a circular queue as there are applications where the producer is adding to +one end whilst the consumer is taking off the other. + +To use this, the caller should do:: + + #include <linux/bvecq.h> + +The publicly accessible members of the structure are:: + + struct bvecq { + struct bvecq *next; + struct bvecq *prev; + u32 priv; + u16 nr_slots; + u16 max_slots; + enum bvecq_mem mem_type:3; + struct bio_vec *bv; + ... + }; + +A pair of pointers are provided, ``next`` and ``prev``, that point to the +segments on either side of the segment being accessed. Whilst this is a +doubly-linked list, it is intentionally not a circular list; the outward +sibling pointers in terminal segments should be NULL. + +The ``priv`` member is for the owner of the bvecq to use as they will. + +The ``bv`` pointer points at an array of bio_vecs that list a number of regions +of memory. ``max_slots`` indicates how large the array is and ``nr_slots`` +says how much of the array is being used. Note that there are access functions +for nr_slots that insert appropriate barriering in the case that there are +separate producer and consumer threads. + +The ``mem_type`` member informs bvecq_put() of how to clean up the bio_vecs +attached to this segment. It can be one of: + + * ``BVECQ_MEM_EXTERNAL`` - Externally managed; no cleanup. + * ``BVECQ_MEM_PAGECACHE`` - Part of pagecache; put_page() used. + * ``BVECQ_MEM_GUP`` - Pinned userspace memory; unpin_user_page() used. + * ``BVECQ_MEM_ALLOCED`` - Allocated page; __free_pages() used, may be pooled. + + +Allocating bvecqs +================= + +A number of functions are provided to allocate bvecqs:: + + struct bvecq *bvecq_alloc_one(size_t nr_slots, + gfp_t gfp, + unsigned int *flags); + struct bvecq *bvecq_alloc_chain(size_t nr_slots, + gfp_t gfp, + unsigned int *flags); + struct bvecq *bvecq_alloc_buffer(size_t size, + gfp_t gfp, + unsigned int *flags); + struct bvecq *bvecq_alloc_buffer2(size_t size, + unsigned int pre_slots, + gfp_t gfp, + unsigned int *flags); + +The first of these, ``bvecq_alloc_one()`` allocates a single bvecq with at +least the requested number of inline slots. The size of the object is rounded +up to the size of the slab granule that it will be allocated in and the number +of slots increased appropriately to match. + +Two convenience slot count constants are provided: + + * ``BVECQ_POOL_SLOTS`` - The right number of slots to alloc from the mempool + * ``BVECQ_4KB_SLOTS`` - The right number of slots to get a 4K allocation + +``bvecq_alloc_chain()`` will allocate a sufficiently large chain of bvecqs to +hold the requested number of slots, capping each allocation at 4K. + +``bvecq_alloc_buffer()`` will allocate a chain and populate it, allocating +sufficient pages to provide a buffer of the specified size attached to the +chain. ``bvecq_alloc_buffer2()`` is similar, but it reserves some unused slots +at the front of the chain. Note that these functions may add multipage pages +in the future. + +With regard to the ``*flags``, if ``BVECQ_USE_REQ_POOL`` or +``BVECQ_USE_SUBREQ_POOL`` is set therein, then the netfslib writeback bvecq +pools will be used to backstop the allocation if the initial slab request +fails; in such a case, the number of slots in the affected bvecq will be +``BVECQ_POOL_SLOTS`` and ``*flags`` will have ``BVECQ_HIT_OOM`` set in it. + +There are two pools, one for use at the request level and one at the subrequest +level so that a process which has allocated at one level won't then deadlock +against the other. The request-level pool allows 50 processes in and offers +them 2 bvecqs each. The subrequest-level pool allows 2 processes in and offers +them sufficient bvecqs to point at 16MiB worth of pages each. + +For the chain and buffer allocation functions, if ``BVECQ_SHORT_OKAY`` is set +in ``*flags`` then the returned chain may be shorter than requested if an OOM +condition is hit but the appropriate mempool is tapped. In such a case, +``BVECQ_HIT_OOM`` will have been set in ``*flags``. + +Access to the bvecq mempools are throttled. Only a certain number of processes +may be using each pool at any one time and if the calling process is granted +access, ``BVECQ_HOLD_POOL_SEM`` is set in the flags. To end the emergency pool +state, the caller must call:: + + void bvecq_pool_unlock(unsigned int *flags); + +after it is done allocating with the flags left by the last allocation call. +It is not required that a process has returned all the objects from the pool +when it calls this - but it is required to have returned all the objects before +it can allocate from the pool again. + +Whilst holding a count on the semaphore, it may over-allocate a little from the +pool, but must return what it has allocated as soon as possible and may then +continue allocating as long as it holds the semaphore. + + +Refcounting +=========== + +Each bvecq has a refcount and two functions are provided to handle that:: + + struct bvecq *bvecq_get(struct bvecq *bq); + void bvecq_put(struct bvecq *bq); + +The first of these gets a new reference and the second drops a reference. When +the refcount hits zero, the second iteratively drops the refs along the chain +until it hits a segment that has a refcount that doesn't reduce to zero. + + +Expandable buffer +================= + +This facility provides a function to manage an expandable buffer:: + + int bvecq_expand_buffer(struct bvecq **_buffer, + size_t *_cur_size, + size_t size, + gfp_t gfp); + +Pages are added to the bvecq chain and the chain expanded such that, on +returning, it holds ``size`` amount of space (without losing any content). The +caller needs to retain ``*_cur_size`` between calls as that tracks the amount +of space currently allocated. Note that pages in the chain may be allocated as +compound pages larger than PAGE_SIZE. + + +Producer functions +================== + +When building and filling a bvecq, there are a number of utility functions that +can be used to provide the correct barriering and to query the fields in the +bvecq currently being filled. + +Firstly, there's a function to check to see if a bvecq has any spare slots +left:: + + bool bvecq_is_full(const struct bvecq *bvecq); + +If not, ``bvecq_alloc_one()`` can be called to allocate a new segment which can +be initialised and then added to the chain with:: + + void bvecq_append(struct bvecq *to, struct bvecq *add); + +bio_vec slots in the segment can then be filled and the number of occupied +slots updated by calling:: + + void bvecq_filled_to(struct bvecq *bvecq, + unsigned int to); + +to insert the appropriate barrier. + + +Consumer functions +================== + +When consuming and destroying a bvecq, there are some utility functions for +that too, providing the needed barriering. Firstly, there's a function to find +out how many slots have been used in the current bvecq:: + + unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq); + +Or, more conveniently, if the current slot position is valid within the bvecq:: + + bool bvecq_acquire_slot(const struct bvecq *bvecq, + unsigned int slot); + +If this returns true, the slot is valid and can be looked at; if it returns +false, the bvecq may have been consumed. If it has, the next bvecq can be +obtained by calling:: + + struct bvecq *bvecq_next(const struct bvecq *bq); + +which inserts the appropriate barrier - though bvecq_acquire_slot should be +rechecked after calling this in case the producer added more slots and then +extended the list. + + +Utility functions +================= + +A utility function is provided to list the bio_vecs pointed to by a bvecq +chain:: + + void bvecq_dump(const struct bvecq *bq); + + +I/O iterators +============= + +bvecq chains can be used with the ``iov_iter`` facility too. An iterator can +be initialised to point at a chain by calling:: + + void iov_iter_bvec_queue(struct iov_iter *i, + unsigned int direction, + const struct bvecq *bvecq, + unsigned int first_slot, + unsigned int offset, + size_t count); + +In addition to setting the iterator, direction and count, the first bvecq in +the chain to be iterated over is set, along with the starting slot in that +iterator and the offset into the slot. Once the iterator is set up, it can be +treated like any other I/O iterator. + +In a producer/consumer environment, it is expected that when an iterator is set +up, all the bvecqs and all the slots within the count of the iterator must +exist and must be populated before this function is called. + + +API Function Reference +====================== + +.. kernel-doc:: include/linux/bvecq.h +.. kernel-doc:: fs/netfs/bvecq.c
Documentation/core-api/index.rst +1 -0
@@ -39,6 +39,7 @@ Library functionality that is used throughout the kernel. kref cleanup assoc_array + bvecq folio_queue xarray maple_tree
MAINTAINERS +1 -0
@@ -10386,6 +10386,7 @@ M: Paulo Alcantara <[email protected]> L: [email protected] L: [email protected] S: Supported +F: Documentation/core-api/bvecq.rst F: Documentation/filesystems/caching/ F: Documentation/filesystems/netfs_library.rst F: fs/netfs/
fs/netfs/bvecq.c +2 -0
@@ -3,6 +3,8 @@* * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved. * Written by David Howells ([email protected]) +* +* See also Documentation/core-api/bvecq.rst */ #include <linux/bvecq.h>
include/linux/bvecq.h +2 -0
@@ -3,6 +3,8 @@* * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved. * Written by David Howells ([email protected]) +* +* See also Documentation/core-api/bvecq.rst */ #ifndef _LINUX_BVECQ_H
[email protected]
AI reviewRe: [PATCH v19 05/17] iov_iter: Add an API document for the bvecq structure
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] docs: bvecq: malformed kernel-doc markup in bvecq.c and bvecq.h --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] docs: bvecq: malformed kernel-doc markup in bvecq.c and bvecq.h --
commit 4351a64f55127d5c3a801238a9cedf6a1d7b17ac
Author: David Howells <[email protected]>
iov_iter: Add an API document for the bvecq structure
This commit adds Documentation/core-api/bvecq.rst documenting the bvecq structure and its API, and links it into the core-api index and MAINTAINERS.
Quoted 5 lines
Documentation/core-api/bvecq.rst new file +0
-0
[ ... ]
Documentation/core-api/bvecq.rst line 259
Quoted 5 lines
Documentation/core-api/bvecq.rst new file +5
-0+API Function Reference +====================== + +.. kernel-doc:: include/linux/bvecq.h +.. kernel-doc:: fs/netfs/bvecq.c
[Severity: Low]
Pulling these two files into the documentation build exposes a few kernel-doc issues in the comments introduced earlier in the series.
In fs/netfs/bvecq.c, the kernel-doc comments for bvecq_alloc_one(), bvecq_alloc_chain() and bvecq_alloc_buffer2() write *flags and *%flags instead of *@flags, which leaves unmatched single asterisks in the generated reStructuredText and triggers docutils inline emphasis warnings during make htmldocs:
fs/netfs/bvecq.c:bvecq_alloc_chain()
* Allocate a chain of bvecq nodes providing at least the requested cumulative
* number of slots. Each node is a maximum of 4KiB in size. If
* %BVECQ_SHORT_OKAY is set in *%flags, then the chain may be shortened on OOM. *
* Return: The first node pointer or NULL on allocation failure. BVECQ_HIT_OOM
* is set in *flags if an allocation failure occurs.
Also, in include/linux/bvecq.h, the kernel-doc summary line for bvecq_append() appears to be copy-pasted from bvecq_next():
include/linux/bvecq.h:bvecq_append()
/**
* bvecq_append - Get the next bvecq with appropriate barrier
* @to: The bvecq to append to
* @add: The bvecq to append
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
David Howells
maintainerPATCH v19 6/17 iov_iter: Add a bvecq kunit test
In reply to David HowellsAdd a bvecq kunit test to exercise the bvecq iterator in a bunch of different ways. Each kunit test sets up a series of folios of different sizes (1, 2, 4 or 8 pages) and fills each subpage thereof with a distinct predictable pattern that it can then sample to check that the iterator appears to be in the right place.
The folios are then assembled into a bvecq chain depending on a particular recipe and partialness list and then advance, revert and copy-out is tested, along with checking iov_iter_npages() at each point.
Each test is a combination of a bvecq chain recipe:
- A single bvecq with 16 folios with no gaps (zero-len bio_vec).
- A chain of two bvecqs, each with 8 folios and no gaps.
- As the previous, but with a gap at the end of the first chain.
- A chain of four bvecqs, each with 4 folios and scattered gaps, including at the front and back.
- A chain of five bvecqs, 3 with folios and two with no folios, one empty and one with a couple of gaps.
- A chain of seven bvecqs, 3 with folios, 3 completely empty (including first and last bvecqs in the chain) and 1 with just a couple of gaps.
and a selection of folio parts:
- Whole folios only.
- Scattered parts of folios, with some contiguities.
- Every folio with the first and last bytes excluded.
and a test algorithm:
- For each possible starting position in the bvecq, do a small advance, check fill, a couple of small reverts, each checked and then advance to the next starting position.
- Starting at the beginning, advance in jumps by a largish step forward, checking after each jump; then when the end is hit revert in jumps by a slightly larger step back, checking each time; then double the step and repeat until the step exceeds the size of the iterator.
iov_iter_alignment() is compared at the beginning of one of the test sets to a compound of alignments from an ITER_BVEC. Note that the ITER_BVEC alignment check malfunctions if given a zero-length bio_vec array.
Exercising extraction, copy-from and copy-to is done in kunit_iov_iter.c, with those bits added in a different patch.
--- MAINTAINERS | 1 + lib/Kconfig.debug | 13 + lib/tests/Makefile | 1 + lib/tests/kunit_bvecq.c | 603 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 618 insertions(+) create mode 100644 lib/tests/kunit_bvecq.c
MAINTAINERS +1 -0
@@ -10395,6 +10395,7 @@ F: include/linux/fscache*.h F: include/linux/netfs.h F: include/trace/events/fscache.h F: include/trace/events/netfs.h +F: lib/tests/kunit_bvecq.c FILESYSTEMS [STACKABLE] M: Miklos Szeredi <[email protected]>
lib/Kconfig.debug +13 -0
@@ -2385,6 +2385,19 @@ config TEST_IOV_ITER If unsure, say N. +config TEST_BVECQ + tristate "Test bvecq operation" if !KUNIT_ALL_TESTS + depends on KUNIT + depends on MMU + depends on NETFS_SUPPORT + default KUNIT_ALL_TESTS + help + Enable this to turn on testing of the operation of the bvecq + structure. This test is executed only once during system + boot (so affects only boot time), or at module load time. + + If unsure, say N. + config KPROBES_SANITY_TEST tristate "Kprobes sanity tests" if !KUNIT_ALL_TESTS depends on DEBUG_KERNEL
lib/tests/Makefile +1 -0
@@ -27,6 +27,7 @@ obj-$(CONFIG_GLOB_KUNIT_TEST) += glob_kunit.o obj-$(CONFIG_HASHTABLE_KUNIT_TEST) += hashtable_test.o obj-$(CONFIG_HASH_KUNIT_TEST) += test_hash.o obj-$(CONFIG_TEST_IOV_ITER) += kunit_iov_iter.o +obj-$(CONFIG_TEST_BVECQ) += kunit_bvecq.o obj-$(CONFIG_IS_SIGNED_TYPE_KUNIT_TEST) += is_signed_type_kunit.o obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o obj-$(CONFIG_LIST_KUNIT_TEST) += list-test.o
lib/tests/kunit_bvecq.c new file +603 -0
603 lines not on this page · raw message
[email protected]
AI reviewRe: [PATCH v19 06/17] iov_iter: Add a bvecq kunit test
Sashiko has reviewed this patch and found no issues.
Quoted 17 lines
Add a bvecq kunit test to exercise the bvecq iterator in a bunch of different ways. Each kunit test sets up a series of folios of different sizes (1, 2, 4 or 8 pages) and fills each subpage thereof with a distinct predictable pattern that it can then sample to check that the iterator appears to be in the right place.
The folios are then assembled into a bvecq chain depending on a particular recipe and partialness list and then advance, revert and copy-out is tested, along with checking
iov_iter_npages()at each point.Each test is a combination of a bvecq chain recipe:
- A single bvecq with 16 folios with no gaps (zero-len
bio_vec).
- A chain of two bvecqs, each with 8 folios and no gaps.
- As the previous, but with a gap at the end of the first chain.
[ ... ]
Quoted 4 lines
Exercising extraction, copy-from and copy-to is done in
kunit_iov_iter.c, with those bits added in a different patch.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
David Howells
maintainerPATCH v19 7/17 afs: Fix an inconsistency in the symlink code
In reply to David HowellsFix the afs symlink code to set ->directory_size to the size of the buffer, not PAGE_SIZE-1 or i_size. This doesn't actually change the operation since the value is ignored by the symlink writeback path (which writes to the cache).
This was spotted by Claude.
--- fs/afs/symlink.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/afs/symlink.c +1 -2
@@ -99,7 +99,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode) ret =netfs_alloc_folioq_buffer (NULL , &vnode->directory, &cur_size,PAGE_SIZE ,mapping_gfp_mask (vnode->netfs.inode.i_mapping)); - vnode->directory_size =PAGE_SIZE -1 ; + vnode->directory_size =PAGE_SIZE ;if (ret <0 )return ret; }
@@ -116,7 +116,6 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)trace_afs_file_error (vnode, -EFBIG , afs_file_error_dir_big);return -EFBIG ; } - vnode->directory_size = i_size;/* Copy the symlink. */ symlink = kmalloc_flex(struct afs_symlink, content, i_size + 1);
[email protected]
AI reviewRe: [PATCH v19 07/17] afs: Fix an inconsistency in the symlink code
Sashiko has reviewed this patch and found no issues.
Quoted 8 lines
Fix the afs symlink code to set ->directory_size to the size of the buffer, not PAGE_SIZE-1 or
i_size. This doesn't actually change the operation since the value is ignored by the symlink writeback path (which writes to the cache).This was spotted by Claude.
Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
David Howells
maintainerPATCH v19 8/17 netfs: Explicitly pass tail padding to netfs_writeback_single()
In reply to David HowellsChange netfs_writeback_single() so that the function is now given an explicit content length in addition to the buffer iterator. The buffer must be rounded out to the cache DIO block size and netfslib will then draw on this excess space to write padding into the cache beyond the content length specified. The caller must also now clear the padding space.
AFS directory code is modified to comply with this, including allocating pre-zeroed buffer space and clearing excess buffer space if the content is shrunk by a third party. It's not done during writeback as that may happen many times for a directory.
AFS symlink code, however, is modified such that the clearing is done when the content is written to the cache (expected to be a one-off per symlink, if it happens).
A further patch will make better use of the two different lengths.
--- Documentation/filesystems/netfs_library.rst | 47 +++++++++++++++------ fs/afs/dir.c | 17 ++++++-- fs/afs/dir_edit.c | 2 +- fs/afs/symlink.c | 16 ++++++- fs/netfs/write_issue.c | 13 ++++-- include/linux/netfs.h | 2 +- 6 files changed, 73 insertions(+), 24 deletions(-)
Documentation/filesystems/netfs_library.rst +34 -13
@@ -437,22 +437,43 @@ Monolithic Files API There is also a special API set for files for which the content must be read in a single RPC (and not written back) and is maintained as a monolithic blob -(e.g. an AFS directory), though it can be stored and updated in the local cache:: +(e.g. an AFS directory), though it can be stored and updated in the local +cache. + +To start with, there's a function to synchronously read the contents of such a +file into a buffer in a single operation:: + + ssize_t netfs_read_single(struct inode *inode, + struct file *file, + struct iov_iter *iter); + +If such a file is modified locally, it can be marked dirty with:: - ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_iter *iter); void netfs_single_mark_inode_dirty(struct inode *inode); + +which will schedule it for writeback later. When the writeback eventually +happens, the filesystem should call:: + int netfs_writeback_single(struct address_space *mapping, struct writeback_control *wbc, - struct iov_iter *iter); + struct iov_iter *iter, + size_t len); + +to write the data to the cache from its writepages method. -The first function reads from a file into the given buffer, reading from the -cache in preference if the data is cached there; the second function allows the -inode to be marked dirty, causing a later writeback; and the third function can -be called from the writeback code to write the data to the cache, if there is -one. +In both the read and writeback functions, the buffer needs to be rounded up to +a multiple of the cache DIO size, with ``iter->count`` indicating the size of +the buffer. For the write function, ``len`` should indicate the size of the +content which may be less than the buffer size. The gap between ``len`` and the +buffer size should be cleared by the filesystem as this will be used as padding +if the write size has to be increased (e.g. for the cache DIO). + +Note that, for the moment, rounding the buffer up to a multiple of PAGE_SIZE +instead of the cache DIO size should suffice. DIO size > PAGE_SIZE is not +currently supported. The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be -used. The writeback function requires the buffer to be of ITER_FOLIOQ type. +used. Clearing Stale Post-EOF Pagecache ---------------------------------
@@ -859,12 +880,12 @@ The methods defined in the table are: paths to ask the filesystem to update its idea of the file size. If not given, netfslib will set i_size and i_blocks and update the local cache cookie. - + * ``post_modify()`` [Optional] This is called after netfslib writes to the pagecache or when it allows an mmap'd page to be marked as writable. - + * ``begin_writeback()`` [Optional] Netfslib calls this when processing a writeback request if it
@@ -872,7 +893,7 @@ The methods defined in the table are: indicating it must be written to the server. This allows the filesystem to only set up writeback resources when it knows it's going to have to perform a write. - + * ``prepare_write()`` [Optional] This is called to allow the filesystem to limit the size of a
@@ -911,7 +932,7 @@ The methods defined in the table are: allows the filesystem to examine the state of the request, the subrequests in the indicated stream and of its own data and make adjustments or renegotiate resources. - + * ``invalidate_cache()`` [Optional] This is called by netfslib to invalidate data stored in the local
fs/afs/dir.c +13 -4
@@ -252,7 +252,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file) ret =netfs_alloc_folioq_buffer (NULL , &dvnode->directory, &cur_size, i_size, -mapping_gfp_mask (dvnode->netfs.inode.i_mapping)); +mapping_gfp_mask (dvnode->netfs.inode.i_mapping) | + __GFP_ZERO); dvnode->directory_size = cur_size;if (ret <0 )return ret;
@@ -278,6 +279,13 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)if (ret2 <0 ) ret = ret2; } + +if (ret >=0 && ret < dvnode->directory_size) { +iov_iter_folio_queue (&iter,ITER_DEST , dvnode->directory, +0 ,0 , dvnode->directory_size); +iov_iter_advance (&iter, ret); +iov_iter_zero (dvnode->directory_size - ret, &iter); + } }return ret;
@@ -2228,9 +2236,10 @@ static int afs_dir_writepages(struct address_space *mapping, }if (test_bit (AFS_VNODE_DIR_VALID , &dvnode->flags)) { -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , -i_size_read (&dvnode->netfs.inode)); - ret =netfs_writeback_single (mapping, wbc, &iter); +iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory, +0 ,0 , dvnode->directory_size); + ret =netfs_writeback_single (mapping, wbc, &iter, +i_size_read (&dvnode->netfs.inode));if (ret ==1 ) ret =0 ;/* Skipped write due to lock conflict. */ }
fs/afs/dir_edit.c +1 -1
@@ -122,7 +122,7 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz ret =netfs_alloc_folioq_buffer (NULL , &dvnode->directory, &cur_size, blend, -mapping_gfp_mask (dvnode->netfs.inode.i_mapping)); +mapping_gfp_mask (dvnode->netfs.inode.i_mapping) | __GFP_ZERO); dvnode->directory_size = cur_size;if (ret <0 ) goto fail;
fs/afs/symlink.c +14 -2
@@ -228,6 +228,10 @@ int afs_readlink(struct dentry *dentry, char __user *buffer, int buflen)/* * Write the symlink contents to the cache as a single blob. We then throw * away the page we used to receive it. +* +* Note that the buffer tail is cleared under a shared lock on ->validate_lock +* - though hopefully there won't be racing writers since the only +* 'modification' is to be either read from the server or locally created. */ int afs_symlink_writepages (struct address_space *mapping,struct writeback_control *wbc)
@@ -247,9 +251,17 @@ int afs_symlink_writepages(struct address_space *mapping,if (vnode->directory &&atomic64_read (&vnode->cb_expires_at) !=AFS_NO_CB_PROMISE ) { +size_t i_size =i_size_read (&vnode->netfs.inode); + +if (i_size < vnode->directory_size) { +iov_iter_folio_queue (&iter,ITER_DEST , vnode->directory,0 ,0 , + vnode->directory_size); +iov_iter_advance (&iter, i_size); +iov_iter_zero (vnode->directory_size - i_size, &iter); + }iov_iter_folio_queue (&iter,ITER_SOURCE , vnode->directory,0 ,0 , -i_size_read (&vnode->netfs.inode)); - ret =netfs_writeback_single (mapping, wbc, &iter); + vnode->directory_size); + ret =netfs_writeback_single (mapping, wbc, &iter, i_size); }if (ret ==0 ) {
fs/netfs/write_issue.c +10 -3
@@ -749,26 +749,33 @@ static int netfs_write_folio_single(struct netfs_io_request *wreq,* netfs_writeback_single - Write back a monolithic payload * @mapping: The mapping to write from * @wbc: Hints from the VM -* @iter: Data to write, must be ITER_FOLIOQ. +* @iter: Data buffer to write from, must be ITER_FOLIOQ. +* @len: Amount of data to write. * * Write a monolithic, non-pagecache object back to the server and/or * the cache. * +* The data buffer may be bigger than @len (as set in @iter's count), and if +* so, this function will use the extra as needed to provide padding into the +* cache to handle DIO rounding, so the caller should clear it. +* * Return: 0 if successful; 1 if skipped due to lock conflict and WB_SYNC_NONE; * or a negative error code. */ int netfs_writeback_single (struct address_space *mapping,struct writeback_control *wbc, -struct iov_iter *iter) +struct iov_iter *iter,size_t len) {struct netfs_io_request *wreq;struct netfs_inode *ictx =netfs_inode (mapping->host);struct folio_queue *fq; -size_t size =iov_iter_count (iter); +size_t size = len;int ret;if (WARN_ON_ONCE (!iov_iter_is_folioq (iter)))return -EIO ; +if (WARN_ON_ONCE (iov_iter_count (iter) < len)) +return -EIO ;if (!netfs_wb_begin (ictx, wbc->sync_mode ==WB_SYNC_NONE )) {/* The VFS will have undirtied the inode. */
include/linux/netfs.h +1 -1
@@ -430,7 +430,7 @@ void netfs_single_mark_inode_dirty(struct inode *inode);ssize_t netfs_read_single (struct inode *inode,struct file *file,struct iov_iter *iter);int netfs_writeback_single (struct address_space *mapping,struct writeback_control *wbc, -struct iov_iter *iter); +struct iov_iter *iter,size_t len);/* Address operations API */ struct readahead_control ;
[email protected]
AI reviewRe: [PATCH v19 08/17] netfs: Explicitly pass tail padding to netfs_writeback_single()
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] afs: uninitialized cache write or WARN in afs_symlink_writepages() --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] afs: uninitialized cache write or WARN in afs_symlink_writepages()
--
commit 95ee548dcf53267f4aa069e5911e552ec816490b
Author: David Howells <[email protected]>
netfs: Explicitly pass tail padding to netfs_writeback_single()
This commit updates netfs_writeback_single() to take an explicit content length alongside the buffer iterator and updates AFS directory and symlink writeback to clear tail padding in the buffer.
Quoted 25 lines
fs/afs/symlink.c +10
-0@@ -228,6 +228,10 @@ int afs_readlink(struct dentry *dentry, char __user *buffer, int buflen)/* * Write the symlink contents to the cache as a single blob. We then throw * away the page we used to receive it. +* +* Note that the buffer tail is cleared under a shared lock on ->validate_lock +* - though hopefully there won't be racing writers since the only +* 'modification' is to be either read from the server or locally created. */ int afs_symlink_writepages (struct address_space *mapping,struct writeback_control *wbc)@@ -247,9 +251,17 @@ int afs_symlink_writepages(struct address_space *mapping,if (vnode->directory &&atomic64_read (&vnode->cb_expires_at) !=AFS_NO_CB_PROMISE ) { +size_t i_size =i_size_read (&vnode->netfs.inode); + +if (i_size < vnode->directory_size) { +iov_iter_folio_queue (&iter,ITER_DEST , vnode->directory,0 ,0 , + vnode->directory_size); +iov_iter_advance (&iter, i_size);
fs/afs/symlink.c int afs_symlink_writepages(struct address_space *mapping, · line 267
Quoted 10 lines
fs/afs/symlink.c +4
-2+iov_iter_zero (vnode->directory_size - i_size, &iter); + }iov_iter_folio_queue (&iter,ITER_SOURCE , vnode->directory,0 ,0 , -i_size_read (&vnode->netfs.inode)); - ret =netfs_writeback_single (mapping, wbc, &iter); + vnode->directory_size); + ret =netfs_writeback_single (mapping, wbc, &iter, i_size); }if (ret ==0 ) {
[Severity: Medium]
Can afs_symlink_writepages() write uninitialized kernel memory to fscache or trigger WARN_ON_ONCE(iov_iter_count(iter) < len) in netfs_writeback_single() if the symlink's server status size grows before writeback runs?
Unlike afs_do_read_single() and afs_dir_get_block(), afs_init_new_symlink() and afs_do_read_symlink() allocate the PAGE_SIZE vnode->directory buffer without __GFP_ZERO, populate only the initial symlink length, and mark the inode dirty. If a status fetch (such as afs_getattr() or FS.InlineBulkStatus) observes a larger size on the server before writeback, afs_apply_status() updates inode->i_size and afs_apply_callback() sets a valid cb_expires_at without freeing vnode->directory (or during afs_do_read_symlink(), FS.FetchData updates i_size > PAGE_SIZE and marks the inode dirty before returning -EFBIG).
When afs_symlink_writepages() then reads the updated i_size, a new size between the populated length and PAGE_SIZE advances past the unzeroed gap and writes those uninitialized bytes to fscache. A new size above vnode->directory_size (PAGE_SIZE) skips zeroing and passes len > iov_iter_count(&iter) to netfs_writeback_single(), hitting WARN_ON_ONCE() and returning -EIO without freeing vnode->directory. Passing __GFP_ZERO when allocating the symlink buffer and invalidating or clamping against vnode->directory_size when status changes would avoid both issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
David Howells
maintainerPATCH v19 9/17 netfs: Change netfs_writeback_single() to not look inside the iterator
In reply to David HowellsChange netfs_writeback_single() so that it doesn't look around inside the data buffer iterator it is given and doesn't try to submit the folios from the list individually. Instead, it treats it as a DIO write and pushes the iterator directly, unsliced, to the cache (or the filesystem) to write from.
There are also some changes of behaviour:
(1) netfs_writeback_single() will make a single unbounded monolithic write to the cache.
(2) When writing, sreq_max_len/segs are ignored as the "single" type mandates all the data be transferred in a single op. Whilst this is not strictly necessary for the cache as the contract is really with the server, it simplifies things for now.
(3) An explicit wait is added to the function to make sure that the write is concluded before returning as there's currently no mechanism to check if the write is completed as page flags aren't used. This can be fixed later, but may require AFS's dir and symlink handling to check for write completion before allowing further changes be made.
--- fs/netfs/write_issue.c | 172 +++++++---------------------------------- 1 file changed, 28 insertions(+), 144 deletions(-)
fs/netfs/write_issue.c +28 -144
@@ -627,133 +627,15 @@ int netfs_writepages(struct address_space *mapping, }EXPORT_SYMBOL (netfs_writepages); -/* -* Write some of a pending folio data back to the server and/or the cache. -*/ -static int netfs_write_folio_single (struct netfs_io_request *wreq, -struct folio *folio) -{ -struct netfs_io_stream *upload = &wreq->io_streams[0 ]; -struct netfs_io_stream *cache = &wreq->io_streams[1 ]; -struct netfs_io_stream *stream; -size_t iter_off =0 ; -size_t fsize =folio_size (folio), flen; -uoff_t fpos =folio_pos (folio); -ssize_t ret; -bool to_eof = false; -bool no_debug = false; - -_enter ("" ); - - flen =folio_size (folio); -if (flen > wreq->i_size - fpos) { - flen = wreq->i_size - fpos; -folio_zero_segment (folio, flen, fsize); - to_eof = true; - }else if (flen == wreq->i_size - fpos) { - to_eof = true; - } - -_debug ("folio %zx/%zx" , flen, fsize); - -if (!upload->avail && !cache->avail) { -trace_netfs_folio (folio, netfs_folio_trace_cancel_store); -return 0 ; - } - -if (!upload->construct) -trace_netfs_folio (folio, netfs_folio_trace_store); -else -trace_netfs_folio (folio, netfs_folio_trace_store_plus); - -/* Attach the folio to the rolling buffer. */ -folio_get (folio); - ret =rolling_buffer_append (&wreq->buffer, folio,NETFS_ROLLBUF_PUT_MARK , wreq->gfp); -if (ret <0 ) { -folio_put (folio); -return ret; - } - -/* Move the submission point forward to allow for write-streaming data -* not starting at the front of the page. We don't do write-streaming -* with the cache as the cache requires DIO alignment. -* -* Also skip uploading for data that's been read and just needs copying -* to the cache. -*/ -for (int s =0 ; s <NR_IO_STREAMS ; s++) { - stream = &wreq->io_streams[s]; - stream->submit_off =0 ; - stream->submit_len = flen; -if (!stream->avail) { - stream->submit_off =UINT_MAX ; - stream->submit_len =0 ; - } - } - -/* Attach the folio to one or more subrequests. For a big folio, we -* could end up with thousands of subrequests if the wsize is small - -* but we might need to wait during the creation of subrequests for -* network resources (eg. SMB credits). -*/ -for (;;) { -ssize_t part; -size_t lowest_off =ULONG_MAX ; -int choose_s =-1 ; - -/* Always add to the lowest-submitted stream first. */ -for (int s =0 ; s <NR_IO_STREAMS ; s++) { - stream = &wreq->io_streams[s]; -if (stream->submit_len >0 && - stream->submit_off < lowest_off) { - lowest_off = stream->submit_off; - choose_s = s; - } - } - -if (choose_s <0 ) -break ; - stream = &wreq->io_streams[choose_s]; - -/* Advance the iterator(s). */ -if (stream->submit_off > iter_off) { -rolling_buffer_advance (&wreq->buffer, stream->submit_off - iter_off); - iter_off = stream->submit_off; - } - -atomic64_set (&wreq->issued_to, fpos + stream->submit_off); - stream->submit_extendable_to = fsize - stream->submit_off; - part =netfs_advance_write (wreq, stream, fpos + stream->submit_off, - stream->submit_len, to_eof); - stream->submit_off += part; -if (part > stream->submit_len) - stream->submit_len =0 ; -else - stream->submit_len -= part; -if (part >0 ) - no_debug = true; - } - - wreq->buffer.iter.iov_offset =0 ; -if (fsize > iter_off) -rolling_buffer_advance (&wreq->buffer, fsize - iter_off); -atomic64_set (&wreq->issued_to, fpos + fsize); - -if (!no_debug) -kdebug ("R=%x: No submit" , wreq->debug_id); -_leave (" = 0" ); -return 0 ; -} -/** -* netfs_writeback_single - Write back a monolithic payload +* netfs_writeback_single - Write back a monolithic payload to cache * @mapping: The mapping to write from * @wbc: Hints from the VM -* @iter: Data buffer to write from, must be ITER_FOLIOQ. +* @iter: Data buffer to write from. * @len: Amount of data to write. * -* Write a monolithic, non-pagecache object back to the server and/or -* the cache. +* Write a monolithic, non-pagecache object back to the cache only. The data +* is not written to the server. * * The data buffer may be bigger than @len (as set in @iter's count), and if * so, this function will use the extra as needed to provide padding into the
@@ -768,12 +650,8 @@ int netfs_writeback_single(struct address_space *mapping, {struct netfs_io_request *wreq;struct netfs_inode *ictx =netfs_inode (mapping->host); -struct folio_queue *fq; -size_t size = len;int ret; -if (WARN_ON_ONCE (!iov_iter_is_folioq (iter))) -return -EIO ;if (WARN_ON_ONCE (iov_iter_count (iter) < len))return -EIO ;
@@ -789,36 +667,42 @@ int netfs_writeback_single(struct address_space *mapping, goto couldnt_start; } + wreq->buffer.iter = *iter; + wreq->len = len; + wreq->submitted = wreq->len; +__set_bit (NETFS_RREQ_OFFLOAD_COLLECTION , &wreq->flags);trace_netfs_write (wreq, netfs_write_trace_writeback_single);netfs_stat (&netfs_n_wh_writepages); -if (__test_and_set_bit (NETFS_RREQ_UPLOAD_TO_SERVER , &wreq->flags)) - wreq->netfs_ops->begin_writeback (wreq); +for (int s =0 ; s <NR_IO_STREAMS ; s++) { +struct netfs_io_subrequest *subreq; +struct netfs_io_stream *stream = &wreq->io_streams[s]; + +if (!stream->avail) +continue ; -for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) { -for (int slot =0 ; slot <folioq_count (fq); slot++) { -struct folio *folio =folioq_folio (fq, slot); -size_t part =umin (folioq_folio_size (fq, slot), size); +netfs_prepare_write (wreq, stream,0 ); +/* Allocation not allowed to fail in writeback. */ -_debug ("wbiter %lx %llx" , folio->index,atomic64_read (&wreq->issued_to)); + subreq = stream->construct; + subreq->len = wreq->len; + stream->submit_len = subreq->len; + stream->submit_extendable_to =iov_iter_count (iter) - len; - ret =netfs_write_folio_single (wreq, folio); -if (ret <0 ) - goto stop; - size -= part; -if (size <=0 ) - goto stop; - } +netfs_issue_write (wreq, stream); } -stop: -for (int s =0 ; s <NR_IO_STREAMS ; s++) -netfs_issue_write (wreq, &wreq->io_streams[s]);netfs_all_subreqs_queued (wreq); -netfs_wake_collector (wreq); +/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to +* wait before modifying. +*/ + ret =netfs_wait_for_write (wreq); +if (ret >0 ) + ret =0 ; +netfs_put_request (wreq, netfs_rreq_trace_put_return);_leave (" = %d" , ret);return ret;
[email protected]
AI reviewRe: [PATCH v19 09/17] netfs: Change netfs_writeback_single() to not look inside the iterator
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] netfs: netfs_write_collection() wakes waiter before netfs_wb_end() --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] netfs: netfs_write_collection() wakes waiter before netfs_wb_end()
--
commit 551b21c53964c5580e9753fdbf5df8744277ee38
Author: David Howells <[email protected]>
netfs: Change netfs_writeback_single() to not look inside the iterator
This commit changes netfs_writeback_single() to submit the entire buffer iterator as a single unsliced subrequest per available stream and wait synchronously for the write to complete before returning.
Quoted 4 lines
fs/netfs/write_issue.c +0
-0
[ ... ]
Quoted 47 lines
fs/netfs/write_issue.c +17
-17@@ -789,36 +667,42 @@ int netfs_writeback_single(struct address_space *mapping, goto couldnt_start; } + wreq->buffer.iter = *iter; + wreq->len = len; + wreq->submitted = wreq->len; +__set_bit (NETFS_RREQ_OFFLOAD_COLLECTION , &wreq->flags);trace_netfs_write (wreq, netfs_write_trace_writeback_single);netfs_stat (&netfs_n_wh_writepages); -if (__test_and_set_bit (NETFS_RREQ_UPLOAD_TO_SERVER , &wreq->flags)) - wreq->netfs_ops->begin_writeback (wreq); +for (int s =0 ; s <NR_IO_STREAMS ; s++) { +struct netfs_io_subrequest *subreq; +struct netfs_io_stream *stream = &wreq->io_streams[s]; + +if (!stream->avail) +continue ; -for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) { -for (int slot =0 ; slot <folioq_count (fq); slot++) { -struct folio *folio =folioq_folio (fq, slot); -size_t part =umin (folioq_folio_size (fq, slot), size); +netfs_prepare_write (wreq, stream,0 ); +/* Allocation not allowed to fail in writeback. */ -_debug ("wbiter %lx %llx" , folio->index,atomic64_read (&wreq->issued_to)); + subreq = stream->construct; + subreq->len = wreq->len; + stream->submit_len = subreq->len; + stream->submit_extendable_to =iov_iter_count (iter) - len; - ret =netfs_write_folio_single (wreq, folio); -if (ret <0 ) - goto stop; - size -= part; -if (size <=0 ) - goto stop; - } +netfs_issue_write (wreq, stream); } -stop: -for (int s =0 ; s <NR_IO_STREAMS ; s++) -netfs_issue_write (wreq, &wreq->io_streams[s]);
fs/netfs/write_issue.c int netfs_writeback_single(struct address_space *mapping, · line 704
Quoted 10 lines
fs/netfs/write_issue.c +6
-1netfs_all_subreqs_queued (wreq); -netfs_wake_collector (wreq); +/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to +* wait before modifying. +*/ + ret =netfs_wait_for_write (wreq); +if (ret >0 ) + ret =0 ;
[Severity: Medium]
Can netfs_writeback_single() return while NETFS_ICTX_WB_LOCK is still held by the collector worker?
Because NETFS_RREQ_OFFLOAD_COLLECTION is set, netfs_wait_for_write() waits for the workqueue thread running netfs_write_collection() to clear NETFS_RREQ_IN_PROGRESS, which happens before netfs_wb_end() releases NETFS_ICTX_WB_LOCK:
CPU0 CPU1 (kworker)netfs_writeback_single()
netfs_wb_begin(ictx, ...)
netfs_wake_collector(wreq)
netfs_wait_for_write(wreq)
netfs_write_collection()
netfs_wake_rreq_flag(wreq,
NETFS_RREQ_IN_PROGRESS, ...)
/* returns to caller */
netfs_wb_begin(ictx, true)
/* sees NETFS_ICTX_WB_LOCK set */ <-- spurious lock conflict
netfs_wb_end(ictx)An immediate subsequent WB_SYNC_NONE writeback on the same inode (such as afs_dir_writepages() after releasing validate_lock) can see NETFS_ICTX_WB_LOCK still set, re-mark the inode dirty via netfs_single_mark_inode_dirty(), and skip the writeback pass.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
David Howells
maintainerPATCH v19 10/17 netfs: Pass the write length to request creation where appropriate
In reply to David HowellsFor unbuffered/DIO writes and writes to single-type objects, pass the length down into the request creation function so that wreq->len gets set correctly. For buffered writeback and pgpriv2 copying, this remains 0 as we don't know in advance how big the request will be.
A subsequent patch will use this to set as the i_size on single-type objects (e.g. afs directories) as we don't care what the old i_size was and we don't want to see inode->i_size in the netfs file changing due to third-party invalidation before we update fscache to say how big the object is going to be.
--- fs/netfs/direct_write.c | 4 ++-- fs/netfs/internal.h | 2 +- fs/netfs/read_pgpriv2.c | 2 +- fs/netfs/write_issue.c | 9 +++++---- 4 files changed, 9 insertions(+), 8 deletions(-)
fs/netfs/direct_write.c +2 -2
@@ -222,7 +222,7 @@ static void netfs_unbuffered_write_async(struct work_struct *work)* encrypted file. This can also be used for direct I/O writes. */ ssize_t netfs_unbuffered_write_iter_locked (struct kiocb *iocb,struct iov_iter *iter, -struct netfs_group *netfs_group) +struct netfs_group *netfs_group) {struct netfs_io_request *wreq;ssize_t ret, n;
@@ -241,7 +241,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *_debug ("uw %llx-%llx" , start, end); - wreq =netfs_create_write_req (iocb->ki_filp->f_mapping, iocb->ki_filp, start, + wreq =netfs_create_write_req (iocb->ki_filp->f_mapping, iocb->ki_filp, start, len, iocb->ki_flags &IOCB_DIRECT ?NETFS_DIO_WRITE :NETFS_UNBUFFERED_WRITE );if (IS_ERR (wreq))
fs/netfs/internal.h +1 -1
@@ -240,7 +240,7 @@ void netfs_write_collection_worker(struct work_struct *work);*/ struct netfs_io_request *netfs_create_write_req (struct address_space *mapping,struct file *file, -uoff_t start, +uoff_t start,size_t len,enum netfs_io_origin origin); void netfs_prepare_write(struct netfs_io_request *wreq, struct netfs_io_stream *stream,
fs/netfs/read_pgpriv2.c +1 -1
@@ -103,7 +103,7 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(if (!fscache_resources_valid (&rreq->cache_resources)) goto cancel; - creq =netfs_create_write_req (rreq->mapping,NULL ,folio_pos (folio), + creq =netfs_create_write_req (rreq->mapping,NULL ,folio_pos (folio),0 ,NETFS_PGPRIV2_COPY_TO_CACHE );if (IS_ERR (creq)) goto cancel;
fs/netfs/write_issue.c +5 -4
@@ -89,7 +89,7 @@ static void netfs_kill_dirty_pages(struct address_space *mapping,*/ struct netfs_io_request *netfs_create_write_req (struct address_space *mapping,struct file *file, -uoff_t start, +uoff_t start,size_t len,enum netfs_io_origin origin) {struct netfs_io_request *wreq;
@@ -98,7 +98,7 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping, origin ==NETFS_WRITEBACK_SINGLE || origin == NETFS_PGPRIV2_COPY_TO_CACHE); - wreq =netfs_alloc_request (mapping, file, start,0 , origin); + wreq =netfs_alloc_request (mapping, file, start, len, origin);if (IS_ERR (wreq))return wreq;
@@ -575,7 +575,8 @@ int netfs_writepages(struct address_space *mapping,if (!folio) goto out; - wreq =netfs_create_write_req (mapping,NULL ,folio_pos (folio),NETFS_WRITEBACK ); + wreq =netfs_create_write_req (mapping,NULL ,folio_pos (folio),0 , +NETFS_WRITEBACK );if (IS_ERR (wreq)) { error =PTR_ERR (wreq); goto couldnt_start;
@@ -661,7 +662,7 @@ int netfs_writeback_single(struct address_space *mapping,return 1 ; } - wreq =netfs_create_write_req (mapping,NULL ,0 ,NETFS_WRITEBACK_SINGLE ); + wreq =netfs_create_write_req (mapping,NULL ,0 , len,NETFS_WRITEBACK_SINGLE );if (IS_ERR (wreq)) { ret =PTR_ERR (wreq); goto couldnt_start;
[email protected]
AI reviewRe: [PATCH v19 10/17] netfs: Pass the write length to request creation where appropriate
Sashiko has reviewed this patch and found no issues.
Quoted 5 lines
For unbuffered/DIO writes and writes to single-type objects, pass the length down into the request creation function so that
wreq->lengets set correctly. For buffered writeback and pgpriv2 copying, this remains 0 as we don't know in advance how big the request will be.
Quoted 7 lines
A subsequent patch will use this to set as the
i_sizeon single-type objects (e.g. afs directories) as we don't care what the oldi_sizewas and we don't want to seeinode->i_sizein the netfs file changing due to third-party invalidation before we update fscache to say how big the object is going to be.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
David Howells
maintainerPATCH v19 11/17 afs: Use a bvecq to hold dir content rather than folioq
In reply to David HowellsUse a bvecq to hold the contents of a directory rather than the folioq so that the latter can be phased out. This is also applied to the symlink I/O buffer.
Also use the bvecq chain being refcounted to fix an issue in afs directory handling and netfs_writeback_single() whereby the application of third-party induced invalidation can race with the writeback code because afs doesn't hold the vnode validate_lock when updating the inode info.
Now, AFS FS RPC calls typically return a status update any time they access a vnode (file) on the server and the Linux afs fs driver applies it directly in the postprocessing of the RPC and, ideally, it would take the vnode's validate_lock whilst doing so - unfortunately, however, the caller may already be holding the validate_lock, as does afs_dir_writepages().
Worse, afs_dir_writepages() takes a shared lock so the inode info can't be updated exclusively under that lock.
Fix this by getting the directory contents, dir buffer size and inode size inside the cb_lock so that the set is at least consistent (if not currently valid). Invalidation will replace the cachefile anyway to discard in-progress old writebacks.
Further, in netfs_alloc_request() called from netfs_writeback_single(), use the content length parameter rather than i_size (we don't care about the cachefile size) on behalf of single-writeback.
--- fs/afs/dir.c | 59 ++++++++++++++++++++++++-------------------- fs/afs/dir_edit.c | 44 +++++++++++++++------------------ fs/afs/dir_search.c | 35 ++++++++++++-------------- fs/afs/inode.c | 2 +- fs/afs/internal.h | 6 ++--- fs/afs/symlink.c | 60 ++++++++++++++++++++++++++++----------------- fs/netfs/objects.c | 4 +++ 7 files changed, 114 insertions(+), 96 deletions(-)
fs/afs/dir.c +33 -26
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)pr_warn ("DIR %llx:%llx is=%llx\n" , dvnode->fid.vid, dvnode->fid.vnode, i_size); -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size); -iterate_folioq (&iter,iov_iter_count (&iter),NULL ,NULL , - afs_dir_dump_step); +iov_iter_bvec_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size); +iterate_bvecq (&iter,iov_iter_count (&iter),NULL ,NULL , + afs_dir_dump_step); }/*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)if (unlikely (!i_size))return 0 ; -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size); - checked =iterate_folioq (&iter,iov_iter_count (&iter), dvnode,NULL , - afs_dir_check_step); +iov_iter_bvec_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size); + checked =iterate_bvecq (&iter,iov_iter_count (&iter), dvnode,NULL , + afs_dir_check_step);if (checked != i_size) {afs_dir_dump (dvnode);return -EIO ;
@@ -250,16 +250,16 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)if (dvnode->directory_size < i_size) {size_t cur_size = dvnode->directory_size; - ret =netfs_alloc_folioq_buffer (NULL , - &dvnode->directory, &cur_size, i_size, -mapping_gfp_mask (dvnode->netfs.inode.i_mapping) | - __GFP_ZERO); + ret =bvecq_expand_buffer (&dvnode->directory, &cur_size, +round_up (i_size,PAGE_SIZE ), +mapping_gfp_mask (dvnode->netfs.inode.i_mapping) | + __GFP_ZERO); dvnode->directory_size = cur_size;if (ret <0 )return ret; } -iov_iter_folio_queue (&iter,ITER_DEST , dvnode->directory,0 ,0 , dvnode->directory_size); +iov_iter_bvec_queue (&iter,ITER_DEST , dvnode->directory,0 ,0 , dvnode->directory_size);/* AFS requires us to perform the read of a directory synchronously as * a single unit to avoid issues with the directory contents being
@@ -281,8 +281,8 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file) }if (ret >=0 && ret < dvnode->directory_size) { -iov_iter_folio_queue (&iter,ITER_DEST , dvnode->directory, -0 ,0 , dvnode->directory_size); +iov_iter_bvec_queue (&iter,ITER_DEST , dvnode->directory, +0 ,0 , dvnode->directory_size);iov_iter_advance (&iter, ret);iov_iter_zero (dvnode->directory_size - ret, &iter); }
@@ -302,8 +302,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file) }/* -* Read the directory into a folio_queue buffer in one go, scrubbing the -* previous contents. We return -ESTALE if the caller needs to call us again. +* Read the directory into the buffer in one go, scrubbing the previous +* contents. We return -ESTALE if the caller needs to call us again. */ ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file) __acquires(&dvnode->validate_lock)
@@ -491,7 +491,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len, }/* -* Iterate through the directory folios. +* Iterate through the directory content. */ static int afs_dir_iterate_contents (struct inode *dir,struct dir_context *dir_ctx) {
@@ -506,11 +506,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_cif (i_size <=0 || dir_ctx->pos >= i_size)return 0 ; -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size); +iov_iter_bvec_queue (&iter,ITER_SOURCE , dvnode->directory,0 ,0 , i_size);iov_iter_advance (&iter,round_down (dir_ctx->pos,AFS_DIR_BLOCK_SIZE )); -iterate_folioq (&iter,iov_iter_count (&iter), dvnode, &ctx, - afs_dir_iterate_step); +iterate_bvecq (&iter,iov_iter_count (&iter), dvnode, &ctx, + afs_dir_iterate_step);if (ctx.error == -ESTALE )afs_invalidate_dir (dvnode, afs_dir_invalid_iter_stale);
@@ -2220,12 +2220,11 @@ static int afs_dir_writepages(struct address_space *mapping, struct writeback_control *wbc) {struct afs_vnode *dvnode =AFS_FS_I (mapping->host); +struct bvecq *directory =NULL ;struct iov_iter iter;int ret =0 ; -/* Need to lock to prevent the folio queue and folios from being thrown -* away. -*/ +/* Need to lock to prevent the bvecq and folios from being discarded. */ if (!down_read_trylock (&dvnode->validate_lock)) {if (wbc->sync_mode ==WB_SYNC_NONE ) {/* The VFS will have undirtied the inode. */
@@ -2235,15 +2234,23 @@ static int afs_dir_writepages(struct address_space *mapping,down_read (&dvnode->validate_lock); } +read_seqlock_excl (&dvnode->cb_lock);if (test_bit (AFS_VNODE_DIR_VALID , &dvnode->flags)) { -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory, -0 ,0 , dvnode->directory_size); - ret =netfs_writeback_single (mapping, wbc, &iter, -i_size_read (&dvnode->netfs.inode)); +size_t i_size =i_size_read (&dvnode->netfs.inode); +size_t d_size = dvnode->directory_size; + + directory =bvecq_get (dvnode->directory); +read_sequnlock_excl (&dvnode->cb_lock); + +iov_iter_bvec_queue (&iter,ITER_SOURCE , directory,0 ,0 , d_size); + ret =netfs_writeback_single (mapping, wbc, &iter, i_size);if (ret ==1 ) ret =0 ;/* Skipped write due to lock conflict. */ + }else { +read_sequnlock_excl (&dvnode->cb_lock); }up_read (&dvnode->validate_lock); +bvecq_put (directory);return ret; }
fs/afs/dir_edit.c +20 -24
77 lines not on this page · raw message
fs/afs/dir_search.c +16 -19
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)*/ union afs_xdr_dir_block *afs_dir_find_block (struct afs_dir_iter *iter,size_t block) { -struct folio_queue *fq = iter->fq;struct afs_vnode *dvnode = iter->dvnode; -struct folio *folio; +struct bvecq *bq = iter->bq;size_t blpos = block *AFS_DIR_BLOCK_SIZE ;size_t blend = (block +1 ) *AFS_DIR_BLOCK_SIZE , fpos = iter->fpos; -int slot = iter->fq_slot; +int slot = iter->bq_slot;_enter ("%zx,%d" , block, slot);
@@ -80,36 +79,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t blif (dvnode->directory_size < blend) goto fail; -if (!fq || blpos < fpos) { - fq = dvnode->directory; +if (!bq || blpos < fpos) { + bq = dvnode->directory; slot =0 ; fpos =0 ; } -/* Search the folio queue for the folio containing the block... */ -for (; fq; fq = fq->next) { -for (; slot <folioq_count (fq); slot++) { -size_t fsize =folioq_folio_size (fq, slot); +/* Search the bvecq for the folio containing the block... */ +for (; bq; bq = bq->next) { +for (; slot < bq->nr_slots; slot++) { +struct bio_vec *bv = &bq->bv[slot]; +size_t bsize = bv->bv_len; -if (blend <= fpos + fsize) { +if (blend <= fpos + bsize) {/* ... and then return the mapped block. */ - folio =folioq_folio (fq, slot); -if (WARN_ON_ONCE (folio_pos (folio) != fpos)) - goto fail; - iter->fq = fq; - iter->fq_slot = slot; + iter->bq = bq; + iter->bq_slot = slot; iter->fpos = fpos; - iter->block =kmap_local_folio (folio, blpos - fpos); + iter->block =bvec_kmap_partial (bv, blpos - fpos);return iter->block; } - fpos += fsize; + fpos += bsize; } slot =0 ; } fail: - iter->fq =NULL ; - iter->fq_slot =0 ; + iter->bq =NULL ; + iter->bq_slot =0 ;afs_invalidate_dir (dvnode, afs_dir_invalid_edit_get_block);return NULL ; }
fs/afs/inode.c +1 -1
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)flush_delayed_work (&vnode->lock_work);netfs_wait_for_outstanding_io (inode);truncate_inode_pages_final (&inode->i_data); -netfs_free_folioq_buffer (vnode->directory); +bvecq_put (vnode->directory);if (vnode->symlink)afs_evict_symlink (vnode);
fs/afs/internal.h +3 -3
@@ -712,7 +712,7 @@ struct afs_vnode {#define AFS_VNODE_MODIFYING 10/* Set if we're performing a modification op */ #define AFS_VNODE_DIR_READ 11/* Set if we've read a dir's contents */ -struct folio_queue *directory;/* Directory contents */ +struct bvecq *directory;/* Directory contents */ struct afs_symlink __rcu *symlink;/* Symlink content */ struct list_head wb_keys;/* List of keys available for writeback */ struct list_head pending_locks;/* locks waiting to be granted */
@@ -993,9 +993,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int flstruct afs_dir_iter {struct afs_vnode *dvnode;union afs_xdr_dir_block *block; -struct folio_queue *fq; +struct bvecq *bq;unsigned int fpos; -int fq_slot; +int bq_slot;unsigned int loop_check;u8 nr_slots;u8 bucket;
fs/afs/symlink.c +37 -23
127 lines not on this page · raw message
fs/netfs/objects.c +4 -0
@@ -78,6 +78,10 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping, rreq->io_streams[0 ].avail = true; }else {INIT_WORK (&rreq->work, netfs_write_collection_worker); + +/* For single-type, we disregard the previous content. */ +if (origin ==NETFS_WRITEBACK_SINGLE ) + rreq->i_size = rreq->len; }__set_bit (NETFS_RREQ_IN_PROGRESS , &rreq->flags);
[email protected]
AI reviewRe: [PATCH v19 11/17] afs: Use a bvecq to hold dir content rather than folioq
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [High] afs: inconsistent i_size and uninitialized buffer in afs_symlink_writepages() - [Medium] afs…
Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [High] afs: inconsistent i_size and uninitialized buffer in afs_symlink_writepages()
- [Medium] afs: inconsistent directory state and size in afs_dir_writepages()
--
commit 38825177e122ffa7a8e67a676f8d6170b75ad709
Author: David Howells <[email protected]>
afs: Use a bvecq to hold dir content rather than folioq
This commit replaces folio_queue buffers with bvecq buffers for AFS directories and symlinks, and snapshots the buffer, buffer size, and inode size under cb_lock during writeback.
Quoted 4 lines
fs/afs/dir.c +0
-0
[ ... ]
Quoted 28 lines
fs/afs/dir.c +6
-7@@ -2220,12 +2220,11 @@ static int afs_dir_writepages(struct address_space *mapping, struct writeback_control *wbc) {struct afs_vnode *dvnode =AFS_FS_I (mapping->host); +struct bvecq *directory =NULL ;struct iov_iter iter;int ret =0 ; -/* Need to lock to prevent the folio queue and folios from being thrown -* away. -*/ +/* Need to lock to prevent the bvecq and folios from being discarded. */ if (!down_read_trylock (&dvnode->validate_lock)) {if (wbc->sync_mode ==WB_SYNC_NONE ) {/* The VFS will have undirtied the inode. */ @@ -2235,15 +2234,23 @@ static int afs_dir_writepages(struct address_space *mapping,down_read (&dvnode->validate_lock); } +read_seqlock_excl (&dvnode->cb_lock);if (test_bit (AFS_VNODE_DIR_VALID , &dvnode->flags)) { -iov_iter_folio_queue (&iter,ITER_SOURCE , dvnode->directory, -0 ,0 , dvnode->directory_size); - ret =netfs_writeback_single (mapping, wbc, &iter, -i_size_read (&dvnode->netfs.inode)); +size_t i_size =i_size_read (&dvnode->netfs.inode); +size_t d_size = dvnode->directory_size; +
fs/afs/dir.c static int afs_dir_writepages(struct address_space *mapping, · line 2251
Quoted 10 lines
fs/afs/dir.c +7
-0+ directory =bvecq_get (dvnode->directory); +read_sequnlock_excl (&dvnode->cb_lock); + +iov_iter_bvec_queue (&iter,ITER_SOURCE , directory,0 ,0 , d_size); + ret =netfs_writeback_single (mapping, wbc, &iter, i_size);if (ret ==1 ) ret =0 ;/* Skipped write due to lock conflict. */ + }else { +read_sequnlock_excl (&dvnode->cb_lock); }
[Severity: Medium]
Can afs_dir_writepages() still observe i_size > d_size or write an unvalidated directory buffer to fscache?
In afs_read_dir(), set_bit(AFS_VNODE_DIR_VALID, &dvnode->flags) runs outside cb_lock after afs_read_single() has dropped AFS_VNODE_IO_LOCK:
CPU0 CPU1afs_read_dir()
afs_read_single()
... (allocates d_size)
afs_do_lookup() (FS.InlineBulkStatus)
afs_vnode_commit_status()
write_seqlock(&dvnode->cb_lock)
afs_apply_status()
afs_invalidate_dir()
i_size_write(..., new_size)
write_sequnlock(&dvnode->cb_lock)
set_bit(AFS_VNODE_DIR_VALID)
downgrade_write(&validate_lock)
afs_dir_writepages()
netfs_writeback_single(..., i_size)
<-- WARN_ON_ONCE(d_size < i_size)Also, new directory vnodes start with AFS_VNODE_DIR_VALID set and AFS_VNODE_DIR_READ clear. If afs_do_read_single() fails in afs_dir_check() after netfs_read_single() has already dirtied the inode, afs_read_dir() unlocks validate_lock without clearing AFS_VNODE_DIR_VALID, and afs_dir_writepages() (which does not check AFS_VNODE_DIR_READ) writes the corrupt buffer into fscache.
[ ... ]
Quoted 33 lines
fs/afs/symlink.c +8
-5@@ -56,7 +56,7 @@ void afs_evict_symlink(struct afs_vnode *vnode)void afs_init_new_symlink (struct afs_vnode *vnode,struct afs_operation *op) {struct afs_symlink *symlink = op->create.symlink; -size_t dsize =0 ; +unsigned int alloc_flags =0 ;size_t size =strlen (symlink->content) +1 ;char *p;@@ -66,12 +66,15 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)if (!fscache_cookie_enabled (netfs_i_cookie (&vnode->netfs)))return ; -if (netfs_alloc_folioq_buffer (NULL , &vnode->directory, &dsize, size, -mapping_gfp_mask (vnode->netfs.inode.i_mapping)) <0 ) + vnode->directory = +bvecq_alloc_buffer (PAGE_SIZE , +mapping_gfp_mask (vnode->netfs.inode.i_mapping), + &alloc_flags); +if (!vnode->directory)return ; - vnode->directory_size = dsize; - p =kmap_local_folio (folioq_folio (vnode->directory,0 ),0 ); + vnode->directory_size =PAGE_SIZE ; + p =bvec_kmap_partial (&vnode->directory->bv[0 ],0 );memcpy (p, symlink->content, size);kunmap_local (p);netfs_single_mark_inode_dirty (&vnode->netfs.inode);
[ ... ]
Quoted 21 lines
fs/afs/symlink.c +8
-4@@ -249,27 +251,39 @@ int afs_symlink_writepages(struct address_space *mapping,down_read (&vnode->validate_lock); } +read_seqlock_excl (&vnode->cb_lock);if (vnode->directory &&atomic64_read (&vnode->cb_expires_at) !=AFS_NO_CB_PROMISE ) { +struct bvecq *bq =bvecq_get (vnode->directory);size_t i_size =i_size_read (&vnode->netfs.inode); +size_t d_size = vnode->directory_size; -if (i_size < vnode->directory_size) { -iov_iter_folio_queue (&iter,ITER_DEST , vnode->directory,0 ,0 , - vnode->directory_size); +read_sequnlock_excl (&vnode->cb_lock); + +if (i_size < d_size) { +iov_iter_bvec_queue (&iter,ITER_DEST , bq,0 ,0 , d_size);iov_iter_advance (&iter, i_size); -iov_iter_zero (vnode->directory_size - i_size, &iter); +iov_iter_zero (d_size - i_size, &iter);
fs/afs/symlink.c int afs_symlink_writepages(struct address_space *mapping, · line 274
Quoted 10 lines
fs/afs/symlink.c +5
-2} -iov_iter_folio_queue (&iter,ITER_SOURCE , vnode->directory,0 ,0 , - vnode->directory_size); + +iov_iter_bvec_queue (&iter,ITER_SOURCE , bq,0 ,0 , d_size); ret =netfs_writeback_single (mapping, wbc, &iter, i_size); +bvecq_put (bq); + }else { +read_sequnlock_excl (&vnode->cb_lock); }
[Severity: High]
Can afs_symlink_writepages() write uninitialized page memory to fscache or pass i_size > d_size to netfs_writeback_single()?
Both afs_init_new_symlink() and afs_do_read_symlink() allocate vnode->directory via bvecq_alloc_buffer() without __GFP_ZERO and only populate the initial symlink length. Unlike directories (which clear AFS_VNODE_DIR_VALID in afs_apply_status()), when a status update arrives for a symlink with a new data version, afs_vnode_commit_status() sets AFS_VNODE_ZAP_DATA, updates inode->i_size, and installs a new valid cb_expires_at != AFS_NO_CB_PROMISE:
fs/afs/inode.c:afs_apply_status() {
...
if (vnode->status.type == AFS_FTYPE_DIR)
afs_invalidate_dir(vnode, afs_dir_invalid_dv_mismatch);
else
set_bit(AFS_VNODE_ZAP_DATA, &vnode->flags);
change_size = true;
...
}Because afs_symlink_writepages() does not check AFS_VNODE_ZAP_DATA (and afs_invalidate_symlink() only clears vnode->symlink, not vnode->directory), an updated i_size larger than the old symlink length causes afs_symlink_writepages() to zero only from the new i_size to d_size, writing stale content plus uninitialized kernel memory in [old_size, i_size) into fscache. If the server reports i_size > PAGE_SIZE (including when afs_do_read_symlink() returns -EFBIG after netfs_read_single() has updated i_size and dirtied the inode), i_size > d_size triggers WARN_ON_ONCE(iov_iter_count(iter) < len) in netfs_writeback_single().
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
David Howells
maintainerPATCH v19 12/17 cifs: Use a bvecq for buffering instead of a folioq
In reply to David HowellsUse a bvecq for internal buffering for crypto purposes instead of a folioq so that the latter can be phased out.
--- fs/smb/client/cifsglob.h | 2 +- fs/smb/client/smb2ops.c | 97 ++++++++++++++++++++++------------------ 2 files changed, 54 insertions(+), 45 deletions(-)
fs/smb/client/cifsglob.h +1 -1
@@ -290,7 +290,7 @@ struct smb_rqst {struct kvec *rq_iov;/* array of kvecs */ unsigned int rq_nvec;/* number of kvecs in array */ struct iov_iter rq_iter;/* Data iterator */ -struct folio_queue *rq_buffer;/* Buffer for encryption */ +struct bvecq *rq_buffer;/* Buffer for encryption */ };struct mid_q_entry ;
fs/smb/client/smb2ops.c +53 -44
211 lines not on this page · raw message
[email protected]
AI reviewRe: [PATCH v19 12/17] cifs: Use a bvecq for buffering instead of a folioq
Sashiko has reviewed this patch and found no issues.
Quoted 4 lines
Use a bvecq for internal buffering for crypto purposes instead of a folioq so that the latter can be phased out.
Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
David Howells
maintainerPATCH v19 13/17 smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
In reply to David HowellsAdd support for ITER_BVECQ to smbdirect_map_sges_from_iter().
--- fs/smb/smbdirect/connection.c | 70 +++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+)
fs/smb/smbdirect/connection.c +70 -0
@@ -5,6 +5,7 @@*/ #include "internal.h" +#include <linux/bvecq.h> #include <linux/folio_queue.h> struct smbdirect_map_sges {
@@ -2015,6 +2016,72 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,return ret; } +/* +* Extract memory fragments from a BVECQ-class iterator and add them to an RDMA +* list. The fragments are not pinned. +*/ +static ssize_t smbdirect_map_sges_from_bvecq (struct iov_iter *iter, +struct smbdirect_map_sges *state, +ssize_t maxsize) +{ +const struct bvecq *bq = iter->bvecq, *next; +unsigned int slot = iter->bvecq_slot; +ssize_t extracted =0 ; +size_t offset = iter->iov_offset; + + maxsize =umin (maxsize,iov_iter_count (iter)); + +for (;;) { +struct bio_vec *bv; +size_t bsize; + +while (slot >= bq->nr_slots) { + next =bvecq_next (bq); +if (!next) { +if (WARN_ON_ONCE (maxsize >0 )) +return -EIO ; + goto out; + } + bq = next; + slot =0 ; + } + +if (state->num_sge >= state->max_sge || maxsize <=0 ) +break ; + + bv = &bq->bv[slot]; + bsize = bv->bv_len; + +if (offset < bsize) { +size_t part =umin (maxsize, bsize - offset); +bool ok; + + ok =smbdirect_map_sges_single_page (state, + bv->bv_page, + bv->bv_offset + offset, + part); +if (!ok) +return -EIO ; + + offset += part; + extracted += part; + maxsize -= part; + } + +if (offset >= bsize) { + offset =0 ; + slot++; + } + } + +out: + iter->bvecq = bq; + iter->bvecq_slot = slot; + iter->iov_offset = offset; + iter->count -= extracted; +return extracted; +} +/* * Extract fragments from a KVEC-class iterator and add them to an ib_sge list. * This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2164,6 +2231,9 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,case ITER_BVEC : ret =smbdirect_map_sges_from_bvec (iter, state, len);break ; +case ITER_BVECQ : + ret =smbdirect_map_sges_from_bvecq (iter, state, len); +break ;case ITER_KVEC : ret =smbdirect_map_sges_from_kvec (iter, state, len);break ;
[email protected]
AI reviewRe: [PATCH v19 13/17] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
Sashiko has reviewed this patch and found no issues.
Quoted 3 lines
Add support for
ITER_BVECQtosmbdirect_map_sges_from_iter().Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
David Howells
maintainerPATCH v19 14/17 netfs: Switch folioq to bvecq
In reply to David HowellsIn netfslib, perform more or less a straight switch from using folio_queue to using bvecq to hold the lists of folios involved in various sorts of buffered read and in buffered writeback.
It's not quite a straight swap, however, because:
(1) The bvecq_alloc_*() routines want to know if the caller is doing writeback for emergency pool use.
(2) folio_queue includes a fixed capacity folio_batch, but bvecq has a variable size bio_vec array.
(3) folio_queue has a set of per-folio marks that are mostly unused; the one exception is for netfs_prefetch_for_write() - but the marked folio is then ignored because no_unlock_folio points to it (and the bvecq mem cleanup type can handle that anyway). This will become an issue if/when netfs_prefetch_for_write() expands the read to cope with large cache granularity.
(4) The bvecq API doesn't have functions to get a folio's length or to clear a folio pointer by slot, instead accessing the bio_vec array directly. Also bvecq stores the folio length in bv_len instead of storing the folio orders.
(5) rolling_buffer_bulk_load_from_ra() needs to work a bit differently as the folio_queue contains a folio_batch and bvecq doesn't.
(6) rolling_buffer_delete_spent() has to get a ref on the next link before putting the bvecq to avoid rolling up the entire list. On the other hand, rolling_buffer_clear() just needs to put the tail.
(7) The bvecq chain needs to be pinned by the subrequests to prevent the retry reverting the iterator and hitting a spent link in the chain that has been removed. (Note that the intention is to remove the iterator from the netfs_io_subrequest struct and push the use of iterators down into the filesystem in a later patchset).
--- fs/netfs/buffered_read.c | 48 ++++++----- fs/netfs/iterator.c | 42 ++++----- fs/netfs/objects.c | 1 + fs/netfs/read_collect.c | 48 +++++------ fs/netfs/read_pgpriv2.c | 43 +++++++--- fs/netfs/read_retry.c | 53 ++++++++---- fs/netfs/rolling_buffer.c | 151 ++++++++++++++------------------- fs/netfs/write_collect.c | 44 +++++++--- fs/netfs/write_issue.c | 40 +++++++-- fs/netfs/write_retry.c | 15 +++- fs/smb/client/smb2ops.c | 5 ++ include/linux/netfs.h | 1 + include/linux/rolling_buffer.h | 42 +++------ include/trace/events/netfs.h | 2 +- 14 files changed, 306 insertions(+), 229 deletions(-)
fs/netfs/buffered_read.c +27 -21
@@ -131,6 +131,8 @@ static ssize_t netfs_prepare_read_iterator(struct netfs_io_subrequest *subreq) } subreq->io_iter = rreq->buffer.iter; +if (iov_iter_is_bvecq (&subreq->io_iter)) + subreq->bvecq_pin =bvecq_get ((struct bvecq *)subreq->io_iter.bvecq);iov_iter_truncate (&subreq->io_iter, subreq->len);rolling_buffer_advance (&rreq->buffer, subreq->len);
@@ -215,9 +217,9 @@ static void netfs_issue_read(struct netfs_io_request *rreq,* otherwise we set the deprecated PG_private_2. */ static void netfs_mark_copy_to_cache (struct netfs_io_request *rreq, -struct folio_queue **fq, -unsigned int *offset, +struct bvecq **bq,int *slot, +unsigned int *offset,size_t len,bool copy) {
@@ -225,21 +227,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,struct folio *folio;size_t fsize, overlap; -if (!*fq) +if (!*bq)break ; -if (*slot >=folioq_count (*fq)) { - *fq = (*fq)->next; +if (!bvecq_acquire_slot (*bq, *slot)) { + *bq =bvecq_next (*bq); *slot =0 ; *offset =0 ;continue ; }/* Determine how much the subreq overlaps the folio, if at all. */ - fsize =folioq_folio_size (*fq, *slot); + fsize = (*bq)->bv[*slot].bv_len; overlap =min (len, fsize - *offset);if (overlap >0 && copy) { - folio =folioq_folio (*fq, *slot); + folio =bvec_folio (&(*bq)->bv[*slot]);if (netfs_using_pgpriv2 (rreq)) {if (!folio_test_private_2 (folio))folio_start_private_2 (folio);
@@ -275,11 +277,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq) .cached_to[1] = ULLONG_MAX, };struct fscache_occupancy *occ = &_occ; -struct folio_queue *fq = rreq->buffer.tail; -unsigned int offset =0 ; +struct bvecq *mark_bq = rreq->buffer.tail, *mark_pin; +unsigned int mark_offset =0 ;ssize_t size = rreq->len;uoff_t start = rreq->start; -int ret =0 , slot =0 ; +int ret =0 , mark_slot =0 ; + +/* Pin the bvecq whilst we mark folios for copying to the cache. */ + mark_pin =bvecq_get (mark_bq); do { int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
@@ -408,10 +413,11 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)if (size <=0 )netfs_all_subreqs_queued (rreq); -if (fq) { +if (mark_bq) {/* See if the cache indicated this should be cached. */ copy =test_bit (NETFS_SREQ_COPY_TO_CACHE , &subreq->flags); -netfs_mark_copy_to_cache (rreq, &fq, &slot, &offset, slice, copy); +netfs_mark_copy_to_cache (rreq, &mark_bq, &mark_slot, + &mark_offset, slice, copy); }trace_netfs_sreq (subreq, netfs_sreq_trace_submit);
@@ -430,6 +436,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)netfs_wake_collector (rreq); } +bvecq_put (mark_pin); +/* Defer error return as we may need to wait for outstanding I/O. */ cmpxchg (&rreq->error,0 , ret); }
@@ -479,8 +487,7 @@ void netfs_readahead(struct readahead_control *ractl)* acquires a ref on each folio that we will need to release later - * but we don't want to do that until after we've started the I/O. */ - added =rolling_buffer_bulk_load_from_ra (&rreq->buffer, ractl, - rreq->debug_id, rreq->gfp); + added =rolling_buffer_bulk_load_from_ra (&rreq->buffer, ractl, rreq->gfp);if (added <0 ) { ret = added; goto cleanup_free;
@@ -503,15 +510,14 @@ EXPORT_SYMBOL(netfs_readahead);/* * Create a rolling buffer with a single occupying folio. */ -static int netfs_create_singular_buffer (struct netfs_io_request *rreq,struct folio *folio, -unsigned int rollbuf_flags) +static int netfs_create_singular_buffer (struct netfs_io_request *rreq,struct folio *folio) {ssize_t added; -if (rolling_buffer_init (&rreq->buffer, rreq->debug_id,ITER_DEST , rreq->gfp) <0 ) +if (rolling_buffer_init (&rreq->buffer,ITER_DEST , rreq->gfp, false) <0 )return -ENOMEM ; - added =rolling_buffer_append (&rreq->buffer, folio, rollbuf_flags, rreq->gfp); + added =rolling_buffer_append (&rreq->buffer, folio, rreq->gfp);if (added <0 )return added; rreq->submitted = rreq->start + added;
@@ -668,7 +674,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)trace_netfs_read (rreq, rreq->start, rreq->len, netfs_read_trace_readpage);/* Set up the output buffer */ - ret =netfs_create_singular_buffer (rreq, folio,0 ); + ret =netfs_create_singular_buffer (rreq, folio);if (ret <0 ) goto discard;
@@ -825,7 +831,7 @@ int netfs_write_begin(struct netfs_inode *ctx,trace_netfs_read (rreq, pos, len, netfs_read_trace_write_begin);/* Set up the output buffer */ - ret =netfs_create_singular_buffer (rreq, folio,0 ); + ret =netfs_create_singular_buffer (rreq, folio);if (ret <0 ) goto error_put;
@@ -890,7 +896,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,trace_netfs_read (rreq, start, flen, netfs_read_trace_prefetch_for_write);/* Set up the output buffer */ - ret =netfs_create_singular_buffer (rreq, folio,NETFS_ROLLBUF_PAGECACHE_MARK ); + ret =netfs_create_singular_buffer (rreq, folio);if (ret <0 ) goto error_put;
fs/netfs/iterator.c +21 -21
73 lines not on this page · raw message
fs/netfs/objects.c +1 -0
@@ -254,6 +254,7 @@ static void netfs_free_subrequest(struct netfs_io_subrequest *subreq)trace_netfs_sreq (subreq, netfs_sreq_trace_free);if (rreq->netfs_ops->free_subrequest) rreq->netfs_ops->free_subrequest (subreq); +bvecq_put (subreq->bvecq_pin);mempool_free (subreq, rreq->netfs_ops->subrequest_pool ?: &netfs_subrequest_pool);netfs_stat_d (&netfs_n_rh_sreq);netfs_put_request (rreq, netfs_rreq_trace_put_subreq);
fs/netfs/read_collect.c +24 -24
128 lines not on this page · raw message
fs/netfs/read_pgpriv2.c +31 -12
80 lines not on this page · raw message
fs/netfs/read_retry.c +38 -15
111 lines not on this page · raw message
fs/netfs/rolling_buffer.c +65 -86
243 lines not on this page · raw message
fs/netfs/write_collect.c +31 -13
82 lines not on this page · raw message
fs/netfs/write_issue.c +33 -7
@@ -107,7 +107,10 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping, ictx =netfs_inode (wreq->inode);if (is_cacheable)fscache_begin_write_operation (&wreq->cache_resources,netfs_i_cookie (ictx)); -if (rolling_buffer_init (&wreq->buffer, wreq->debug_id,ITER_SOURCE , wreq->gfp) <0 ) +if ((origin ==NETFS_WRITEBACK || + origin ==NETFS_PGPRIV2_COPY_TO_CACHE ) && +rolling_buffer_init (&wreq->buffer,ITER_SOURCE , wreq->gfp, + origin ==NETFS_WRITEBACK ) <0 ) goto nomem; wreq->cleaned_to = wreq->start;
@@ -163,12 +166,12 @@ void netfs_prepare_write(struct netfs_io_request *wreq,struct netfs_io_subrequest *subreq;struct iov_iter *wreq_iter = &wreq->buffer.iter; -/* Make sure we don't point the iterator at a used-up folio_queue -* struct being used as a placeholder to prevent the queue from -* collapsing. In such a case, extend the queue. +/* Make sure we don't point the iterator at a used-up bvecq struct +* being used as a placeholder to prevent the queue from collapsing. +* In such a case, extend the queue. */ -if (iov_iter_is_folioq (wreq_iter) && - wreq_iter->folioq_slot >=folioq_nr_slots (wreq_iter->folioq)) +if (iov_iter_is_bvecq (wreq_iter) && + !bvecq_acquire_slot (wreq_iter->bvecq, wreq_iter->bvecq_slot))rolling_buffer_make_space (&wreq->buffer, wreq->gfp); subreq =netfs_alloc_subrequest (wreq, stream->source);
@@ -177,6 +180,8 @@ void netfs_prepare_write(struct netfs_io_request *wreq, subreq->start = start; subreq->stream_nr = stream->stream_nr; subreq->io_iter = *wreq_iter; +if (iov_iter_is_bvecq (wreq_iter)) + subreq->bvecq_pin =bvecq_get ((struct bvecq *)wreq->buffer.iter.bvecq);_enter ("R=%x[%x]" , wreq->debug_id, subreq->debug_index);
@@ -267,6 +272,14 @@ void netfs_reissue_write(struct netfs_io_stream *stream,iov_iter_advance (source, size);iov_iter_truncate (&subreq->io_iter, size); +if (iov_iter_is_bvecq (&subreq->io_iter) && + subreq->bvecq_pin != subreq->io_iter.bvecq) { +struct bvecq *old = subreq->bvecq_pin; + + subreq->bvecq_pin =bvecq_get ((struct bvecq *)subreq->io_iter.bvecq); +bvecq_put (old); + } + subreq->retry_count++;/* Keep prepare_write() errno; do_issue_write() terminates FAILED. */ if (!test_bit (NETFS_SREQ_FAILED , &subreq->flags))
@@ -453,7 +466,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq, }/* Attach the folio to the rolling buffer. */ -rolling_buffer_append (&wreq->buffer, folio,0 , wreq->gfp); +rolling_buffer_append (&wreq->buffer, folio, wreq->gfp);/* Move the submission point forward to allow for write-streaming data * not starting at the front of the page. We don't do write-streaming
@@ -529,6 +542,12 @@ static int netfs_write_folio(struct netfs_io_request *wreq,for (int s =0 ; s <NR_IO_STREAMS ; s++)netfs_issue_write (wreq, &wreq->io_streams[s]); +/* Accelerate the cycling of bvecqs when they're in short supply. */ +if (unlikely (wreq->buffer.alloc_flags &BVECQ_HOLD_POOL_SEM ) && +bvecq_is_full (wreq->buffer.head)) +for (int s =0 ; s <NR_IO_STREAMS ; s++) +netfs_issue_write (wreq, &wreq->io_streams[s]); +_leave (" = 0" );return 0 ; }
@@ -605,6 +624,13 @@ int netfs_writepages(struct address_space *mapping, } }while ((folio =writeback_iter (mapping, wbc, folio, &error))); +/* Unlock the request-level emergency pool now (if we have it locked) +* as we are no longer intending to make any allocations from it. The +* bvecqs we have allocated will be returned to the pool as the +* subrequests are collected. +*/ +bvecq_pool_unlock (&wreq->buffer.alloc_flags); +netfs_end_issue_write (wreq);netfs_wake_collector (wreq);
fs/netfs/write_retry.c +14 -1
@@ -18,6 +18,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq, struct netfs_io_stream *stream) {struct list_head *next; +struct bvecq *pin =NULL ;_enter ("R=%x[%x:]" , wreq->debug_id, stream->stream_nr);
@@ -59,6 +60,9 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,size_t part;bool boundary = false; +bvecq_put (pin); + pin =NULL ; +/* Go through the stream and find the next span of contiguous * data that we then rejig (cifs, for example, needs the wsize * renegotiating) and reissue.
@@ -70,7 +74,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,if (test_bit (NETFS_SREQ_FAILED , &from->flags) || !test_bit (NETFS_SREQ_NEED_RETRY , &from->flags)) -return ; +break ;for (;;) {/* Read pointer to subreq before reading subreq state. */
@@ -87,6 +91,13 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq, len += to->len; } +/* Pin start of the bvecq from where the first subreq in the +* sequence started. We will advance the subreqs' pins to +* their new starting points and then drop this pin. +*/ +if (from->bvecq_pin) + pin =bvecq_get (from->bvecq_pin); +/* Determine the set of buffers we're going to use. Each * subreq gets a subset of a single overall contiguous buffer. */
@@ -197,6 +208,8 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq, }while (len); }while (!list_is_head (next, &stream->subrequests)); + +bvecq_put (pin); }/*
fs/smb/client/smb2ops.c +5 -0
@@ -5014,6 +5014,11 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst, goto err_free; out: +/* Unlock the subrequest-level emergency pool now (if we have it +* locked) as we are no longer intending to make any allocations from +* it. The bvecqs we have allocated will be returned to the pool when +* this subrequest is collected. +*/ bvecq_pool_unlock (&flags);return rc;
include/linux/netfs.h +1 -0
@@ -188,6 +188,7 @@ struct netfs_io_subrequest {struct netfs_io_request *rreq;/* Supervising I/O request */ struct work_struct work;struct list_head rreq_link;/* Link in rreq->subrequests */ +struct bvecq *bvecq_pin;/* Pin in request's bvecq chain (holds ref) */ struct iov_iter io_iter;/* Iterator for this subrequest */ uoff_t start;/* Where to start the I/O */ size_t len;/* Size of the I/O */
include/linux/rolling_buffer.h +14 -28
63 lines not on this page · raw message
include/trace/events/netfs.h +1 -1
@@ -398,7 +398,7 @@ TRACE_EVENT(netfs_sreq, __entry->len = sreq->len; __entry->transferred = sreq->transferred; __entry->start = sreq->start; - __entry->slot = sreq->io_iter.folioq_slot; + __entry->slot = sreq->io_iter.bvecq_slot; ), TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
[email protected]
AI reviewRe: [PATCH v19 14/17] netfs: Switch folioq to bvecq
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Critical] netfs: NULL pointer dereference in netfs_read_to_pagecache() --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Critical] netfs: NULL pointer dereference in netfs_read_to_pagecache()
--
commit 8e713b4a482fc07f13d650f4826e027df83215a0
Author: David Howells <[email protected]>
netfs: Switch folioq to bvecq
This commit switches netfslib from using folio_queue to using bvecq for holding lists of folios during buffered reads and writeback.
Quoted 4 lines
fs/netfs/buffered_read.c +0
-0
[ ... ]
Quoted 5 lines
fs/netfs/buffered_read.c +0
-1@@ -275,11 +277,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq) .cached_to[1] = ULLONG_MAX, };struct fscache_occupancy *occ = &_occ; -struct folio_queue *fq = rreq->buffer.tail;
fs/netfs/buffered_read.c static void netfs_read_to_pagecache(struct netfs_io_request *rreq) · line 287
Quoted 10 lines
fs/netfs/buffered_read.c +6
-2-unsigned int offset =0 ; +struct bvecq *mark_bq = rreq->buffer.tail, *mark_pin; +unsigned int mark_offset =0 ;ssize_t size = rreq->len;uoff_t start = rreq->start; -int ret =0 , slot =0 ; +int ret =0 , mark_slot =0 ; + +/* Pin the bvecq whilst we mark folios for copying to the cache. */ + mark_pin =bvecq_get (mark_bq);
[Severity: Critical]
Can this dereference a NULL mark_bq in bvecq_get()?
When netfs_read_folio() reads a dirty, non-uptodate folio created by a streaming write, it calls netfs_read_gaps(), which sets up an ITER_BVEC iterator in rreq->buffer.iter without initializing rreq->buffer.tail and then calls netfs_read_to_pagecache(). Because bvecq_get() does not check for NULL before calling refcount_inc(&bq->ref), this triggers a NULL pointer dereference (unlike the loop below, which checks if (mark_bq)).
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
David Howells
maintainerRe: [PATCH v19 14/17] netfs: Switch folioq to bvecq
In reply to David HowellsHi Christian,
If you apply this patch, can you apply the attached change to it to fix a potential NULL deref spotted by Sashiko[1]?
Thanks,
David
[1] https://sashiko.dev/#/patchset/20261119083851.3306418-1-dhowells%40redhat.com
---
fs/netfs/buffered_read.c +3 -2
@@ -277,14 +277,15 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq) .cached_to[1] = ULLONG_MAX, };struct fscache_occupancy *occ = &_occ; -struct bvecq *mark_bq = rreq->buffer.tail, *mark_pin; +struct bvecq *mark_bq = rreq->buffer.tail, *mark_pin =NULL ;unsigned int mark_offset =0 ;ssize_t size = rreq->len;uoff_t start = rreq->start;int ret =0 , mark_slot =0 ;/* Pin the bvecq whilst we mark folios for copying to the cache. */ - mark_pin =bvecq_get (mark_bq); +if (mark_bq) + mark_pin =bvecq_get (mark_bq); do { int (*prepare_read)(struct netfs_io_subrequest *subreq) = NULL;
David Howells
maintainerPATCH v19 15/17 smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()
In reply to David Howellsnetfslib now only presents an bvecq queue and an associated ITER_BVECQ iterator to the filesystem, so it isn't going to see the ITER_FOLIOQ iterator. So remove that code.
Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect might; further in future, it won't supply iterators at all, but rather a bvecq slice (that can be used to construct an iterator).
--- fs/smb/smbdirect/connection.c | 68 ----------------------------------- 1 file changed, 68 deletions(-)
fs/smb/smbdirect/connection.c +0 -68
@@ -6,7 +6,6 @@#include "internal.h" #include <linux/bvecq.h> -#include <linux/folio_queue.h> struct smbdirect_map_sges {struct ib_sge *sge;
@@ -2143,70 +2142,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,return ret; } -/* -* Extract folio fragments from a FOLIOQ-class iterator and add them to an -* ib_sge list. The folios are not pinned. -*/ -static ssize_t smbdirect_map_sges_from_folioq (struct iov_iter *iter, -struct smbdirect_map_sges *state, -ssize_t maxsize) -{ -const struct folio_queue *folioq = iter->folioq; -unsigned int slot = iter->folioq_slot; -ssize_t ret =0 ; -size_t offset = iter->iov_offset; - -if (WARN_ON_ONCE (!folioq)) -return -EIO ; - -if (slot >=folioq_nr_slots (folioq)) { - folioq = folioq->next; -if (WARN_ON_ONCE (!folioq)) -return -EIO ; - slot =0 ; - } - -do { -struct folio *folio =folioq_folio (folioq, slot); -size_t fsize =folioq_folio_size (folioq, slot); - -if (offset < fsize) { -size_t part =umin (maxsize, fsize - offset); -bool ok; - - ok =smbdirect_map_sges_single_page (state, -folio_page (folio,0 ), - offset, - part); -if (!ok) -return -EIO ; - - offset += part; - ret += part; - maxsize -= part; - } - -if (offset >= fsize) { - offset =0 ; - slot++; -if (slot >=folioq_nr_slots (folioq)) { -if (!folioq->next) { -WARN_ON_ONCE (ret < iter->count); -break ; - } - folioq = folioq->next; - slot =0 ; - } - } - }while (state->num_sge < state->max_sge && maxsize >0 ); - - iter->folioq = folioq; - iter->folioq_slot = slot; - iter->iov_offset = offset; - iter->count -= ret; -return ret; -} -/* * Extract page fragments from up to the given amount of the source iterator * and build up an ib_sge list that refers to all of those bits. The ib_sge list
@@ -2237,9 +2172,6 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,case ITER_KVEC : ret =smbdirect_map_sges_from_kvec (iter, state, len);break ; -case ITER_FOLIOQ : - ret =smbdirect_map_sges_from_folioq (iter, state, len); -break ;default :WARN_ONCE (1 ,"iov_iter_type[%u]\n" ,iov_iter_type (iter));return -EIO ;
[email protected]
AI reviewRe: [PATCH v19 15/17] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter()
Sashiko has reviewed this patch and found no issues.
Quoted 9 lines
netfslib now only presents an bvecq queue and an associated
ITER_BVECQiterator to the filesystem, so it isn't going to see theITER_FOLIOQiterator. So remove that code.Netfslib also won't supply ITER_BVEC/KVEC iterators, though smbdirect might; further in future, it won't supply iterators at all, but rather a bvecq slice (that can be used to construct an iterator).
Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
David Howells
maintainerPATCH v19 16/17 iov_iter: Remove ITER_FOLIOQ
In reply to David HowellsRemove ITER_FOLIOQ as it's no longer used.
--- include/linux/iov_iter.h | 65 +-------- include/linux/uio.h | 12 -- lib/iov_iter.c | 238 +------------------------------- lib/scatterlist.c | 69 +--------- lib/tests/kunit_iov_iter.c | 271 ------------------------------------- 5 files changed, 9 insertions(+), 646 deletions(-)
include/linux/iov_iter.h +2 -63
@@ -11,7 +11,6 @@#include <linux/uio.h> #include <linux/bvec.h> #include <linux/bvecq.h> -#include <linux/folio_queue.h> typedef size_t (*iov_step_f )(void *iter_base,size_t progress,size_t len,void *priv,void *priv2);
@@ -210,62 +209,6 @@ size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,return progress; } -/* -* Handle ITER_FOLIOQ. -*/ -static __always_inline -size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2, -iov_step_f step) -{ -const struct folio_queue *folioq = iter->folioq; -unsigned int slot = iter->folioq_slot; -size_t progress =0 , skip = iter->iov_offset; - -if (slot ==folioq_nr_slots (folioq)) { -/* The iterator may have been extended. */ - folioq = folioq->next; - slot =0 ; - } - -do { -struct folio *folio =folioq_folio (folioq, slot); -size_t part, remain =0 , consumed; -size_t fsize; -void *base; - -if (!folio) -break ; - - fsize =folioq_folio_size (folioq, slot); -if (skip < fsize) { - base =kmap_local_folio (folio, skip); - part =umin (len,PAGE_SIZE - skip %PAGE_SIZE ); - remain =step (base, progress, part, priv, priv2); -kunmap_local (base); - consumed = part - remain; - len -= consumed; - progress += consumed; - skip += consumed; - } -if (skip >= fsize) { - skip =0 ; - slot++; -if (slot ==folioq_nr_slots (folioq) && folioq->next) { - folioq = folioq->next; - slot =0 ; - } - } -if (remain) -break ; - }while (len); - - iter->folioq_slot = slot; - iter->folioq = folioq; - iter->iov_offset = skip; - iter->count -= progress; -return progress; -} -/* * Handle ITER_XARRAY. */
@@ -377,8 +320,6 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,return iterate_kvec (iter, len, priv, priv2, step);if (iov_iter_is_bvecq (iter))return iterate_bvecq (iter, len, priv, priv2, step); -if (iov_iter_is_folioq (iter)) -return iterate_folioq (iter, len, priv, priv2, step);if (iov_iter_is_xarray (iter))return iterate_xarray (iter, len, priv, priv2, step);if (WARN_ON_ONCE (iov_iter_is_dmabuf_map (iter)))
@@ -415,8 +356,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,* buffer is presented in segments, which for kernel iteration are broken up by * physical pages and mapped, with the mapped address being presented. * -* [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and -* DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators. +* [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type +* iterators; it will not handle UBUF or IOVEC-type iterators. * * A step functions, @step, must be provided, one for handling mapped kernel * addresses and the other is given user addresses which have the potential to
@@ -445,8 +386,6 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,return iterate_kvec (iter, len, priv, priv2, step);if (iov_iter_is_bvecq (iter))return iterate_bvecq (iter, len, priv, priv2, step); -if (iov_iter_is_folioq (iter)) -return iterate_folioq (iter, len, priv, priv2, step);if (iov_iter_is_xarray (iter))return iterate_xarray (iter, len, priv, priv2, step);return iterate_discard (iter, len, priv, priv2, step);
include/linux/uio.h +0 -12
@@ -11,7 +11,6 @@#include <uapi/linux/uio.h> struct page ; -struct folio_queue ;struct dma_buf_io_map ;typedef unsigned int __bitwise iov_iter_extraction_t ;
@@ -28,7 +27,6 @@ enum iter_type {ITER_BVEC ,ITER_KVEC ,ITER_BVECQ , -ITER_FOLIOQ ,ITER_XARRAY ,ITER_DISCARD , ITER_DMABUF_MAP,
@@ -72,7 +70,6 @@ struct iov_iter {const struct kvec *kvec;const struct bio_vec *bvec;const struct bvecq *bvecq; -const struct folio_queue *folioq;struct xarray *xarray;void __user *ubuf;struct dma_buf_io_map *dmabuf_map;
@@ -83,7 +80,6 @@ struct iov_iter {union {unsigned long nr_segs;u16 bvecq_slot; -u8 folioq_slot;loff_t xarray_start; }; };
@@ -156,11 +152,6 @@ static inline bool iov_iter_is_bvecq(const struct iov_iter *i)return iov_iter_type (i) ==ITER_BVECQ ; } -static inline bool iov_iter_is_folioq (const struct iov_iter *i) -{ -return iov_iter_type (i) ==ITER_FOLIOQ ; -} -static inline bool iov_iter_is_xarray (const struct iov_iter *i) {return iov_iter_type (i) ==ITER_XARRAY ;
@@ -314,9 +305,6 @@ void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);void iov_iter_bvec_queue (struct iov_iter *i,unsigned int direction,const struct bvecq *bvecq,unsigned int first_slot,unsigned int offset,size_t count); -void iov_iter_folio_queue (struct iov_iter *i,unsigned int direction, -const struct folio_queue *folioq, -unsigned int first_slot,unsigned int offset,size_t count);void iov_iter_xarray (struct iov_iter *i,unsigned int direction,struct xarray *xarray,loff_t start,size_t count); void iov_iter_dmabuf_map(struct iov_iter *i, unsigned int direction,
lib/iov_iter.c +5 -233
316 lines not on this page · raw message
lib/scatterlist.c +2 -67
93 lines not on this page · raw message
lib/tests/kunit_iov_iter.c +0 -271
310 lines not on this page · raw message
[email protected]
AI reviewRe: [PATCH v19 16/17] iov_iter: Remove ITER_FOLIOQ
Sashiko has reviewed this patch and found no issues.
Quoted 3 lines
Remove
ITER_FOLIOQas it's no longer used.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
David Howells
maintainerPATCH v19 17/17 netfs: Remove folio_queue
In reply to David HowellsRemove folio_queue as it's no longer used.
--- Documentation/core-api/folio_queue.rst | 209 ------------------ Documentation/core-api/index.rst | 1 - fs/netfs/internal.h | 5 - fs/netfs/main.c | 8 - fs/netfs/misc.c | 96 --------- fs/netfs/rolling_buffer.c | 46 ---- fs/netfs/stats.c | 4 +- include/linux/folio_queue.h | 282 ------------------------- include/linux/netfs.h | 15 -- include/trace/events/netfs.h | 35 --- kernel/bpf/btf.c | 2 - 11 files changed, 1 insertion(+), 702 deletions(-) delete mode 100644 Documentation/core-api/folio_queue.rst delete mode 100644 include/linux/folio_queue.h
Documentation/core-api/folio_queue.rst deleted +0 -209
@@ -1,209 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0+ - -=========== -Folio Queue -=========== - -:Author: David Howells <[email protected]> - -.. Contents: - - * Overview - * Initialisation - * Adding and removing folios - * Querying information about a folio - * Querying information about a folio_queue - * Folio queue iteration - * Folio marks - * Lockless simultaneous production/consumption issues - - -Overview -======== - -The folio_queue struct forms a single segment in a segmented list of folios -that can be used to form an I/O buffer. As such, the list can be iterated over -using the ITER_FOLIOQ iov_iter type. - -The publicly accessible members of the structure are:: - - struct folio_queue { - struct folio_queue *next; - struct folio_queue *prev; - ... - }; - -A pair of pointers are provided, ``next`` and ``prev``, that point to the -segments on either side of the segment being accessed. Whilst this is a -doubly-linked list, it is intentionally not a circular list; the outward -sibling pointers in terminal segments should be NULL. - -Each segment in the list also stores: - - * an ordered sequence of folio pointers, - * the size of each folio and - * three 1-bit marks per folio, - -but these should not be accessed directly as the underlying data structure may -change, but rather the access functions outlined below should be used. - -The facility can be made accessible by:: - - #include <linux/folio_queue.h> - -and to use the iterator:: - - #include <linux/uio.h> - - -Initialisation -============== - -A segment should be initialised by calling:: - - void folioq_init(struct folio_queue *folioq); - -with a pointer to the segment to be initialised. Note that this will not -necessarily initialise all the folio pointers, so care must be taken to check -the number of folios added. - - -Adding and removing folios -========================== - -Folios can be set in the next unused slot in a segment struct by calling one -of:: - - unsigned int folioq_append(struct folio_queue *folioq, - struct folio *folio); - - unsigned int folioq_append_mark(struct folio_queue *folioq, - struct folio *folio); - -Both functions update the stored folio count, store the folio and note its -size. The second function also sets the first mark for the folio added. Both -functions return the number of the slot used. [!] Note that no attempt is made -to check that the capacity wasn't overrun and the list will not be extended -automatically. - -A folio can be excised by calling:: - - void folioq_clear(struct folio_queue *folioq, unsigned int slot); - -This clears the slot in the array and also clears all the marks for that folio, -but doesn't change the folio count - so future accesses of that slot must check -if the slot is occupied. - - -Querying information about a folio -================================== - -Information about the folio in a particular slot may be queried by the -following function:: - - struct folio *folioq_folio(const struct folio_queue *folioq, - unsigned int slot); - -If a folio has not yet been set in that slot, this may yield an undefined -pointer. The size of the folio in a slot may be queried with either of:: - - unsigned int folioq_folio_order(const struct folio_queue *folioq, - unsigned int slot); - - size_t folioq_folio_size(const struct folio_queue *folioq, - unsigned int slot); - -The first function returns the size as an order and the second as a number of -bytes. - - -Querying information about a folio_queue -======================================== - -Information may be retrieved about a particular segment with the following -functions:: - - unsigned int folioq_nr_slots(const struct folio_queue *folioq); - - unsigned int folioq_count(struct folio_queue *folioq); - - bool folioq_full(struct folio_queue *folioq); - -The first function returns the maximum capacity of a segment. It must not be -assumed that this won't vary between segments. The second returns the number -of folios added to a segments and the third is a shorthand to indicate if the -segment has been filled to capacity. - -Not that the count and fullness are not affected by clearing folios from the -segment. These are more about indicating how many slots in the array have been -initialised, and it assumed that slots won't get reused, but rather the segment -will get discarded as the queue is consumed. - - -Folio marks -=========== - -Folios within a queue can also have marks assigned to them. These marks can be -used to note information such as if a folio needs folio_put() calling upon it. -There are three marks available to be set for each folio. - -The marks can be set by:: - - void folioq_mark(struct folio_queue *folioq, unsigned int slot); - void folioq_mark2(struct folio_queue *folioq, unsigned int slot); - -Cleared by:: - - void folioq_unmark(struct folio_queue *folioq, unsigned int slot); - void folioq_unmark2(struct folio_queue *folioq, unsigned int slot); - -And the marks can be queried by:: - - bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot); - bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot); - -The marks can be used for any purpose and are not interpreted by this API. - - -Folio queue iteration -===================== - -A list of segments may be iterated over using the I/O iterator facility using -an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type. The iterator may be -initialised with:: - - void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction, - const struct folio_queue *folioq, - unsigned int first_slot, unsigned int offset, - size_t count); - -This may be told to start at a particular segment, slot and offset within a -queue. The iov iterator functions will follow the next pointers when advancing -and prev pointers when reverting when needed. - - -Lockless simultaneous production/consumption issues -=================================================== - -If properly managed, the list can be extended by the producer at the head end -and shortened by the consumer at the tail end simultaneously without the need -to take locks. The ITER_FOLIOQ iterator inserts appropriate barriers to aid -with this. - -Care must be taken when simultaneously producing and consuming a list. If the -last segment is reached and the folios it refers to are entirely consumed by -the IOV iterators, an iov_iter struct will be left pointing to the last segment -with a slot number equal to the capacity of that segment. The iterator will -try to continue on from this if there's another segment available when it is -used again, but care must be taken lest the segment got removed and freed by -the consumer before the iterator was advanced. - -It is recommended that the queue always contain at least one segment, even if -that segment has never been filled or is entirely spent. This prevents the -head and tail pointers from collapsing. - - -API Function Reference -====================== - -.. kernel-doc:: include/linux/folio_queue.h
Documentation/core-api/index.rst +0 -1
@@ -40,7 +40,6 @@ Library functionality that is used throughout the kernel. cleanup assoc_array bvecq - folio_queue xarray maple_tree idr
fs/netfs/internal.h +0 -5
@@ -7,7 +7,6 @@#include <linux/slab.h> #include <linux/seq_file.h> -#include <linux/folio_queue.h> #include <linux/netfs.h> #include <linux/fscache.h> #include <linux/fscache-cache.h>
@@ -47,7 +46,6 @@ extern struct list_head netfs_io_requests;extern spinlock_t netfs_proc_lock;extern mempool_t netfs_request_pool;extern mempool_t netfs_subrequest_pool; -extern mempool_t netfs_folioq_pool;extern mempool_t netfs_bvecq_pool[4 ];extern struct semaphore netfs_bvecq_pool_sem[4 ];
@@ -74,8 +72,6 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}/* * misc.c */ -struct folio_queue *netfs_buffer_make_space (struct netfs_io_request *rreq, -enum netfs_folioq_trace trace);void netfs_reset_iter (struct netfs_io_subrequest *subreq);void netfs_wake_collector (struct netfs_io_request *rreq);void netfs_subreq_clear_in_progress (struct netfs_io_subrequest *subreq);
@@ -208,7 +204,6 @@ extern atomic_t netfs_n_wh_retry_write_req;extern atomic_t netfs_n_wh_retry_write_subreq;extern atomic_t netfs_n_wb_lock_skip;extern atomic_t netfs_n_wb_lock_wait; -extern atomic_t netfs_n_folioq;extern atomic_t netfs_n_bvecq;int netfs_stats_show (struct seq_file *m,void *v);
fs/netfs/main.c +0 -8
@@ -27,8 +27,6 @@ static struct kmem_cache *netfs_request_slab;static struct kmem_cache *netfs_subrequest_slab;mempool_t netfs_request_pool;mempool_t netfs_subrequest_pool; -mempool_t netfs_folioq_pool; -mempool_t netfs_bvecq_pool[4 ];/* unused (NO_POOL), REQ_POOL, SUBREQ_POOL, unused */ struct semaphore netfs_bvecq_pool_sem[4 ];
@@ -110,9 +108,6 @@ static int __init netfs_init(void) {int ret = -ENOMEM ; -if (mempool_init_kmalloc_pool (&netfs_folioq_pool,100 ,sizeof (struct folio_queue )) <0 ) - goto error_folioq_pool; -/* We set up multiple bvecq emergency pools: one for requests and one * for subrequests, though there may also need to be one for * reconnection. This is needed to avoid deadlock as a process will
@@ -205,8 +200,6 @@ static int __init netfs_init(void) error_bvecq_pool_2:mempool_exit (&netfs_bvecq_pool[1 ]); error_bvecq_pool_1: -mempool_exit (&netfs_folioq_pool); -error_folioq_pool:return ret; }fs_initcall (netfs_init);
@@ -221,6 +214,5 @@ static void __exit netfs_exit(void)kmem_cache_destroy (netfs_request_slab);mempool_exit (&netfs_bvecq_pool[2 ]);mempool_exit (&netfs_bvecq_pool[1 ]); -mempool_exit (&netfs_folioq_pool); }module_exit (netfs_exit);
fs/netfs/misc.c +0 -96
102 lines not on this page · raw message
fs/netfs/rolling_buffer.c +0 -46
52 lines not on this page · raw message
fs/netfs/stats.c +1 -3
@@ -46,7 +46,6 @@ atomic_t netfs_n_wh_retry_write_req;atomic_t netfs_n_wh_retry_write_subreq;atomic_t netfs_n_wb_lock_skip;atomic_t netfs_n_wb_lock_wait; -atomic_t netfs_n_folioq;atomic_t netfs_n_bvecq;int netfs_stats_show (struct seq_file *m,void *v)
@@ -89,11 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)atomic_read (&netfs_n_rh_retry_read_subreq),atomic_read (&netfs_n_wh_retry_write_req), atomic_read(&netfs_n_wh_retry_write_subreq)); -seq_printf (m,"Objs : rr=%u sr=%u bq=%u foq=%u bqs=%d,%d wsc=%u\n" , +seq_printf (m,"Objs : rr=%u sr=%u bq=%u bqs=%d,%d wsc=%u\n" ,atomic_read (&netfs_n_rh_rreq),atomic_read (&netfs_n_rh_sreq),atomic_read (&netfs_n_bvecq), -atomic_read (&netfs_n_folioq),READ_ONCE (netfs_bvecq_pool_sem[1 ].count),READ_ONCE (netfs_bvecq_pool_sem[2 ].count),atomic_read (&netfs_n_wh_wstream_conflict));
include/linux/folio_queue.h deleted +0 -282
282 lines not on this page · raw message
include/linux/netfs.h +0 -15
@@ -24,7 +24,6 @@enum netfs_sreq_ref_trace ;typedef struct mempool mempool_t ;struct fscache_occupancy ; -struct folio_queue ;/** * folio_start_private_2 - Start an fscache write on a folio. [DEPRECATED]
@@ -250,8 +249,6 @@ struct netfs_io_request {#define NR_IO_STREAMS 2 //wreq->nr_io_streamsstruct netfs_group *group;/* Writeback group being written back */ struct rolling_buffer buffer;/* Unencrypted buffer */ -#define NETFS_ROLLBUF_PUT_MARK ROLLBUF_MARK_1 -#define NETFS_ROLLBUF_PAGECACHE_MARK ROLLBUF_MARK_2wait_queue_head_t waitq;/* Processor waiter */ void *netfs_priv;/* Private data for the netfs */ void *netfs_priv2;/* Private data for the netfs */
@@ -473,18 +470,6 @@ void netfs_end_io_write(struct inode *inode);int netfs_start_io_direct (struct inode *inode);void netfs_end_io_direct (struct inode *inode); -/* Miscellaneous APIs. */ -struct folio_queue *netfs_folioq_alloc (unsigned int rreq_id,gfp_t gfp, -unsigned int trace/*enum netfs_folioq_trace*/ ); -void netfs_folioq_free (struct folio_queue *folioq, -unsigned int trace/*enum netfs_trace_folioq*/ ); - -/* Buffer wrangling helpers API. */ -int netfs_alloc_folioq_buffer (struct address_space *mapping, -struct folio_queue **_buffer, -size_t *_cur_size,ssize_t size,gfp_t gfp); -void netfs_free_folioq_buffer (struct folio_queue *fq); -/* Writeback exclusion API. */ bool netfs_wb_begin (struct netfs_inode *ictx,bool nowait);void netfs_wb_end (struct netfs_inode *ictx);
include/trace/events/netfs.h +0 -35
@@ -199,7 +199,6 @@ EM(netfs_streaming_filled_page, "mod-streamw-f") \ EM(netfs_streaming_cont_filled_page, "mod-streamw-f+") \ EM(netfs_folio_trace_abandon, "abandon") \ - EM(netfs_folio_trace_alloc_buffer, "alloc-buf") \ EM(netfs_folio_trace_cancel_copy, "cancel-copy") \ EM(netfs_folio_trace_cancel_store, "cancel-store") \EM (netfs_folio_trace_end_copy,"end-copy" ) \
@@ -222,7 +221,6 @@ EM(netfs_folio_trace_not_locked, "!locked") \ EM(netfs_folio_trace_not_under_wback, "!wback") \ EM(netfs_folio_trace_pgpriv2_copy, "pgpriv2-copy") \ - EM(netfs_folio_trace_put, "put") \ EM(netfs_folio_trace_read, "read") \ EM(netfs_folio_trace_read_done, "read-done") \EM (netfs_folio_trace_read_gaps,"read-gaps" ) \
@@ -244,14 +242,6 @@ EM(netfs_trace_donate_to_next, "to-next") \ E_(netfs_trace_donate_to_deferred_next, "defer-next") -#define netfs_folioq_traces \ - EM(netfs_trace_folioq_alloc_buffer, "alloc-buf") \ - EM(netfs_trace_folioq_clear, "clear") \ - EM(netfs_trace_folioq_delete, "delete") \ - EM(netfs_trace_folioq_make_space, "make-space") \ - EM(netfs_trace_folioq_rollbuf_init, "roll-init") \ - E_(netfs_trace_folioq_read_progress, "r-progress") -#ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY#define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
@@ -270,7 +260,6 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);enum netfs_folio_trace { netfs_folio_traces }__mode (byte );enum netfs_collect_contig_trace { netfs_collect_contig_traces }__mode (byte );enum netfs_donate_trace { netfs_donate_traces }__mode (byte ); -enum netfs_folioq_trace { netfs_folioq_traces }__mode (byte );#endif
@@ -294,7 +283,6 @@ netfs_sreq_ref_traces; netfs_folio_traces; netfs_collect_contig_traces; netfs_donate_traces; -netfs_folioq_traces;/* * Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -773,29 +761,6 @@ TRACE_EVENT(netfs_collect_stream, __entry->collected_to, __entry->issued_to) ); -TRACE_EVENT (netfs_folioq, - TP_PROTO(const struct folio_queue *fq, - enum netfs_folioq_trace trace), - -TP_ARGS (fq, trace), - -TP_STRUCT__entry ( - __field(unsigned int, rreq) - __field(unsigned int, id) - __field(enum netfs_folioq_trace, trace) - ), - -TP_fast_assign ( - __entry->rreq = fq ? fq->rreq_id : 0; - __entry->id = fq ? fq->debug_id : 0; - __entry->trace = trace; - ), - -TP_printk ("R=%08x fq=%x %s" , - __entry->rreq, __entry->id, -__print_symbolic (__entry->trace, netfs_folioq_traces)) - ); - TRACE_EVENT(netfs_read_progress_at, TP_PROTO(const struct netfs_io_request *rreq),
kernel/bpf/btf.c +0 -2
@@ -7353,8 +7353,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {/* amdgpu */ { "amdgpu_vm_bo_map", 0x1 }, { "amdgpu_vm_bo_unmap", 0x1 }, -/* netfs */ - { "netfs_folioq", 0x1 },/* xfs from xfs_defer_pending_class */ { "xfs_defer_create_intent", 0x1 }, { "xfs_defer_cancel_list", 0x1 },
[email protected]
AI reviewRe: [PATCH v19 17/17] netfs: Remove folio_queue
Sashiko has reviewed this patch and found no issues.
Quoted 3 lines
Remove
folio_queueas it's no longer used.Signed-off-by David Howells <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=17
Quoted 6 lines · David Howells
Could you pull these patches please into your vfs-7.5.netfs branch? This is the third of five batches (two already having gone upstream), in this case replacing the
folio_queuestruct with a bvecq struct. It has been rebased on v7.4-rc3. This was split from v11 of a larger netfslib series[1].
Quoted 8 lines
This series adds a new type, struct bvecq. This points to a bounded array of
bio_vecswith information about how to clean up the memory they point to. A number of bvecq structs can be linked together to form a chain and an iterator type,ITER_BVECQ, is provided that can be pointed at the chain and can walk it. Netfslib is then switched to use the new type andfolio_queueis removed along withITER_FOLIOQ.[...]
Applied to the vfs-7.5.netfs branch of the vfs/vfs.git tree. Patches in the vfs-7.5.netfs branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-7.5.netfs
[01/17] iov_iter: Fix kunit test to remove __init and __refdata marks
https://git.kernel.org/vfs/vfs/c/1bce9def76dd [02/17] block: Add a function to kmap one page of a multipage bio_vec https://git.kernel.org/vfs/vfs/c/14e2292a9706 [03/17] iov_iter: Add a segmented queue of bio_vec[] https://git.kernel.org/vfs/vfs/c/5f1dd24265f0 [04/17] netfs: Add some tools for managing bvecq chains https://git.kernel.org/vfs/vfs/c/b4e0ea3853e5 [05/17] iov_iter: Add an API document for the bvecq structure https://git.kernel.org/vfs/vfs/c/ca90a27f2d94 [06/17] iov_iter: Add a bvecq kunit test https://git.kernel.org/vfs/vfs/c/0a2f9551262e [07/17] afs: Fix an inconsistency in the symlink code https://git.kernel.org/vfs/vfs/c/44921a81054c [08/17] netfs: Explicitly pass tail padding to netfs_writeback_single() https://git.kernel.org/vfs/vfs/c/7a3a80623cca [09/17] netfs: Change netfs_writeback_single() to not look inside the iterator https://git.kernel.org/vfs/vfs/c/6703bfa120ef [10/17] netfs: Pass the write length to request creation where appropriate https://git.kernel.org/vfs/vfs/c/c7496c6ed431 [11/17] afs: Use a bvecq to hold dir content rather than folioq https://git.kernel.org/vfs/vfs/c/65752f84cf61 [12/17] cifs: Use a bvecq for buffering instead of a folioq https://git.kernel.org/vfs/vfs/c/145d329f12ee [13/17] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter() https://git.kernel.org/vfs/vfs/c/adf3cafd9933 [14/17] netfs: Switch folioq to bvecq https://git.kernel.org/vfs/vfs/c/5b95afa0aeaf [15/17] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter() https://git.kernel.org/vfs/vfs/c/c7879d1a4e06 [16/17] iov_iter: Remove ITER_FOLIOQ https://git.kernel.org/vfs/vfs/c/3d9d0bbb7fcd [17/17] netfs: Remove folio_queue https://git.kernel.org/vfs/vfs/c/d6d5b8979e5d