netfs: Miscellaneous fixes
Subsystems (12)
David Howells
maintainercover netfs: Miscellaneous fixes
Hi Christian,
Here are some miscellaneous fixes and changes for netfslib, if you could pull them:
(1) Fix an oops in write-retry due to mis-resetting the I/O iterator.
(2) Fix the recording of transferred bytes for short DIO reads.
(3) Fix a request's work item to not require a reference, thereby avoiding the need to get rid of it in BH/IRQ context.
(4) Fix waiting and waking to be consistent about the waitqueue used.
The patches can also be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-fixes
Thanks,
David
David Howells (3):
netfs: Fix oops in write-retry from mis-resetting the subreq iterator
netfs: Fix the request's work item to not require a ref
netfs: Fix wait/wake to be consistent about the waitqueue used
Paulo Alcantara (1):
netfs: Fix setting of transferred bytes with short DIO reads
fs/9p/vfs_addr.c | 2 +- fs/afs/write.c | 8 +- fs/cachefiles/io.c | 16 +-- fs/ceph/addr.c | 2 +- fs/erofs/fscache.c | 6 +- fs/netfs/buffered_read.c | 32 +++-- fs/netfs/buffered_write.c | 2 +- fs/netfs/direct_read.c | 10 +- fs/netfs/direct_write.c | 12 +- fs/netfs/fscache_io.c | 10 +- fs/netfs/internal.h | 42 +++++-- fs/netfs/misc.c | 218 +++++++++++++++++++++++++++++++++++ fs/netfs/objects.c | 47 ++++---- fs/netfs/read_collect.c | 178 ++++------------------------ fs/netfs/read_pgpriv2.c | 4 +- fs/netfs/read_retry.c | 26 +---- fs/netfs/read_single.c | 6 +- fs/netfs/write_collect.c | 81 +++++-------- fs/netfs/write_issue.c | 38 +++--- fs/netfs/write_retry.c | 19 ++- fs/smb/client/cifsproto.h | 3 +- fs/smb/client/cifssmb.c | 4 +- fs/smb/client/file.c | 7 +- fs/smb/client/smb2pdu.c | 4 +- include/linux/fscache.h | 2 +- include/linux/netfs.h | 14 +-- include/trace/events/netfs.h | 7 +- net/9p/client.c | 6 +- 28 files changed, 427 insertions(+), 379 deletions(-)
David Howells
maintainerPATCH 1/4 netfs: Fix oops in write-retry from mis-resetting the subreq iterator
Fix the resetting of the subrequest iterator in netfs_retry_write_stream() to use the iterator-reset function as the iterator may have been shortened by a previous retry. In such a case, the amount of data to be written by the subrequest is not "subreq->len" but "subreq->len - subreq->transferred".
Without this, KASAN may see an error in iov_iter_revert():
BUG: KASAN: slab-out-of-bounds in iov_iter_revert lib/iov_iter.c:633 [inline]
BUG: KASAN: slab-out-of-bounds in iov_iter_revert+0x443/0x5a0 lib/iov_iter.c:611
Read of size 4 at addr ffff88802912a0b8 by task kworker/u32:7/1147
CPU: 1 UID: 0 PID: 1147 Comm: kworker/u32:7 Not tainted 6.15.0-rc6-syzkaller-00052-g9f35e33144ae #0 PREEMPT(full)
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Workqueue: events_unbound netfs_write_collection_worker
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x116/0x1f0 lib/dump_stack.c:120
print_address_description mm/kasan/report.c:408 [inline]
print_report+0xc3/0x670 mm/kasan/report.c:521
kasan_report+0xe0/0x110 mm/kasan/report.c:634
iov_iter_revert lib/iov_iter.c:633 [inline]
iov_iter_revert+0x443/0x5a0 lib/iov_iter.c:611
netfs_retry_write_stream fs/netfs/write_retry.c:44 [inline]
netfs_retry_writes+0x166d/0x1a50 fs/netfs/write_retry.c:231
netfs_collect_write_results fs/netfs/write_collect.c:352 [inline]
netfs_write_collection_worker+0x23fd/0x3830 fs/netfs/write_collect.c:374
process_one_work+0x9cf/0x1b70 kernel/workqueue.c:3238
process_scheduled_works kernel/workqueue.c:3319 [inline]
worker_thread+0x6c8/0xf10 kernel/workqueue.c:3400
kthread+0x3c2/0x780 kernel/kthread.c:464
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:153
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
</TASK>--- fs/netfs/write_retry.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
fs/netfs/write_retry.c +3 -2
@@ -39,9 +39,10 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,if (test_bit (NETFS_SREQ_FAILED , &subreq->flags))break ;if (__test_and_clear_bit (NETFS_SREQ_NEED_RETRY , &subreq->flags)) { -struct iov_iter source = subreq->io_iter; +struct iov_iter source; -iov_iter_revert (&source, subreq->len - source.count); +netfs_reset_iter (subreq); + source = subreq->io_iter;netfs_get_subrequest (subreq, netfs_sreq_trace_get_resubmit);netfs_reissue_write (stream, subreq, &source); }
David Howells
maintainerPATCH 2/4 netfs: Fix setting of transferred bytes with short DIO reads
In reply to David HowellsFrom: Paulo Alcantara <[email protected]>
A netfslib request comprises an ordered stream of subrequests that, when doing an unbuffered/DIO read, are contiguous. The subrequests may be performed in parallel, but may not be fully completed.
For instance, if we try and make a 256KiB DIO read from a 3-byte file with a 64KiB rsize and 256KiB bsize, netfslib will attempt to make a read of 256KiB, broken up into four 64KiB subreads, with the expectation that the first will be short and the subsequent three be completely devoid - but we do all four on the basis that the file may have been changed by a third party.
The read-collection code, however, walks through all the subreqs and advances the notion of how much data has been read in the stream to the start of each subreq plus its amount transferred (which are 3, 0, 0, 0 for the example above) - which gives an amount apparently read of 3*64KiB - which is incorrect.
Fix the collection code to cut short the calculation of the transferred amount with the first short subrequest in an unbuffered read; everything beyond that must be ignored as there's a hole that cannot be filled. This applies both to shortness due to hitting the EOF and shortness due to an error.
This is achieved by setting a flag on the request when we collect the first short subrequest (collection is done in ascending order).
This can be tested by mounting a cifs volume with rsize=65536,bsize=262144 and doing a 256k DIO read of a very small file (e.g. 3 bytes). read() should return 3, not >3.
This problem came in when netfs_read_collection() set rreq->transferred to stream->transferred, even for DIO. Prior to that, netfs_rreq_assess_dio() just went over the list and added up the subreqs till it met a short one - but now the subreqs are discarded earlier.
--- fs/netfs/read_collect.c | 21 +++++---------------- include/linux/netfs.h | 1 + 2 files changed, 6 insertions(+), 16 deletions(-)
fs/netfs/read_collect.c +5 -16
@@ -280,9 +280,13 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq) stream->need_retry = true; notes |=NEED_RETRY |MADE_PROGRESS ;break ; + }else if (test_bit (NETFS_RREQ_SHORT_TRANSFER , &rreq->flags)) { + notes |=MADE_PROGRESS ; }else {if (!stream->failed) - stream->transferred = stream->collected_to - rreq->start; + stream->transferred += transferred; +if (front->transferred < front->len) +set_bit (NETFS_RREQ_SHORT_TRANSFER , &rreq->flags); notes |=MADE_PROGRESS ; }
@@ -342,23 +346,8 @@ static void netfs_collect_read_results(struct netfs_io_request *rreq)*/ static void netfs_rreq_assess_dio (struct netfs_io_request *rreq) { -struct netfs_io_subrequest *subreq; -struct netfs_io_stream *stream = &rreq->io_streams[0 ];unsigned int i; -/* Collect unbuffered reads and direct reads, adding up the transfer -* sizes until we find the first short or failed subrequest. -*/ -list_for_each_entry (subreq, &stream->subrequests, rreq_link) { - rreq->transferred += subreq->transferred; - -if (subreq->transferred < subreq->len || -test_bit (NETFS_SREQ_FAILED , &subreq->flags)) { - rreq->error = subreq->error; -break ; - } - } -if (rreq->origin ==NETFS_DIO_READ ) {for (i =0 ; i < rreq->direct_bv_count; i++) {flush_dcache_page (rreq->direct_bv[i].bv_page);
include/linux/netfs.h +1 -0
@@ -279,6 +279,7 @@ struct netfs_io_request {#define NETFS_RREQ_USE_IO_ITER 12/* Use ->io_iter rather than ->i_pages */ #define NETFS_RREQ_ALL_QUEUED 13/* All subreqs are now queued */ #define NETFS_RREQ_RETRYING 14/* Set if we're in the retry path */ +#define NETFS_RREQ_SHORT_TRANSFER 15/* Set if we have a short transfer */ #define NETFS_RREQ_USE_PGPRIV2 31/* [DEPRECATED] Use PG_private_2 to mark * write to cache on read */ const struct netfs_request_ops *netfs_ops;
David Howells
maintainerPATCH 3/4 netfs: Fix the request's work item to not require a ref
In reply to David HowellsWhen the netfs_io_request struct's work item is queued, it must be supplied with a ref to the work item struct to prevent it being deallocated whilst on the queue or whilst it is being processed. This is tricky to manage as we have to get a ref before we try and queue it and then we may find it's already queued and is thus already holding a ref - in which case we have to try and get rid of the ref again.
The problem comes if we're in BH or IRQ context and need to drop the ref:
if netfs_put_request() reduces the count to 0, we have to do the cleanup - but the cleanup may need to wait.
Fix this by adding a new work item to the request, ->cleanup_work, and dispatching that when the refcount hits zero. That can then synchronously cancel any outstanding work on the main work item before doing the cleanup.
Adding a new work item also deals with another problem upstream where it's sometimes changing the work func in the put function and requeuing it - which has occasionally in the past caused the cleanup to happen incorrectly.
As a bonus, this allows us to get rid of the 'was_async' parameter from a bunch of functions. This indicated whether the put function might not be permitted to sleep.
--- fs/9p/vfs_addr.c | 2 +- fs/afs/write.c | 8 ++--- fs/cachefiles/io.c | 16 +++++----- fs/ceph/addr.c | 2 +- fs/erofs/fscache.c | 6 ++-- fs/netfs/buffered_read.c | 30 +++++++++--------- fs/netfs/direct_read.c | 6 ++-- fs/netfs/direct_write.c | 2 +- fs/netfs/fscache_io.c | 10 +++--- fs/netfs/internal.h | 11 +++---- fs/netfs/objects.c | 47 +++++++++++++-------------- fs/netfs/read_collect.c | 44 ++++++++++++++++---------- fs/netfs/read_pgpriv2.c | 4 +-- fs/netfs/read_retry.c | 2 +- fs/netfs/read_single.c | 6 ++-- fs/netfs/write_collect.c | 61 +++++++++++++++++------------------- fs/netfs/write_issue.c | 16 +++++----- fs/netfs/write_retry.c | 2 +- fs/smb/client/cifsproto.h | 3 +- fs/smb/client/cifssmb.c | 4 +-- fs/smb/client/file.c | 7 ++--- fs/smb/client/smb2pdu.c | 4 +-- include/linux/fscache.h | 2 +- include/linux/netfs.h | 13 ++++---- include/trace/events/netfs.h | 7 ++--- net/9p/client.c | 6 ++-- 26 files changed, 159 insertions(+), 162 deletions(-)
fs/9p/vfs_addr.c +1 -1
@@ -59,7 +59,7 @@ static void v9fs_issue_write(struct netfs_io_subrequest *subreq) len =p9_client_write (fid, subreq->start, &subreq->io_iter, &err);if (len >0 )__set_bit (NETFS_SREQ_MADE_PROGRESS , &subreq->flags); -netfs_write_subrequest_terminated (subreq, len ?: err, false); +netfs_write_subrequest_terminated (subreq, len ?: err); }/**
fs/afs/write.c +4 -4
@@ -120,17 +120,17 @@ static void afs_issue_write_worker(struct work_struct *work)#if 0 // Error injection if (subreq->debug_index ==3 ) -return netfs_write_subrequest_terminated (subreq, -ENOANO , false); +return netfs_write_subrequest_terminated (subreq, -ENOANO );if (!subreq->retry_count) {set_bit (NETFS_SREQ_NEED_RETRY , &subreq->flags); -return netfs_write_subrequest_terminated (subreq, -EAGAIN , false); +return netfs_write_subrequest_terminated (subreq, -EAGAIN ); }#endif op =afs_alloc_operation (wreq->netfs_priv, vnode->volume);if (IS_ERR (op)) -return netfs_write_subrequest_terminated (subreq, -EAGAIN , false); +return netfs_write_subrequest_terminated (subreq, -EAGAIN );afs_op_set_vnode (op,0 , vnode); op->file[0 ].dv_delta =1 ;
@@ -166,7 +166,7 @@ static void afs_issue_write_worker(struct work_struct *work)break ; } -netfs_write_subrequest_terminated (subreq, ret <0 ? ret : subreq->len, false); +netfs_write_subrequest_terminated (subreq, ret <0 ? ret : subreq->len); }void afs_issue_write (struct netfs_io_subrequest *subreq)
fs/cachefiles/io.c +8 -8
@@ -63,7 +63,7 @@ static void cachefiles_read_complete(struct kiocb *iocb, long ret) ret = -ESTALE ; } - ki->term_func (ki->term_func_priv, ret, ki->was_async); + ki->term_func (ki->term_func_priv, ret); }cachefiles_put_kiocb (ki);
@@ -188,7 +188,7 @@ static int cachefiles_read(struct netfs_cache_resources *cres, presubmission_error:if (term_func) -term_func (term_func_priv, ret <0 ? ret : skipped, false); +term_func (term_func_priv, ret <0 ? ret : skipped);return ret; }
@@ -271,7 +271,7 @@ static void cachefiles_write_complete(struct kiocb *iocb, long ret)atomic_long_sub (ki->b_writing, &object->volume->cache->b_writing);set_bit (FSCACHE_COOKIE_HAVE_DATA , &object->cookie->flags);if (ki->term_func) - ki->term_func (ki->term_func_priv, ret, ki->was_async); + ki->term_func (ki->term_func_priv, ret);cachefiles_put_kiocb (ki); }
@@ -301,7 +301,7 @@ int __cachefiles_write(struct cachefiles_object *object, ki =kzalloc (sizeof (struct cachefiles_kiocb ),GFP_KERNEL );if (!ki) {if (term_func) -term_func (term_func_priv, -ENOMEM , false); +term_func (term_func_priv, -ENOMEM );return -ENOMEM ; }
@@ -366,7 +366,7 @@ static int cachefiles_write(struct netfs_cache_resources *cres, {if (!fscache_wait_for_operation (cres,FSCACHE_WANT_WRITE )) {if (term_func) -term_func (term_func_priv, -ENOBUFS , false); +term_func (term_func_priv, -ENOBUFS );trace_netfs_sreq (term_func_priv, netfs_sreq_trace_cache_nowrite);return -ENOBUFS ; }
@@ -665,7 +665,7 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq) pre =CACHEFILES_DIO_BLOCK_SIZE - off;if (pre >= len) {fscache_count_dio_misfit (); -netfs_write_subrequest_terminated (subreq, len, false); +netfs_write_subrequest_terminated (subreq, len);return ; } subreq->transferred += pre;
@@ -691,7 +691,7 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq) len -= post;if (len ==0 ) {fscache_count_dio_misfit (); -netfs_write_subrequest_terminated (subreq, post, false); +netfs_write_subrequest_terminated (subreq, post);return ; }iov_iter_truncate (&subreq->io_iter, len);
@@ -703,7 +703,7 @@ static void cachefiles_issue_write(struct netfs_io_subrequest *subreq) &start, &len, len, true);cachefiles_end_secure (cache, saved_cred);if (ret <0 ) { -netfs_write_subrequest_terminated (subreq, ret, false); +netfs_write_subrequest_terminated (subreq, ret);return ; }
fs/ceph/addr.c +1 -1
@@ -539,7 +539,7 @@ static void ceph_set_page_fscache(struct page *page)folio_start_private_2 (page_folio (page));/* [DEPRECATED] */ } -static void ceph_fscache_write_terminated (void *priv,ssize_t error,bool was_async) +static void ceph_fscache_write_terminated (void *priv,ssize_t error) {struct inode *inode = priv;
fs/erofs/fscache.c +2 -4
@@ -102,8 +102,7 @@ static void erofs_fscache_req_io_put(struct erofs_fscache_io *io)erofs_fscache_req_put (req); } -static void erofs_fscache_req_end_io (void *priv, -ssize_t transferred_or_error,bool was_async) +static void erofs_fscache_req_end_io (void *priv,ssize_t transferred_or_error) {struct erofs_fscache_io *io = priv;struct erofs_fscache_rq *req = io->private;
@@ -180,8 +179,7 @@ struct erofs_fscache_bio {struct bio_vec bvecs[BIO_MAX_VECS ]; }; -static void erofs_fscache_bio_endio (void *priv, -ssize_t transferred_or_error,bool was_async) +static void erofs_fscache_bio_endio (void *priv,ssize_t transferred_or_error) {struct erofs_fscache_bio *io = priv;
fs/netfs/buffered_read.c +14 -16
@@ -262,9 +262,9 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)if (ret <0 ) { subreq->error = ret;/* Not queued - release both refs. */ -netfs_put_subrequest (subreq, false, +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel); -netfs_put_subrequest (subreq, false, +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel);break ; }
@@ -297,8 +297,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq) subreq->error = ret;trace_netfs_sreq (subreq, netfs_sreq_trace_cancel);/* Not queued - release both refs. */ -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_cancel); -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_cancel); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel);break ; } size -= slice;
@@ -365,12 +365,10 @@ void netfs_readahead(struct readahead_control *ractl) goto cleanup_free;netfs_read_to_pagecache (rreq); -netfs_put_request (rreq, true, netfs_rreq_trace_put_return); -return ; +return netfs_put_request (rreq, netfs_rreq_trace_put_return); cleanup_free: -netfs_put_request (rreq, false, netfs_rreq_trace_put_failed); -return ; +return netfs_put_request (rreq, netfs_rreq_trace_put_failed); }EXPORT_SYMBOL (netfs_readahead);
@@ -470,11 +468,11 @@ static int netfs_read_gaps(struct file *file, struct folio *folio)folio_mark_uptodate (folio); }folio_unlock (folio); -netfs_put_request (rreq, false, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return);return ret <0 ? ret :0 ; discard: -netfs_put_request (rreq, false, netfs_rreq_trace_put_discard); +netfs_put_request (rreq, netfs_rreq_trace_put_discard); alloc_error:folio_unlock (folio);return ret;
@@ -530,11 +528,11 @@ int netfs_read_folio(struct file *file, struct folio *folio)netfs_read_to_pagecache (rreq); ret =netfs_wait_for_read (rreq); -netfs_put_request (rreq, false, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return);return ret <0 ? ret :0 ; discard: -netfs_put_request (rreq, false, netfs_rreq_trace_put_discard); +netfs_put_request (rreq, netfs_rreq_trace_put_discard); alloc_error:folio_unlock (folio);return ret;
@@ -689,7 +687,7 @@ int netfs_write_begin(struct netfs_inode *ctx, ret =netfs_wait_for_read (rreq);if (ret <0 ) goto error; -netfs_put_request (rreq, false, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return); have_folio: ret =folio_wait_private_2_killable (folio);
@@ -701,7 +699,7 @@ int netfs_write_begin(struct netfs_inode *ctx,return 0 ; error_put: -netfs_put_request (rreq, false, netfs_rreq_trace_put_failed); +netfs_put_request (rreq, netfs_rreq_trace_put_failed); error:if (folio) {folio_unlock (folio);
@@ -752,11 +750,11 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,netfs_read_to_pagecache (rreq); ret =netfs_wait_for_read (rreq); -netfs_put_request (rreq, false, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return);return ret <0 ? ret :0 ; error_put: -netfs_put_request (rreq, false, netfs_rreq_trace_put_discard); +netfs_put_request (rreq, netfs_rreq_trace_put_discard); error:_leave (" = %d" , ret);return ret;
fs/netfs/direct_read.c +3 -3
@@ -85,7 +85,7 @@ static int netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)if (rreq->netfs_ops->prepare_read) { ret = rreq->netfs_ops->prepare_read (subreq);if (ret <0 ) { -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_cancel); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel);break ; } }
@@ -144,7 +144,7 @@ static ssize_t netfs_unbuffered_read(struct netfs_io_request *rreq, bool sync) ret =netfs_dispatch_unbuffered_reads (rreq);if (!rreq->submitted) { -netfs_put_request (rreq, false, netfs_rreq_trace_put_no_submit); +netfs_put_request (rreq, netfs_rreq_trace_put_no_submit);inode_dio_end (rreq->inode); ret =0 ; goto out;
@@ -236,7 +236,7 @@ ssize_t netfs_unbuffered_read_iter_locked(struct kiocb *iocb, struct iov_iter *i } out: -netfs_put_request (rreq, false, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return);if (ret >0 ) orig_count -= ret;return ret;
fs/netfs/direct_write.c +1 -1
@@ -117,7 +117,7 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter * } out: -netfs_put_request (wreq, false, netfs_rreq_trace_put_return); +netfs_put_request (wreq, netfs_rreq_trace_put_return);return ret; }EXPORT_SYMBOL (netfs_unbuffered_write_iter_locked);
fs/netfs/fscache_io.c +4 -6
@@ -192,8 +192,7 @@ EXPORT_SYMBOL(__fscache_clear_page_bits);/* * Deal with the completion of writing the data to the cache. */ -static void fscache_wreq_done (void *priv,ssize_t transferred_or_error, -bool was_async) +static void fscache_wreq_done (void *priv,ssize_t transferred_or_error) {struct fscache_write_request *wreq = priv;
@@ -202,8 +201,7 @@ static void fscache_wreq_done(void *priv, ssize_t transferred_or_error, wreq->set_bits);if (wreq->term_func) - wreq->term_func (wreq->term_func_priv, transferred_or_error, - was_async); + wreq->term_func (wreq->term_func_priv, transferred_or_error);fscache_end_operation (&wreq->cache_resources);kfree (wreq); }
@@ -255,14 +253,14 @@ void __fscache_write_to_cache(struct fscache_cookie *cookie,return ; abandon_end: -return fscache_wreq_done (wreq, ret, false); +return fscache_wreq_done (wreq, ret); abandon_free:kfree (wreq); abandon:if (using_pgpriv2)fscache_clear_page_bits (mapping, start, len, cond);if (term_func) -term_func (term_func_priv, ret, false); +term_func (term_func_priv, ret); }EXPORT_SYMBOL (__fscache_write_to_cache);
fs/netfs/internal.h +5 -6
@@ -23,7 +23,7 @@/* * buffered_read.c */ -void netfs_cache_read_terminated (void *priv,ssize_t transferred_or_error,bool was_async); +void netfs_cache_read_terminated (void *priv,ssize_t transferred_or_error);int netfs_prefetch_for_write (struct file *file,struct folio *folio,size_t offset,size_t len);
@@ -71,9 +71,8 @@ struct netfs_io_request *netfs_alloc_request(struct address_space *mapping, loff_t start, size_t len, enum netfs_io_origin origin);void netfs_get_request (struct netfs_io_request *rreq,enum netfs_rreq_ref_trace what); -void netfs_clear_subrequests (struct netfs_io_request *rreq,bool was_async); -void netfs_put_request (struct netfs_io_request *rreq,bool was_async, -enum netfs_rreq_ref_trace what); +void netfs_clear_subrequests (struct netfs_io_request *rreq); +void netfs_put_request (struct netfs_io_request *rreq,enum netfs_rreq_ref_trace what);struct netfs_io_subrequest *netfs_alloc_subrequest (struct netfs_io_request *rreq); static inline void netfs_see_request(struct netfs_io_request *rreq,
@@ -94,7 +93,7 @@ static inline void netfs_see_subrequest(struct netfs_io_subrequest *subreq,*/ void netfs_read_collection_worker (struct work_struct *work);void netfs_wake_read_collector (struct netfs_io_request *rreq); -void netfs_cache_read_terminated (void *priv,ssize_t transferred_or_error,bool was_async); +void netfs_cache_read_terminated (void *priv,ssize_t transferred_or_error);ssize_t netfs_wait_for_read (struct netfs_io_request *rreq);void netfs_wait_for_pause (struct netfs_io_request *rreq);
@@ -177,7 +176,7 @@ static inline void netfs_stat_d(atomic_t *stat)*/ int netfs_folio_written_back (struct folio *folio);void netfs_write_collection_worker (struct work_struct *work); -void netfs_wake_write_collector (struct netfs_io_request *wreq,bool was_async); +void netfs_wake_write_collector (struct netfs_io_request *wreq);/* * write_issue.c
fs/netfs/objects.c +24 -23
131 lines not on this page · raw message
fs/netfs/read_collect.c +27 -17
112 lines not on this page · raw message
fs/netfs/read_pgpriv2.c +2 -2
@@ -116,7 +116,7 @@ static struct netfs_io_request *netfs_pgpriv2_begin_copy_to_cache(return creq; cancel_put: -netfs_put_request (creq, false, netfs_rreq_trace_put_return); +netfs_put_request (creq, netfs_rreq_trace_put_return); cancel: rreq->copy_to_cache =ERR_PTR (-ENOBUFS );clear_bit (NETFS_RREQ_FOLIO_COPY_TO_CACHE , &rreq->flags);
@@ -155,7 +155,7 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)smp_wmb ();/* Write lists before ALL_QUEUED. */ set_bit (NETFS_RREQ_ALL_QUEUED , &creq->flags); -netfs_put_request (creq, false, netfs_rreq_trace_put_return); +netfs_put_request (creq, netfs_rreq_trace_put_return); creq->copy_to_cache =NULL ; }
fs/netfs/read_retry.c +1 -1
@@ -173,7 +173,7 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq) &stream->subrequests, rreq_link) {trace_netfs_sreq (subreq, netfs_sreq_trace_superfluous);list_del (&subreq->rreq_link); -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_done); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_done);if (subreq == to)break ; }
fs/netfs/read_single.c +3 -3
@@ -142,7 +142,7 @@ static int netfs_single_dispatch_read(struct netfs_io_request *rreq)set_bit (NETFS_RREQ_ALL_QUEUED , &rreq->flags);return ret; cancel: -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_cancel); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_cancel);return ret; }
@@ -185,11 +185,11 @@ ssize_t netfs_read_single(struct inode *inode, struct file *file, struct iov_itenetfs_single_dispatch_read (rreq); ret =netfs_wait_for_read (rreq); -netfs_put_request (rreq, true, netfs_rreq_trace_put_return); +netfs_put_request (rreq, netfs_rreq_trace_put_return);return ret; cleanup_free: -netfs_put_request (rreq, false, netfs_rreq_trace_put_failed); +netfs_put_request (rreq, netfs_rreq_trace_put_failed);return ret; }EXPORT_SYMBOL (netfs_read_single);
fs/netfs/write_collect.c +28 -33
133 lines not on this page · raw message
fs/netfs/write_issue.c +9 -7
@@ -134,7 +134,7 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,return wreq; nomem: wreq->error = -ENOMEM ; -netfs_put_request (wreq, false, netfs_rreq_trace_put_failed); +netfs_put_request (wreq, netfs_rreq_trace_put_failed);return ERR_PTR (-ENOMEM ); }
@@ -233,7 +233,7 @@ static void netfs_do_issue_write(struct netfs_io_stream *stream,_enter ("R=%x[%x],%zx" , wreq->debug_id, subreq->debug_index, subreq->len);if (test_bit (NETFS_SREQ_FAILED , &subreq->flags)) -return netfs_write_subrequest_terminated (subreq, subreq->error, false); +return netfs_write_subrequest_terminated (subreq, subreq->error);trace_netfs_sreq (subreq, netfs_sreq_trace_submit); stream->issue_write (subreq);
@@ -542,7 +542,7 @@ static void netfs_end_issue_write(struct netfs_io_request *wreq) }if (needs_poke) -netfs_wake_write_collector (wreq, false); +netfs_wake_write_collector (wreq); }/*
@@ -599,8 +599,9 @@ int netfs_writepages(struct address_space *mapping,netfs_end_issue_write (wreq);mutex_unlock (&ictx->wb_lock); +netfs_wake_write_collector (wreq); -netfs_put_request (wreq, false, netfs_rreq_trace_put_return); +netfs_put_request (wreq, netfs_rreq_trace_put_return);_leave (" = %d" , error);return error;
@@ -694,7 +695,7 @@ int netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_contrwait_on_bit (&wreq->flags,NETFS_RREQ_IN_PROGRESS ,TASK_UNINTERRUPTIBLE ); ret = wreq->error; } -netfs_put_request (wreq, false, netfs_rreq_trace_put_return); +netfs_put_request (wreq, netfs_rreq_trace_put_return);return ret; }
@@ -885,7 +886,7 @@ int netfs_writeback_single(struct address_space *mapping, goto couldnt_start; } -trace_netfs_write (wreq, netfs_write_trace_writeback); +trace_netfs_write (wreq, netfs_write_trace_writeback_single);netfs_stat (&netfs_n_wh_writepages);if (__test_and_set_bit (NETFS_RREQ_UPLOAD_TO_SERVER , &wreq->flags))
@@ -914,8 +915,9 @@ int netfs_writeback_single(struct address_space *mapping,set_bit (NETFS_RREQ_ALL_QUEUED , &wreq->flags);mutex_unlock (&ictx->wb_lock); +netfs_wake_write_collector (wreq); -netfs_put_request (wreq, false, netfs_rreq_trace_put_return); +netfs_put_request (wreq, netfs_rreq_trace_put_return);_leave (" = %d" , ret);return ret;
fs/netfs/write_retry.c +1 -1
@@ -132,7 +132,7 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq, &stream->subrequests, rreq_link) {trace_netfs_sreq (subreq, netfs_sreq_trace_discard);list_del (&subreq->rreq_link); -netfs_put_subrequest (subreq, false, netfs_sreq_trace_put_done); +netfs_put_subrequest (subreq, netfs_sreq_trace_put_done);if (subreq == to)break ; }
fs/smb/client/cifsproto.h +1 -2
@@ -151,8 +151,7 @@ extern bool is_size_safe_to_change(struct cifsInodeInfo *cifsInode, __u64 eof, bool from_readdir);extern void cifs_update_eof (struct cifsInodeInfo *cifsi,loff_t offset,unsigned int bytes_written); -void cifs_write_subrequest_terminated (struct cifs_io_subrequest *wdata,ssize_t result, -bool was_async); +void cifs_write_subrequest_terminated (struct cifs_io_subrequest *wdata,ssize_t result);extern struct cifsFileInfo *find_writable_file (struct cifsInodeInfo *,int ); extern int cifs_get_writable_file(struct cifsInodeInfo *cifs_inode, int flags,
fs/smb/client/cifssmb.c +2 -2
@@ -1725,7 +1725,7 @@ cifs_writev_callback(struct mid_q_entry *mid) server->credits, server->in_flight, 0, cifs_trace_rw_credits_write_response_clear); wdata->credits.value =0 ; -cifs_write_subrequest_terminated (wdata, result, true); +cifs_write_subrequest_terminated (wdata, result);release_mid (mid); trace_smb3_rw_credits(credits.rreq_debug_id, credits.rreq_debug_index, 0, server->credits, server->in_flight,
@@ -1813,7 +1813,7 @@ cifs_async_writev(struct cifs_io_subrequest *wdata) out:if (rc) {add_credits_and_wake_if (wdata->server, &wdata->credits,0 ); -cifs_write_subrequest_terminated (wdata, rc, false); +cifs_write_subrequest_terminated (wdata, rc); } }
fs/smb/client/file.c +3 -4
@@ -130,7 +130,7 @@ static void cifs_issue_write(struct netfs_io_subrequest *subreq)else trace_netfs_sreq (subreq, netfs_sreq_trace_fail);add_credits_and_wake_if (wdata->server, &wdata->credits,0 ); -cifs_write_subrequest_terminated (wdata, rc, false); +cifs_write_subrequest_terminated (wdata, rc); goto out; }
@@ -2423,8 +2423,7 @@ int cifs_lock(struct file *file, int cmd, struct file_lock *flock)return rc; } -void cifs_write_subrequest_terminated (struct cifs_io_subrequest *wdata,ssize_t result, -bool was_async) +void cifs_write_subrequest_terminated (struct cifs_io_subrequest *wdata,ssize_t result) {struct netfs_io_request *wreq = wdata->rreq;struct netfs_inode *ictx =netfs_inode (wreq->inode);
@@ -2441,7 +2440,7 @@ void cifs_write_subrequest_terminated(struct cifs_io_subrequest *wdata, ssize_tnetfs_resize_file (ictx, wrend, true); } -netfs_write_subrequest_terminated (&wdata->subreq, result, was_async); +netfs_write_subrequest_terminated (&wdata->subreq, result); } struct cifsFileInfo *find_readable_file(struct cifsInodeInfo *cifs_inode,
fs/smb/client/smb2pdu.c +2 -2
@@ -4888,7 +4888,7 @@ smb2_writev_callback(struct mid_q_entry *mid) 0, cifs_trace_rw_credits_write_response_clear); wdata->credits.value =0 ;trace_netfs_sreq (&wdata->subreq, netfs_sreq_trace_io_progress); -cifs_write_subrequest_terminated (wdata, result ?: written, true); +cifs_write_subrequest_terminated (wdata, result ?: written);release_mid (mid); trace_smb3_rw_credits(rreq_debug_id, subreq_debug_index, 0, server->credits, server->in_flight,
@@ -5061,7 +5061,7 @@ smb2_async_writev(struct cifs_io_subrequest *wdata) -(int )wdata->credits.value, cifs_trace_rw_credits_write_response_clear);add_credits_and_wake_if (wdata->server, &wdata->credits,0 ); -cifs_write_subrequest_terminated (wdata, rc, true); +cifs_write_subrequest_terminated (wdata, rc); } }
include/linux/fscache.h +1 -1
@@ -628,7 +628,7 @@ static inline void fscache_write_to_cache(struct fscache_cookie *cookie, term_func, term_func_priv, using_pgpriv2, caching); else if (term_func) - term_func(term_func_priv, -ENOBUFS, false); + term_func(term_func_priv, -ENOBUFS); }
include/linux/netfs.h +6 -7
@@ -51,8 +51,7 @@ enum netfs_io_source {NETFS_INVALID_WRITE , } __mode(byte); -typedef void (*netfs_io_terminated_t )(void *priv,ssize_t transferred_or_error, -bool was_async); +typedef void (*netfs_io_terminated_t )(void *priv,ssize_t transferred_or_error);/* * Per-inode context. This wraps the VFS inode.
@@ -223,9 +222,10 @@ enum netfs_io_origin {*/ struct netfs_io_request {union { -struct work_struct work; +struct work_struct cleanup_work;/* Deferred cleanup work */ struct rcu_head rcu; }; +struct work_struct work;/* Result collector work */ struct inode *inode;/* The file being accessed */ struct address_space *mapping;/* The mapping being accessed */ struct kiocb *iocb;/* AIO completion vector */
@@ -270,7 +270,7 @@ struct netfs_io_request {#define NETFS_RREQ_NO_UNLOCK_FOLIO 2/* Don't unlock no_unlock_folio on completion */ #define NETFS_RREQ_DONT_UNLOCK_FOLIOS 3/* Don't unlock the folios on completion */ #define NETFS_RREQ_FAILED 4/* The request failed */ -#define NETFS_RREQ_IN_PROGRESS 5/* Unlocked when the request completes */ +#define NETFS_RREQ_IN_PROGRESS 5/* Unlocked when the request completes (has ref) */ #define NETFS_RREQ_FOLIO_COPY_TO_CACHE 6/* Copy current folio to cache from read */ #define NETFS_RREQ_UPLOAD_TO_SERVER 8/* Need to write to the server */ #define NETFS_RREQ_NONBLOCK 9/* Don't block if possible (O_NONBLOCK) */
@@ -440,15 +440,14 @@ void netfs_read_subreq_terminated(struct netfs_io_subrequest *subreq);void netfs_get_subrequest (struct netfs_io_subrequest *subreq,enum netfs_sreq_ref_trace what);void netfs_put_subrequest (struct netfs_io_subrequest *subreq, -bool was_async,enum netfs_sreq_ref_trace what); +enum netfs_sreq_ref_trace what);ssize_t netfs_extract_user_iter (struct iov_iter *orig,size_t orig_len,struct iov_iter *new,iov_iter_extraction_t extraction_flags);size_t netfs_limit_iter (const struct iov_iter *iter,size_t start_offset,size_t max_size,size_t max_segs);void netfs_prepare_write_failed (struct netfs_io_subrequest *subreq); -void netfs_write_subrequest_terminated (void *_op,ssize_t transferred_or_error, -bool was_async); +void netfs_write_subrequest_terminated (void *_op,ssize_t transferred_or_error);void netfs_queue_write_request (struct netfs_io_subrequest *subreq);int netfs_start_io_read (struct inode *inode);
include/trace/events/netfs.h +3 -4
@@ -30,6 +30,7 @@ EM(netfs_write_trace_dio_write, "DIO-WRITE") \ EM(netfs_write_trace_unbuffered_write, "UNB-WRITE") \ EM(netfs_write_trace_writeback, "WRITEBACK") \ + EM(netfs_write_trace_writeback_single, "WB-SINGLE") \ E_(netfs_write_trace_writethrough, "WRITETHRU")#define netfs_rreq_origins \
@@ -128,17 +129,15 @@#define netfs_rreq_ref_traces \ EM(netfs_rreq_trace_get_for_outstanding,"GET OUTSTND") \ EM(netfs_rreq_trace_get_subreq, "GET SUBREQ ") \ - EM(netfs_rreq_trace_get_work, "GET WORK ") \ EM(netfs_rreq_trace_put_complete, "PUT COMPLT ") \ EM(netfs_rreq_trace_put_discard, "PUT DISCARD") \ EM(netfs_rreq_trace_put_failed, "PUT FAILED ") \ EM(netfs_rreq_trace_put_no_submit, "PUT NO-SUBM") \ EM(netfs_rreq_trace_put_return, "PUT RETURN ") \ EM(netfs_rreq_trace_put_subreq, "PUT SUBREQ ") \ - EM(netfs_rreq_trace_put_work, "PUT WORK ") \ - EM(netfs_rreq_trace_put_work_complete, "PUT WORK CP") \ - EM(netfs_rreq_trace_put_work_nq, "PUT WORK NQ") \ + EM(netfs_rreq_trace_put_work_ip, "PUT WORK IP ") \ EM(netfs_rreq_trace_see_work, "SEE WORK ") \ + EM(netfs_rreq_trace_see_work_complete, "SEE WORK CP") \ E_(netfs_rreq_trace_new, "NEW ")#define netfs_sreq_ref_traces \
net/9p/client.c +3 -3
@@ -1704,7 +1704,7 @@ p9_client_write_subreq(struct netfs_io_subrequest *subreq) start, len, &subreq->io_iter); }if (IS_ERR (req)) { -netfs_write_subrequest_terminated (subreq,PTR_ERR (req), false); +netfs_write_subrequest_terminated (subreq,PTR_ERR (req));return ; }
@@ -1712,7 +1712,7 @@ p9_client_write_subreq(struct netfs_io_subrequest *subreq)if (err) {trace_9p_protocol_dump (clnt, &req->rc);p9_req_put (clnt, req); -netfs_write_subrequest_terminated (subreq, err, false); +netfs_write_subrequest_terminated (subreq, err);return ; }
@@ -1724,7 +1724,7 @@ p9_client_write_subreq(struct netfs_io_subrequest *subreq)p9_debug (P9_DEBUG_9P ,"<<< RWRITE count %d\n" , len);p9_req_put (clnt, req); -netfs_write_subrequest_terminated (subreq, written, false); +netfs_write_subrequest_terminated (subreq, written); }EXPORT_SYMBOL (p9_client_write_subreq);
David Howells
maintainerPATCH 4/4 netfs: Fix wait/wake to be consistent about the waitqueue used
In reply to David HowellsFix further inconsistencies in the use of waitqueues (clear_and_wake_up_bit() vs private waitqueue).
Move some of this stuff from the read and write sides into common code so that it can be done in fewer places.
To make this work, async I/O needs to set NETFS_RREQ_OFFLOAD_COLLECTION to indicate that a workqueue will do the collecting and places that call the wait function need to deal with it returning the amount transferred.
--- fs/netfs/buffered_read.c | 2 +- fs/netfs/buffered_write.c | 2 +- fs/netfs/direct_read.c | 4 +- fs/netfs/direct_write.c | 10 +- fs/netfs/internal.h | 33 ++++-- fs/netfs/misc.c | 218 ++++++++++++++++++++++++++++++++++++++ fs/netfs/read_collect.c | 139 +----------------------- fs/netfs/read_retry.c | 24 +---- fs/netfs/write_collect.c | 36 ++----- fs/netfs/write_issue.c | 28 +++-- fs/netfs/write_retry.c | 12 +-- 11 files changed, 284 insertions(+), 224 deletions(-)
fs/netfs/buffered_read.c +1 -1
@@ -312,7 +312,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)if (unlikely (size >0 )) {smp_wmb ();/* Write lists before ALL_QUEUED. */ set_bit (NETFS_RREQ_ALL_QUEUED , &rreq->flags); -netfs_wake_read_collector (rreq); +netfs_wake_collector (rreq); }/* Defer error return as we may need to wait for outstanding I/O. */
fs/netfs/buffered_write.c +1 -1
@@ -386,7 +386,7 @@ ssize_t netfs_perform_write(struct kiocb *iocb, struct iov_iter *iter,wbc_detach_inode (&wbc);if (ret2 == -EIOCBQUEUED )return ret2; -if (ret ==0 ) +if (ret ==0 && ret2 <0 ) ret = ret2; }
fs/netfs/direct_read.c +2 -2
@@ -103,7 +103,7 @@ static int netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq) rreq->netfs_ops->issue_read (subreq);if (test_bit (NETFS_RREQ_PAUSE , &rreq->flags)) -netfs_wait_for_pause (rreq); +netfs_wait_for_paused_read (rreq);if (test_bit (NETFS_RREQ_FAILED , &rreq->flags))break ; if (test_bit(NETFS_RREQ_BLOCKED, &rreq->flags) &&
@@ -115,7 +115,7 @@ static int netfs_dispatch_unbuffered_reads(struct netfs_io_request *rreq)if (unlikely (size >0 )) {smp_wmb ();/* Write lists before ALL_QUEUED. */ set_bit (NETFS_RREQ_ALL_QUEUED , &rreq->flags); -netfs_wake_read_collector (rreq); +netfs_wake_collector (rreq); }return ret;
fs/netfs/direct_write.c +4 -6
@@ -87,6 +87,8 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter * }__set_bit (NETFS_RREQ_USE_IO_ITER , &wreq->flags); +if (async) +__set_bit (NETFS_RREQ_OFFLOAD_COLLECTION , &wreq->flags);/* Copy the data into the bounce buffer and encrypt it. */ // TODO
@@ -105,13 +107,9 @@ ssize_t netfs_unbuffered_write_iter_locked(struct kiocb *iocb, struct iov_iter *if (!async) {trace_netfs_rreq (wreq, netfs_rreq_trace_wait_ip); -wait_on_bit (&wreq->flags,NETFS_RREQ_IN_PROGRESS , -TASK_UNINTERRUPTIBLE ); - ret = wreq->error; -if (ret ==0 ) { - ret = wreq->transferred; + ret =netfs_wait_for_write (wreq); +if (ret >0 ) iocb->ki_pos += ret; - } }else { ret = -EIOCBQUEUED ; }
fs/netfs/internal.h +27 -6
@@ -62,6 +62,14 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}struct folio_queue *netfs_buffer_make_space (struct netfs_io_request *rreq,enum netfs_folioq_trace trace);void netfs_reset_iter (struct netfs_io_subrequest *subreq); +void netfs_wake_collector (struct netfs_io_request *rreq); +void netfs_subreq_clear_in_progress (struct netfs_io_subrequest *subreq); +void netfs_wait_for_in_progress_stream (struct netfs_io_request *rreq, +struct netfs_io_stream *stream); +ssize_t netfs_wait_for_read (struct netfs_io_request *rreq); +ssize_t netfs_wait_for_write (struct netfs_io_request *rreq); +void netfs_wait_for_paused_read (struct netfs_io_request *rreq); +void netfs_wait_for_paused_write (struct netfs_io_request *rreq);/* * objects.c
@@ -91,11 +99,9 @@ static inline void netfs_see_subrequest(struct netfs_io_subrequest *subreq,/* * read_collect.c */ +bool netfs_read_collection (struct netfs_io_request *rreq);void netfs_read_collection_worker (struct work_struct *work); -void netfs_wake_read_collector (struct netfs_io_request *rreq);void netfs_cache_read_terminated (void *priv,ssize_t transferred_or_error); -ssize_t netfs_wait_for_read (struct netfs_io_request *rreq); -void netfs_wait_for_pause (struct netfs_io_request *rreq);/* * read_pgpriv2.c
@@ -175,8 +181,8 @@ static inline void netfs_stat_d(atomic_t *stat)* write_collect.c */ int netfs_folio_written_back (struct folio *folio); +bool netfs_write_collection (struct netfs_io_request *wreq);void netfs_write_collection_worker (struct work_struct *work); -void netfs_wake_write_collector (struct netfs_io_request *wreq);/* * write_issue.c
@@ -197,8 +203,8 @@ struct netfs_io_request *netfs_begin_writethrough(struct kiocb *iocb, size_t lenint netfs_advance_writethrough (struct netfs_io_request *wreq,struct writeback_control *wbc,struct folio *folio,size_t copied,bool to_page_end,struct folio **writethrough_cache); -int netfs_end_writethrough (struct netfs_io_request *wreq,struct writeback_control *wbc, -struct folio *writethrough_cache); +ssize_t netfs_end_writethrough (struct netfs_io_request *wreq,struct writeback_control *wbc, +struct folio *writethrough_cache);int netfs_unbuffered_write (struct netfs_io_request *wreq,bool may_wait,size_t len);/*
@@ -253,6 +259,21 @@ static inline void netfs_put_group_many(struct netfs_group *netfs_group, int nr) netfs_group->free (netfs_group); } +/* +* Clear and wake up a NETFS_RREQ_* flag bit on a request. +*/ +static inline void netfs_wake_rreq_flag (struct netfs_io_request *rreq, +unsigned int rreq_flag, +enum netfs_rreq_trace trace) +{ +if (test_bit (rreq_flag, &rreq->flags)) { +trace_netfs_rreq (rreq, trace); +clear_bit_unlock (rreq_flag, &rreq->flags); +smp_mb__after_atomic ();/* Set flag before task state */ +wake_up (&rreq->waitq); + } +} +/* * fscache-cache.c */
fs/netfs/misc.c +218 -0
221 lines not on this page · raw message
fs/netfs/read_collect.c +5 -134
185 lines not on this page · raw message
fs/netfs/read_retry.c +2 -22
@@ -257,35 +257,15 @@ static void netfs_retry_read_subrequests(struct netfs_io_request *rreq)*/ void netfs_retry_reads (struct netfs_io_request *rreq) { -struct netfs_io_subrequest *subreq;struct netfs_io_stream *stream = &rreq->io_streams[0 ]; -DEFINE_WAIT (myself);netfs_stat (&netfs_n_rh_retry_read_req); -set_bit (NETFS_RREQ_RETRYING , &rreq->flags); -/* Wait for all outstanding I/O to quiesce before performing retries as * we may need to renegotiate the I/O sizes. */ -list_for_each_entry (subreq, &stream->subrequests, rreq_link) { -if (!test_bit (NETFS_SREQ_IN_PROGRESS , &subreq->flags)) -continue ; - -trace_netfs_rreq (rreq, netfs_rreq_trace_wait_queue); -for (;;) { -prepare_to_wait (&rreq->waitq, &myself,TASK_UNINTERRUPTIBLE ); - -if (!test_bit (NETFS_SREQ_IN_PROGRESS , &subreq->flags)) -break ; - -trace_netfs_sreq (subreq, netfs_sreq_trace_wait_for); -schedule (); -trace_netfs_rreq (rreq, netfs_rreq_trace_woke_queue); - } - -finish_wait (&rreq->waitq, &myself); - } +set_bit (NETFS_RREQ_RETRYING , &rreq->flags); +netfs_wait_for_in_progress_stream (rreq, stream);clear_bit (NETFS_RREQ_RETRYING , &rreq->flags);trace_netfs_rreq (rreq, netfs_rreq_trace_resubmit);
fs/netfs/write_collect.c +7 -29
@@ -321,18 +321,14 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)if (notes &NEED_RETRY ) goto need_retry; -if ((notes &MADE_PROGRESS ) &&test_bit (NETFS_RREQ_PAUSE , &wreq->flags)) { -trace_netfs_rreq (wreq, netfs_rreq_trace_unpause); -clear_bit_unlock (NETFS_RREQ_PAUSE , &wreq->flags); -smp_mb__after_atomic ();/* Set PAUSE before task state */ -wake_up (&wreq->waitq); - } -if (notes &NEED_REASSESS ) { +if (notes &MADE_PROGRESS ) { +netfs_wake_rreq_flag (wreq,NETFS_RREQ_PAUSE , netfs_rreq_trace_unpause);//cond_resched(); goto reassess_streams; } -if (notes &MADE_PROGRESS ) { + +if (notes &NEED_REASSESS ) {//cond_resched(); goto reassess_streams; }
@@ -356,7 +352,7 @@ static void netfs_collect_write_results(struct netfs_io_request *wreq)/* * Perform the collection of subrequests, folios and encryption buffers. */ -static bool netfs_write_collection (struct netfs_io_request *wreq) +bool netfs_write_collection (struct netfs_io_request *wreq) {struct netfs_inode *ictx =netfs_inode (wreq->inode);size_t transferred;
@@ -417,8 +413,7 @@ static bool netfs_write_collection(struct netfs_io_request *wreq)inode_dio_end (wreq->inode);_debug ("finished" ); -trace_netfs_rreq (wreq, netfs_rreq_trace_wake_ip); -clear_and_wake_up_bit (NETFS_RREQ_IN_PROGRESS , &wreq->flags); +netfs_wake_rreq_flag (wreq,NETFS_RREQ_IN_PROGRESS , netfs_rreq_trace_wake_ip);/* As we cleared NETFS_RREQ_IN_PROGRESS, we acquired its ref. */ if (wreq->iocb) {
@@ -448,14 +443,6 @@ void netfs_write_collection_worker(struct work_struct *work) } } -/* -* Wake the collection work item. -*/ -void netfs_wake_write_collector (struct netfs_io_request *wreq) -{ -queue_work (system_unbound_wq, &wreq->work); -} -/** * netfs_write_subrequest_terminated - Note the termination of a write operation. * @_op: The I/O request that has terminated.
@@ -479,7 +466,6 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error) {struct netfs_io_subrequest *subreq = _op;struct netfs_io_request *wreq = subreq->rreq; -struct netfs_io_stream *stream = &wreq->io_streams[subreq->stream_nr];_enter ("%x[%x] %zd" , wreq->debug_id, subreq->debug_index, transferred_or_error);
@@ -531,15 +517,7 @@ void netfs_write_subrequest_terminated(void *_op, ssize_t transferred_or_error) }trace_netfs_sreq (subreq, netfs_sreq_trace_terminated); - -clear_and_wake_up_bit (NETFS_SREQ_IN_PROGRESS , &subreq->flags); - -/* If we are at the head of the queue, wake up the collector, -* transferring a ref to it if we were the ones to do so. -*/ -if (list_is_first (&subreq->rreq_link, &stream->subrequests)) -netfs_wake_write_collector (wreq); - +netfs_subreq_clear_in_progress (subreq);netfs_put_subrequest (subreq, netfs_sreq_trace_put_terminated); }EXPORT_SYMBOL (netfs_write_subrequest_terminated);
fs/netfs/write_issue.c +13 -15
@@ -542,7 +542,7 @@ static void netfs_end_issue_write(struct netfs_io_request *wreq) }if (needs_poke) -netfs_wake_write_collector (wreq); +netfs_wake_collector (wreq); }/*
@@ -576,6 +576,7 @@ int netfs_writepages(struct address_space *mapping, goto couldnt_start; } +__set_bit (NETFS_RREQ_OFFLOAD_COLLECTION , &wreq->flags);trace_netfs_write (wreq, netfs_write_trace_writeback);netfs_stat (&netfs_n_wh_writepages);
@@ -599,7 +600,7 @@ int netfs_writepages(struct address_space *mapping,netfs_end_issue_write (wreq);mutex_unlock (&ictx->wb_lock); -netfs_wake_write_collector (wreq); +netfs_wake_collector (wreq);netfs_put_request (wreq, netfs_rreq_trace_put_return);_leave (" = %d" , error);
@@ -674,11 +675,11 @@ int netfs_advance_writethrough(struct netfs_io_request *wreq, struct writeback_c/* * End a write operation used when writing through the pagecache. */ -int netfs_end_writethrough (struct netfs_io_request *wreq,struct writeback_control *wbc, -struct folio *writethrough_cache) +ssize_t netfs_end_writethrough (struct netfs_io_request *wreq,struct writeback_control *wbc, +struct folio *writethrough_cache) {struct netfs_inode *ictx =netfs_inode (wreq->inode); -int ret; +ssize_t ret;_enter ("R=%x" , wreq->debug_id);
@@ -689,12 +690,10 @@ int netfs_end_writethrough(struct netfs_io_request *wreq, struct writeback_contrmutex_unlock (&ictx->wb_lock); -if (wreq->iocb) { +if (wreq->iocb) ret = -EIOCBQUEUED ; - }else { -wait_on_bit (&wreq->flags,NETFS_RREQ_IN_PROGRESS ,TASK_UNINTERRUPTIBLE ); - ret = wreq->error; - } +else + ret =netfs_wait_for_write (wreq);netfs_put_request (wreq, netfs_rreq_trace_put_return);return ret; }
@@ -723,10 +722,8 @@ int netfs_unbuffered_write(struct netfs_io_request *wreq, bool may_wait, size_t start += part; len -= part;rolling_buffer_advance (&wreq->buffer, part); -if (test_bit (NETFS_RREQ_PAUSE , &wreq->flags)) { -trace_netfs_rreq (wreq, netfs_rreq_trace_wait_pause); -wait_event (wreq->waitq, !test_bit (NETFS_RREQ_PAUSE , &wreq->flags)); - } +if (test_bit (NETFS_RREQ_PAUSE , &wreq->flags)) +netfs_wait_for_paused_write (wreq);if (test_bit (NETFS_RREQ_FAILED , &wreq->flags))break ; }
@@ -886,6 +883,7 @@ int netfs_writeback_single(struct address_space *mapping, goto couldnt_start; } +__set_bit (NETFS_RREQ_OFFLOAD_COLLECTION , &wreq->flags);trace_netfs_write (wreq, netfs_write_trace_writeback_single);netfs_stat (&netfs_n_wh_writepages);
@@ -915,7 +913,7 @@ int netfs_writeback_single(struct address_space *mapping,set_bit (NETFS_RREQ_ALL_QUEUED , &wreq->flags);mutex_unlock (&ictx->wb_lock); -netfs_wake_write_collector (wreq); +netfs_wake_collector (wreq);netfs_put_request (wreq, netfs_rreq_trace_put_return);_leave (" = %d" , ret);
fs/netfs/write_retry.c +4 -8
@@ -200,7 +200,6 @@ static void netfs_retry_write_stream(struct netfs_io_request *wreq,*/ void netfs_retry_writes (struct netfs_io_request *wreq) { -struct netfs_io_subrequest *subreq;struct netfs_io_stream *stream;int s;
@@ -209,16 +208,13 @@ void netfs_retry_writes(struct netfs_io_request *wreq)/* Wait for all outstanding I/O to quiesce before performing retries as * we may need to renegotiate the I/O sizes. */ +set_bit (NETFS_RREQ_RETRYING , &wreq->flags);for (s =0 ; s <NR_IO_STREAMS ; s++) { stream = &wreq->io_streams[s]; -if (!stream->active) -continue ; - -list_for_each_entry (subreq, &stream->subrequests, rreq_link) { -wait_on_bit (&subreq->flags,NETFS_SREQ_IN_PROGRESS , -TASK_UNINTERRUPTIBLE ); - } +if (stream->active) +netfs_wait_for_in_progress_stream (wreq, stream); } +clear_bit (NETFS_RREQ_RETRYING , &wreq->flags);// TODO: Enc: Fetch changed partial pages // TODO: Enc: Reencrypt content if needed.
Quoted 8 lines · David Howells
Here are some miscellaneous fixes and changes for netfslib, if you could pull them:
(1) Fix an oops in write-retry due to mis-resetting the I/O iterator.
(2) Fix the recording of transferred bytes for short DIO reads.
[...]
Applied to the vfs-6.16.netfs branch of the vfs/vfs.git tree. Patches in the vfs-6.16.netfs branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.16.netfs
[1/4] netfs: Fix oops in write-retry from mis-resetting the subreq iterator
https://git.kernel.org/vfs/vfs/c/cd084c7184ce [2/4] netfs: Fix setting of transferred bytes with short DIO reads https://git.kernel.org/vfs/vfs/c/2973904c9b79 [3/4] netfs: Fix the request's work item to not require a ref https://git.kernel.org/vfs/vfs/c/537b296114cc [4/4] netfs: Fix wait/wake to be consistent about the waitqueue used https://git.kernel.org/vfs/vfs/c/2a6d0284a4a3