NFS: fix writeback in presence of errors
Subsystems (2)
Olga Kornievskaia
reviewerRFC PATCH 1/1 NFS: fix writeback in presence of errors
After running xfstest generic/751, in certain conditions, can have a writeback IO stuck while experiencing one of the two patterns.
Pattern#1: writeback IO experiences ENOSPC on an offset smaller than the filesize. Example,
write offset=0 len=4096 how=unstable OK
write offset=8192 len=4096 how=unstable OK
write offset=12288 len=4096 how=unstable ENOSPC
write offset=4096 len=4096 how=unstable ENOSPC
client sends a commit and receives a verifier which is different from the last successful write. It marks pages dirty and writeback retries. But it again send writes unstable and gets into the same pattern, running into the ENOSPC error and sending a commit because writes were sent at unstable.
Pattern#2: an unstable write followed by a short write and ENOSPC. write offset=0 len=4096 how=unstable OK
write offset=4096 len=4096 how=unstable returns OK but count=100 write offset=4197 len=3996 how=stable returns ENOSPC client send a commit and receives a verifier different from the last unstable write. The same behaviour is retried in a loop.
Instead, this patch proposes to identify those conditions and mark requests to be done synchronously instead.
--- fs/nfs/pagelist.c | 2 ++ fs/nfs/write.c | 2 ++ include/linux/nfs_page.h | 1 + 3 files changed, 5 insertions(+)
fs/nfs/pagelist.c +2 -0
@@ -1186,6 +1186,8 @@ static int __nfs_pageio_add_request(struct nfs_pageio_descriptor *desc,nfs_page_group_lock (req); +if (test_bit (PG_SYNC , &req->wb_flags)) + desc->pg_ioflags |=FLUSH_STABLE ; subreq = req; subreq_size = subreq->wb_bytes;for (;;) {
fs/nfs/write.c +2 -0
@@ -1554,6 +1554,7 @@ static void nfs_writeback_result(struct rpc_task *task,if (resp->count < argp->count) {static unsigned long complain; +set_bit (PG_SYNC , &hdr->req->wb_flags);/* This a short write! */ nfs_inc_stats (hdr->inode,NFSIOS_SHORTWRITE );
@@ -1837,6 +1838,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data)/* We have a mismatch. Write the page again */ dprintk (" mismatch\n" );nfs_mark_request_dirty (req); +set_bit (PG_SYNC , &req->wb_flags);atomic_long_inc (&NFS_I (data->inode)->redirtied_pages); next:nfs_unlock_and_release_request (req);
include/linux/nfs_page.h +1 -0
@@ -37,6 +37,7 @@ enum {PG_REMOVE ,/* page group sync bit in write path */ PG_CONTENDED1 ,/* Is someone waiting for a lock? */ PG_CONTENDED2 ,/* Is someone waiting for a lock? */ + PG_SYNC, /* writeback must write stable */ };struct nfs_inode ;
Signature · 1 line
2.52.0
Quoted 4 lines · Olga Kornievskaia
After running xfstest generic/751, in certain conditions, can have a writeback IO stuck while experiencing one of the two patterns.
Quoted 11 lines
Pattern#1: writeback IO experiences ENOSPC on an offset smaller than the filesize. Example,
write offset=0 len=4096 how=unstable OK
write offset=8192 len=4096 how=unstable OK
write offset=12288 len=4096 how=unstable ENOSPC
write offset=4096 len=4096 how=unstable ENOSPC
client sends a commit and receives a verifier which is different from the last successful write. It marks pages dirty and writeback retries. But it again send writes unstable and gets into the same pattern, running into the ENOSPC error and sending a commit because writes were sent at unstable.
The patch doesn't actually fix this pattern. I mistakenly thought it was. I continue working on trying to find the solution but it looks like the same approach does not work (ie marking the request in some way so that the next time writeback happens it does so sync doesn't work as each time writeback creates a new request and thus the requests that was marked is not useful).
Quoted 6 lines
Pattern#2: an unstable write followed by a short write and ENOSPC. write offset=0 len=4096 how=unstable OK
write offset=4096 len=4096 how=unstable returns OK but count=100 write offset=4197 len=3996 how=stable returns ENOSPC client send a commit and receives a verifier different from the last unstable write. The same behaviour is retried in a loop.
This pattern the patch does fix though.
Quoted 60 lines
Instead, this patch proposes to identify those conditions and mark requests to be done synchronously instead.
Signed-off-by Olga Kornievskaia <[email protected]>--- fs/nfs/pagelist.c | 2 ++ fs/nfs/write.c | 2 ++ include/linux/nfs_page.h | 1 + 3 files changed, 5 insertions(+)fs/nfs/pagelist.c +2
-0@@ -1186,6 +1186,8 @@ static int __nfs_pageio_add_request(struct nfs_pageio_descriptor *desc,nfs_page_group_lock (req); +if (test_bit (PG_SYNC , &req->wb_flags)) + desc->pg_ioflags |=FLUSH_STABLE ; subreq = req; subreq_size = subreq->wb_bytes;for (;;) {fs/nfs/write.c +2
-0@@ -1554,6 +1554,7 @@ static void nfs_writeback_result(struct rpc_task *task,if (resp->count < argp->count) {static unsigned long complain; +set_bit (PG_SYNC , &hdr->req->wb_flags);/* This a short write! */ nfs_inc_stats (hdr->inode,NFSIOS_SHORTWRITE );@@ -1837,6 +1838,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data)/* We have a mismatch. Write the page again */ dprintk (" mismatch\n" );nfs_mark_request_dirty (req); +set_bit (PG_SYNC , &req->wb_flags);atomic_long_inc (&NFS_I (data->inode)->redirtied_pages); next:nfs_unlock_and_release_request (req);include/linux/nfs_page.h +1
-1@@ -37,6 +37,7 @@ enum {PG_REMOVE ,/* page group sync bit in write path */ PG_CONTENDED1 ,/* Is someone waiting for a lock? */ PG_CONTENDED2 ,/* Is someone waiting for a lock? */ + PG_SYNC, /* writeback must write stable */ };struct nfs_inode ; --2.52.0