Skip to contentSkip to conversation

NFS: fix writeback in presence of errors

RFC PATCH Superseded Olga Kornievskaia 2 participants 2 messages Mar 12 – Mar 24

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Olga Kornievskaia

author reviewer
#0 lore raw

RFC PATCH 1/1 NFS: fix writeback in presence of errors

After running xfstest generic/751, in certain conditions, can have a writeback IO stuck while experiencing one of the two patterns.

Pattern#1: writeback IO experiences ENOSPC on an offset smaller than the filesize. Example,
write offset=0 len=4096 how=unstable OK
write offset=8192 len=4096 how=unstable OK
write offset=12288 len=4096 how=unstable ENOSPC
write offset=4096 len=4096 how=unstable ENOSPC
client sends a commit and receives a verifier which is different from the last successful write. It marks pages dirty and writeback retries. But it again send writes unstable and gets into the same pattern, running into the ENOSPC error and sending a commit because writes were sent at unstable.

Pattern#2: an unstable write followed by a short write and ENOSPC. write offset=0 len=4096 how=unstable OK
write offset=4096 len=4096 how=unstable returns OK but count=100 write offset=4197 len=3996 how=stable returns ENOSPC client send a commit and receives a verifier different from the last unstable write. The same behaviour is retried in a loop.

Instead, this patch proposes to identify those conditions and mark requests to be done synchronously instead.

---
 fs/nfs/pagelist.c        | 2 ++
 fs/nfs/write.c           | 2 ++
 include/linux/nfs_page.h | 1 +
 3 files changed, 5 insertions(+)
fs/nfs/pagelist.c +2 -0
index a9373de891c9..b835b197a223 100644
@@ -1186,6 +1186,8 @@ static int __nfs_pageio_add_request(struct nfs_pageio_descriptor *desc,
 
 	nfs_page_group_lock(req);
 
+	if (test_bit(PG_SYNC, &req->wb_flags))
+		desc->pg_ioflags |= FLUSH_STABLE;
 	subreq = req;
 	subreq_size = subreq->wb_bytes;
 	for(;;) {
fs/nfs/write.c +2 -0
index 1ed4b3590b1a..78f412851d59 100644
@@ -1554,6 +1554,7 @@ static void nfs_writeback_result(struct rpc_task *task,
 	if (resp->count < argp->count) {
 		static unsigned long    complain;
 
+		set_bit(PG_SYNC, &hdr->req->wb_flags);
 		/* This a short write! */
 		nfs_inc_stats(hdr->inode, NFSIOS_SHORTWRITE);
 
@@ -1837,6 +1838,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data)
 		/* We have a mismatch. Write the page again */
 		dprintk(" mismatch\n");
 		nfs_mark_request_dirty(req);
+		set_bit(PG_SYNC, &req->wb_flags);
 		atomic_long_inc(&NFS_I(data->inode)->redirtied_pages);
 	next:
 		nfs_unlock_and_release_request(req);
include/linux/nfs_page.h +1 -0
index afe1d8f09d89..e28599bebd44 100644
@@ -37,6 +37,7 @@ enum {
 	PG_REMOVE,		/* page group sync bit in write path */
 	PG_CONTENDED1,		/* Is someone waiting for a lock? */
 	PG_CONTENDED2,		/* Is someone waiting for a lock? */
+	PG_SYNC,		/* writeback must write stable */
 };
 
 struct nfs_inode;
Signature · 1 line
2.52.0
#1 lore raw
Quoted 4 lines · Olga Kornievskaia

After running xfstest generic/751, in certain conditions, can have a writeback IO stuck while experiencing one of the two patterns.

Quoted 11 lines

Pattern#1: writeback IO experiences ENOSPC on an offset smaller than the filesize. Example,
write offset=0 len=4096 how=unstable OK
write offset=8192 len=4096 how=unstable OK
write offset=12288 len=4096 how=unstable ENOSPC
write offset=4096 len=4096 how=unstable ENOSPC
client sends a commit and receives a verifier which is different from the last successful write. It marks pages dirty and writeback retries. But it again send writes unstable and gets into the same pattern, running into the ENOSPC error and sending a commit because writes were sent at unstable.

The patch doesn't actually fix this pattern. I mistakenly thought it was. I continue working on trying to find the solution but it looks like the same approach does not work (ie marking the request in some way so that the next time writeback happens it does so sync doesn't work as each time writeback creates a new request and thus the requests that was marked is not useful).

Quoted 6 lines

Pattern#2: an unstable write followed by a short write and ENOSPC. write offset=0 len=4096 how=unstable OK
write offset=4096 len=4096 how=unstable returns OK but count=100 write offset=4197 len=3996 how=stable returns ENOSPC client send a commit and receives a verifier different from the last unstable write. The same behaviour is retried in a loop.

This pattern the patch does fix though.

Quoted 60 lines

Instead, this patch proposes to identify those conditions and mark requests to be done synchronously instead.

---
 fs/nfs/pagelist.c        | 2 ++
 fs/nfs/write.c           | 2 ++
 include/linux/nfs_page.h | 1 +
 3 files changed, 5 insertions(+)
fs/nfs/pagelist.c +2 -0
index a9373de891c9..b835b197a223 100644
@@ -1186,6 +1186,8 @@ static int __nfs_pageio_add_request(struct nfs_pageio_descriptor *desc,
 
        nfs_page_group_lock(req);
 
+       if (test_bit(PG_SYNC, &req->wb_flags))
+               desc->pg_ioflags |= FLUSH_STABLE;
        subreq = req;
        subreq_size = subreq->wb_bytes;
        for(;;) {
fs/nfs/write.c +2 -0
index 1ed4b3590b1a..78f412851d59 100644
@@ -1554,6 +1554,7 @@ static void nfs_writeback_result(struct rpc_task *task,
        if (resp->count < argp->count) {
                static unsigned long    complain;
 
+               set_bit(PG_SYNC, &hdr->req->wb_flags);
                /* This a short write! */
                nfs_inc_stats(hdr->inode, NFSIOS_SHORTWRITE);
 
@@ -1837,6 +1838,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data)
                /* We have a mismatch. Write the page again */
                dprintk(" mismatch\n");
                nfs_mark_request_dirty(req);
+               set_bit(PG_SYNC, &req->wb_flags);
                atomic_long_inc(&NFS_I(data->inode)->redirtied_pages);
        next:
                nfs_unlock_and_release_request(req);
include/linux/nfs_page.h +1 -1
index afe1d8f09d89..e28599bebd44 100644
@@ -37,6 +37,7 @@ enum {
        PG_REMOVE,              /* page group sync bit in write path */
        PG_CONTENDED1,          /* Is someone waiting for a lock? */
        PG_CONTENDED2,          /* Is someone waiting for a lock? */
+       PG_SYNC,                /* writeback must write stable */
 };
 
 struct nfs_inode;
--

2.52.0