(unknown)
Subsystems (2)
This is against the pnfs-submit branch of the 2.6.34 tree. They will need to be applied against the 2.6.35-rc1 tree which I can do after comments.
RFC: I would like comments, especially on 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
Remove unused layoutcommit layoutdriver_io_operations. Will be restored in post-submit patches 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch 0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
A cleanup, and call the async error handler. 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
This next patch moves the pnfs_layoutcommit_inode call to nfs_write_inode, and it is the only call other than in layoutreturn. (removed calls in __nfs4_close, nfs_commit_inode, nfs_wb_sync).
This is fine for the file layout, and I think it's OK for the object and block layouts as well.
I left the LAYOUTCOMMIT call in nfs_write_inode a synchronous call, because nfs_commit_unstable_pages sets the FLUSH_SYNC flag. Should this be an asyc LAYOUTCOMMIT call?
pnfs_layoutcommit_inode is called after nfs_commit_unstable_pages() so that if LAYOUTCOMMIT fails, the unstable pages have been processed..
The error handlers (sync and async) call nfs4_map_errors, so unhandled errors (such as NFS4ERR_BADLAYOUT) get returned to nfs_write_ioode as -EIO.
Examining the write_inode call paths, I could not see where the -EIO would be passed back to the application. Testing with pynfs which I had return NFS4ERR_BADLAYOUT to the layout commit call, shows the -EIO return not stopping the client nor is the error reported back to the application.
We will add code to the error handlers for errors such as NFS4ERR_BADLAYOUT that require us to stop using and free the layout, and redo the I/O through the MDS.
Anyway, review is much appreciated.
0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
Testing:
With CONFIG_NFS_V4_1 set
NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. Note: there were exactly the same number of LAYOUTCOMMITS sent as were sent with pnfs_layoutcommit_inode being called from __nfs4_close (never happened), nfs_commit_inode and nfs_wb_sync.
Passed Connectathon general test against pynfs file layout server with the NFS4ERR_BADLAYOUT being returned on every third LAYOUTCOMMIT.
-->Andy
PATCH 1/6 SQUASHME pnfs-submit: remove setup_layoutcommit
From: Andy Adamson <[email protected]>
Not used by the file layout driver
--- fs/nfs/pnfs.c | 7 ------- include/linux/nfs4_pnfs.h | 2 -- 2 files changed, 0 insertions(+), 9 deletions(-)
fs/nfs/pnfs.c +0 -7
@@ -1944,7 +1944,6 @@ pnfs_layoutcommit_setup(struct inode *inode,struct pnfs_layoutcommit_data *data, loff_t write_begin_pos, loff_t write_end_pos, int sync) { - struct nfs_inode *nfsi = NFS_I(inode); struct nfs_server *nfss = NFS_SERVER(inode);int result =0 ;
@@ -1970,12 +1969,6 @@ pnfs_layoutcommit_setup(struct inode *inode, data->args.bitmask = nfss->attr_bitmask; data->res.server = nfss; -/* Call layout driver to set the arguments. -*/ -if (nfss->pnfs_curr_ld->ld_io_ops->setup_layoutcommit) - result = nfss->pnfs_curr_ld->ld_io_ops->setup_layoutcommit ( - &nfsi->layout, &data->args); -dprintk ("%s End Status %d\n" , __func__, result);return result; }
include/linux/nfs4_pnfs.h +0 -2
@@ -146,8 +146,6 @@ struct layoutdriver_io_operations {struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid,struct nfs4_pnfs_layoutget_res *lgr);void (*free_lseg) (struct pnfs_layout_segment *lseg); -int (*setup_layoutcommit) (struct pnfs_layout_type *layoutid, -struct pnfs_layoutcommit_arg *args);void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid,struct xdr_stream *xdr,const struct pnfs_layoutcommit_arg *args);
Signature · 1 line
1.6.6
PATCH 2/6 SQUASHNE pnfs-submit: remove cleanup_layoutcommit
From: Andy Adamson <[email protected]>
Not used by the file layout driver
--- fs/nfs/nfs4proc.c | 4 ++-- fs/nfs/pnfs.c | 24 ------------------------ fs/nfs/pnfs.h | 1 - include/linux/nfs4_pnfs.h | 3 --- 4 files changed, 2 insertions(+), 30 deletions(-)
fs/nfs/nfs4proc.c +2 -2
@@ -5600,7 +5600,7 @@ static int pnfs4_sync_layoutcommit(struct pnfs_layoutcommit_data *data) }while (exception.retry); data->status = err; -pnfs_layoutcommit_done (data); +put_rpccred (data->cred);pnfs_layoutcommit_free (data);return err; }
@@ -5625,7 +5625,7 @@ pnfs_layoutcommit_rpc_done(struct rpc_task *task, void *calldata)struct nfs_server *server =NFS_SERVER (data->args.inode); data->status = task->tk_status; -pnfs_layoutcommit_done (data); +put_rpccred (data->cred);nfs4_sequence_done (server, &data->res.seq_res, task->tk_status); }
fs/nfs/pnfs.c +0 -24
@@ -1912,30 +1912,6 @@ out:return trypnfs; } -/* Called on completion of layoutcommit */ -void -pnfs_layoutcommit_done (struct pnfs_layoutcommit_data *data) -{ -struct nfs_server *nfss =NFS_SERVER (data->args.inode); -struct nfs_inode *nfsi =NFS_I (data->args.inode); - -dprintk ("%s: (status %d)\n" , __func__, data->status); - -if (data->status <0 ) -printk (KERN_ERR "%s, Layoutcommit Failed! = %d\n" , - __func__, data->status); - -/* TODO: Maybe we should avoid this by allowing the layout driver -* to directly xdr its layout on the wire. -*/ -if (nfss->pnfs_curr_ld->ld_io_ops->cleanup_layoutcommit) - nfss->pnfs_curr_ld->ld_io_ops->cleanup_layoutcommit ( - &nfsi->layout, - &data->args, - data->status); -put_rpccred (data->cred); -} -/* * Set up the argument/result storage required for the RPC call. */
fs/nfs/pnfs.h +0 -1
@@ -46,7 +46,6 @@ enum pnfs_try_status _pnfs_try_to_read_data(struct nfs_read_data *, const struct rpc_call_ops *); int pnfs_initialize(void);void pnfs_uninitialize (void ); -void pnfs_layoutcommit_done (struct pnfs_layoutcommit_data *data);void pnfs_layoutcommit_free (struct pnfs_layoutcommit_data *data);int pnfs_layoutcommit_inode (struct inode *inode,int sync);void pnfs_update_last_write (struct nfs_inode *nfsi,loff_t offset,size_t extent);
include/linux/nfs4_pnfs.h +0 -3
@@ -149,9 +149,6 @@ struct layoutdriver_io_operations {void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid,struct xdr_stream *xdr,const struct pnfs_layoutcommit_arg *args); -void (*cleanup_layoutcommit) (struct pnfs_layout_type *layoutid, -struct pnfs_layoutcommit_arg *args, -int status);void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,struct xdr_stream *xdr,const struct nfs4_pnfs_layoutreturn_arg *args);
Signature · 1 line
1.6.6
PATCH 3/6 SQUASHME pnfs-submit: remove encode_layoutcommit
From: Andy Adamson <[email protected]>
Not used by the file layout driver
--- fs/nfs/nfs4xdr.c | 11 ++--------- include/linux/nfs4_pnfs.h | 3 --- 2 files changed, 2 insertions(+), 12 deletions(-)
fs/nfs/nfs4xdr.c +2 -9
@@ -1852,8 +1852,6 @@ encode_layoutcommit(struct xdr_stream *xdr,const struct pnfs_layoutcommit_arg *args, struct compound_hdr *hdr) { - struct layoutdriver_io_operations *ld_io_ops = -NFS_SERVER (args->inode)->pnfs_curr_ld->ld_io_ops; __be32 *p; dprintk("%s: %llu@%llu lbw: %llu type: %d\n", __func__,
@@ -1879,13 +1877,8 @@ encode_layoutcommit(struct xdr_stream *xdr, p =reserve_space (xdr,4 ); *p =cpu_to_be32 (args->layout_type); -if (ld_io_ops->encode_layoutcommit) - ld_io_ops->encode_layoutcommit ( - &NFS_I (args->inode)->layout, xdr, args); -else { - p =reserve_space (xdr,4 ); -xdr_encode_opaque (p,NULL ,0 ); - } + p =reserve_space (xdr,4 ); +xdr_encode_opaque (p,NULL ,0 ); hdr->nops++; hdr->replen += decode_layoutcommit_maxsz;
include/linux/nfs4_pnfs.h +0 -3
@@ -146,9 +146,6 @@ struct layoutdriver_io_operations {struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid,struct nfs4_pnfs_layoutget_res *lgr);void (*free_lseg) (struct pnfs_layout_segment *lseg); -void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid, -struct xdr_stream *xdr, -const struct pnfs_layoutcommit_arg *args);void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,struct xdr_stream *xdr,const struct nfs4_pnfs_layoutreturn_arg *args);
Signature · 1 line
1.6.6
PATCH 4/6 SQUASHME pnfs-submit: cleanup layoutcommit call
From: Andy Adamson <[email protected]>
pnfs_layoutcommit_inode prints its status upon exit.
--- fs/nfs/pnfs.c | 11 ++--------- 1 files changed, 2 insertions(+), 9 deletions(-)
fs/nfs/pnfs.c +2 -9
@@ -766,15 +766,8 @@ _pnfs_return_layout(struct inode *ino, struct nfs4_pnfs_layout_segment *range, !pnfs_return_layout_barrier(nfsi, &arg)); } -if (layoutcommit_needed (nfsi)) { - status =pnfs_layoutcommit_inode (ino, wait); -if (status) { -dprintk ("%s: layoutcommit failed, status=%d. " -"Returning layout anyway\n" , - __func__, status); - status =0 ; - } - } +if (layoutcommit_needed (nfsi)) +pnfs_layoutcommit_inode (ino, wait); } send_return: status =return_layout (ino, &arg, stateid, type, lo, wait);
Signature · 1 line
1.6.6
PATCH 5/6 SQUASHME pnfs-submit: handle async layoutcommit errors
From: Andy Adamson <[email protected]>
nfs4_async_handle_error handles session level and grace,delay errors.
NOTE: Layout specific errors (NFS4ERR_BADIOMODE, NFS4ERR_BAD_LAYOUT, NFS4ERR_UNKNOWN_LAYOUTTYPE) need to be handled.
--- fs/nfs/nfs4proc.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
fs/nfs/nfs4proc.c +3 -0
@@ -5628,6 +5628,9 @@ pnfs_layoutcommit_rpc_done(struct rpc_task *task, void *calldata)put_rpccred (data->cred);nfs4_sequence_done (server, &data->res.seq_res, task->tk_status); + +if (nfs4_async_handle_error (task, server,NULL ) == -EAGAIN ) +nfs_restart_rpc (task, server->nfs_client); }static void pnfs_layoutcommit_release (void *lcdata)
Signature · 1 line
1.6.6
PATCH 6/6 SQUASHME pnfs-submit: move layoutcommit to nfs_write_inode
From: Andy Adamson <[email protected]>
The LAYOUTCOMMIT call indicates an update to the file meta data is needed, and should be called when clearing the I_DIRTY_SYNC state.
A call to LAYOUTCOMMIT in nfs_write_inode replaces the calls in nfs_wb_all, nfs_commit_inode, and __nfs4_close.
--- fs/nfs/nfs4state.c | 2 -- fs/nfs/write.c | 22 ++++++++++------------ 2 files changed, 10 insertions(+), 14 deletions(-)
fs/nfs/nfs4state.c +0 -2
@@ -589,8 +589,6 @@ static void __nfs4_close(struct path *path, struct nfs4_state *state, fmode_t fm#ifdef CONFIG_NFS_V4_1 struct nfs_inode *nfsi =NFS_I (state->inode); -if (layoutcommit_needed (nfsi)) -pnfs_layoutcommit_inode (state->inode, wait);if (has_layout (nfsi) && nfsi->layout.roc_iomode) {struct nfs4_pnfs_layout_segment range;
fs/nfs/write.c +10 -12
@@ -1489,13 +1489,6 @@ static int nfs_commit_inode(struct inode *inode, int how)wait_on_bit (&NFS_I (inode)->flags,NFS_INO_COMMIT , nfs_wait_bit_killable,TASK_KILLABLE ); -#ifdef CONFIG_NFS_V4_1 -if (may_wait &&layoutcommit_needed (NFS_I (inode))) { - error =pnfs_layoutcommit_inode (inode,1 ); -if (error <0 ) -return error; - } -#endif /* CONFIG_NFS_V4_1 */ }else nfs_commit_clear_lock (NFS_I (inode)); out:
@@ -1545,7 +1538,16 @@ static int nfs_commit_unstable_pages(struct inode *inode, struct writeback_contrint nfs_write_inode (struct inode *inode,struct writeback_control *wbc) { -return nfs_commit_unstable_pages (inode, wbc); +int ret; + ret =nfs_commit_unstable_pages (inode, wbc); +#ifdef CONFIG_NFS_V4_1 +if (ret >=0 &&layoutcommit_needed (NFS_I (inode))) { +int err =pnfs_layoutcommit_inode (inode,1 ); +if (err <0 ) + ret = err; + } +#endif /* CONFIG_NFS_V4_1 */ +return ret; }/*
@@ -1562,10 +1564,6 @@ int nfs_wb_all(struct inode *inode) }; ret =sync_inode (inode, &wbc); -#ifdef CONFIG_NFS_V4_1 -if (!ret &&layoutcommit_needed (NFS_I (inode))) - ret =pnfs_layoutcommit_inode (inode,1 ); -#endif return ret; }
Signature · 1 line
1.6.6
Re: [PATCH 6/6] SQUASHME pnfs-submit: move layoutcommit to nfs_write_inode
Quoted 49 lines · On 06/02/2010 06:54 PM, [email protected] wrote:
From: Andy Adamson <[email protected]>
The LAYOUTCOMMIT call indicates an update to the file meta data is needed, and should be called when clearing the
I_DIRTY_SYNCstate.A call to LAYOUTCOMMIT in
nfs_write_inodereplaces the calls innfs_wb_all,nfs_commit_inode, and__nfs4_close.Signed-off-by Andy Adamson <[email protected]>--- fs/nfs/nfs4state.c | 2 -- fs/nfs/write.c | 22 ++++++++++------------ 2 files changed, 10 insertions(+), 14 deletions(-)fs/nfs/nfs4state.c +0
-2@@ -589,8 +589,6 @@ static void __nfs4_close(struct path *path, struct nfs4_state *state, fmode_t fm#ifdef CONFIG_NFS_V4_1 struct nfs_inode *nfsi =NFS_I (state->inode); -if (layoutcommit_needed (nfsi)) -pnfs_layoutcommit_inode (state->inode, wait);if (has_layout (nfsi) && nfsi->layout.roc_iomode) {struct nfs4_pnfs_layout_segment range;fs/nfs/write.c +0
-7@@ -1489,13 +1489,6 @@ static int nfs_commit_inode(struct inode *inode, int how)wait_on_bit (&NFS_I (inode)->flags,NFS_INO_COMMIT , nfs_wait_bit_killable,TASK_KILLABLE ); -#ifdef CONFIG_NFS_V4_1 -if (may_wait &&layoutcommit_needed (NFS_I (inode))) { - error =pnfs_layoutcommit_inode (inode,1 ); -if (error <0 ) -return error; - } -#endif /* CONFIG_NFS_V4_1 */ }else nfs_commit_clear_lock (NFS_I (inode)); out:@@ -1545,7 +1538,16 @@ static int nfs_commit_unstable_pages(struct inode *inode, struct writeback_contrint nfs_write_inode (struct inode *inode,struct writeback_control *wbc) {
fs/nfs/write.c static int nfs_commit_unstable_pages(struct inode *inode, struct writeback_contr · line 1549
Quoted 10 lines
fs/nfs/write.c +9
-1-return nfs_commit_unstable_pages (inode, wbc); +int ret; + ret =nfs_commit_unstable_pages (inode, wbc); +#ifdef CONFIG_NFS_V4_1 +if (ret >=0 &&layoutcommit_needed (NFS_I (inode))) { +int err =pnfs_layoutcommit_inode (inode,1 ); +if (err <0 ) + ret = err; + } +#endif /* CONFIG_NFS_V4_1 */
Here you should do better. Now that it's so easy for sure.
define an inline called pnfs_layoutcommit_inode that does the if() and calls a _pnfs_layoutcommit_inode(). The later is today's original function.
In header define that inline for the pnfs case and define an empty one returning 0 for the not CONFIG_NFS_V4_1 case.
I had a patch like that but after your patch it is much easier and makes more sense.
I would fold it into this patch.
Boaz
Quoted 15 lines
fs/nfs/write.c +1
-4+return ret; }/* @@ -1562,10 +1564,6 @@ int nfs_wb_all(struct inode *inode) }; ret =sync_inode (inode, &wbc); -#ifdef CONFIG_NFS_V4_1 -if (!ret &&layoutcommit_needed (NFS_I (inode))) - ret =pnfs_layoutcommit_inode (inode,1 ); -#endif return ret; }
Re: [PATCH 4/6] SQUASHME pnfs-submit: cleanup layoutcommit call
In reply to [email protected]Quoted 18 lines · On 06/02/2010 06:54 PM, [email protected] wrote:
From: Andy Adamson <[email protected]>
pnfs_layoutcommit_inodeprints its status upon exit.Signed-off-by Andy Adamson <[email protected]>--- fs/nfs/pnfs.c | 11 ++--------- 1 files changed, 2 insertions(+), 9 deletions(-)fs/nfs/pnfs.c +0
-1@@ -766,15 +766,8 @@ _pnfs_return_layout(struct inode *ino, struct nfs4_pnfs_layout_segment *range, !pnfs_return_layout_barrier(nfsi, &arg)); } -if (layoutcommit_needed (nfsi)) {
fs/nfs/pnfs.c _pnfs_return_layout(struct inode *ino, struct nfs4_pnfs_layout_segment *range, · line 770
Quoted 10 lines
fs/nfs/pnfs.c +2
-8- status =pnfs_layoutcommit_inode (ino, wait); -if (status) { -dprintk ("%s: layoutcommit failed, status=%d. " -"Returning layout anyway\n" , - __func__, status); - status =0 ; - } - } +if (layoutcommit_needed (nfsi)) +pnfs_layoutcommit_inode (ino, wait);
The print was not for the fail but for the status ignored by layout_return. But ye we can do with out.
But please leave a big comment why we ignore the status from pnfs_layoutcommit_inode.
Boaz
Quoted 3 lines
fs/nfs/pnfs.c +0
-0} send_return: status =return_layout (ino, &arg, stateid, type, lo, wait);
cover pnfs-submit cleanup layoutcommit for file layout
In reply to [email protected]This time sent with the subject....
Quoted 64 lines · On Jun 2, 2010, at 11:54 AM, [email protected] wrote:
This is against the pnfs-submit branch of the 2.6.34 tree. They will need to be applied against the 2.6.35-rc1 tree which I can do after comments.
RFC: I would like comments, especially on 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
Remove unused layoutcommit
layoutdriver_io_operations. Will be restored in post-submit patches 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch 0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patchA cleanup, and call the async error handler. 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
This next patch moves the
pnfs_layoutcommit_inodecall tonfs_write_inode, and it is the only call other than in layoutreturn. (removed calls in__nfs4_close,nfs_commit_inode,nfs_wb_sync).This is fine for the file layout, and I think it's OK for the object and block layouts as well.
I left the LAYOUTCOMMIT call in
nfs_write_inodea synchronous call, becausenfs_commit_unstable_pagessets theFLUSH_SYNCflag. Should this be an asyc LAYOUTCOMMIT call?
pnfs_layoutcommit_inodeis called afternfs_commit_unstable_pages()so that if LAYOUTCOMMIT fails, the unstable pages have been processed..The error handlers (sync and async) call
nfs4_map_errors, so unhandled errors (such asNFS4ERR_BADLAYOUT) get returned tonfs_write_ioodeas-EIO.Examining the
write_inodecall paths, I could not see where the-EIOwould be passed back to the application. Testing with pynfs which I had returnNFS4ERR_BADLAYOUTto the layout commit call, shows the-EIOreturn not stopping the client nor is the error reported back to the application.We will add code to the error handlers for errors such as
NFS4ERR_BADLAYOUTthat require us to stop using and free the layout, and redo the I/O through the MDS.Anyway, review is much appreciated.
0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
Testing:
WithCONFIG_NFS_V4_1set
NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. Note: there were exactly the same number of LAYOUTCOMMITS sent as were sent withpnfs_layoutcommit_inodebeing called from__nfs4_close(never happened),nfs_commit_inodeandnfs_wb_sync.Passed Connectathon general test against pynfs file layout server with the
NFS4ERR_BADLAYOUTbeing returned on every third LAYOUTCOMMIT.-->Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] pnfs-submit cleanup layoutcommit for file layout
In reply to [email protected]Quoted 10 lines · On 06/02/2010 06:54 PM, [email protected] wrote:
This is against the pnfs-submit branch of the 2.6.34 tree. They will need to be applied against the 2.6.35-rc1 tree which I can do after comments.
RFC: I would like comments, especially on 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
Remove unused layoutcommit
layoutdriver_io_operations. Will be restored in post-submit patches 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch
These two should be combined. The cleanup_ is to clean after what's done in setup_.
Quoted 2 lines
0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
For example objects can do with this one only
Quoted 11 lines
A cleanup, and call the async error handler. 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
This next patch moves the
pnfs_layoutcommit_inodecall tonfs_write_inode, and it is the only call other than in layoutreturn. (removed calls in__nfs4_close,nfs_commit_inode,nfs_wb_sync).This is fine for the file layout, and I think it's OK for the object and block layouts as well.
It sounds very nice. It might have problems though. On the NFS_STABLE path again. Because of this stupid thing I found that when returning NFS_STABLE from writes, and no commits are called, then the internal i_size does not get updated until after the layout commit has returned and the client detects a change_attr on server. (Even if it was this client that caused the update)
But this should be fixed regardless. And currently I'm running with commits on in objlayout. (Which reminds me to send the patch to Benny)
So yes I like this change a lot. It makes tons of sense to me as well.
Quoted 4 lines
I left the LAYOUTCOMMIT call in
nfs_write_inodea synchronous call, becausenfs_commit_unstable_pagessets theFLUSH_SYNCflag. Should this be an asyc LAYOUTCOMMIT call?
look at the struct writeback_control *wbc received, it has a flag which states if this is sync or async do according to that flag. (Tell me if you don't find it)
Quoted 26 lines
pnfs_layoutcommit_inodeis called afternfs_commit_unstable_pages()so that if LAYOUTCOMMIT fails, the unstable pages have been processed..The error handlers (sync and async) call
nfs4_map_errors, so unhandled errors (such asNFS4ERR_BADLAYOUT) get returned tonfs_write_ioodeas-EIO.Examining the
write_inodecall paths, I could not see where the-EIOwould be passed back to the application. Testing with pynfs which I had returnNFS4ERR_BADLAYOUTto the layout commit call, shows the-EIOreturn not stopping the client nor is the error reported back to the application.We will add code to the error handlers for errors such as
NFS4ERR_BADLAYOUTthat require us to stop using and free the layout, and redo the I/O through the MDS.Anyway, review is much appreciated.
0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
Testing:
WithCONFIG_NFS_V4_1set
NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. Note: there were exactly the same number of LAYOUTCOMMITS sent as were sent withpnfs_layoutcommit_inodebeing called from__nfs4_close(never happened),nfs_commit_inodeandnfs_wb_sync.
Quoted 3 lines
Passed Connectathon general test against pynfs file layout server with the
NFS4ERR_BADLAYOUTbeing returned on every third LAYOUTCOMMIT.
Andy you got this patchset all backwards. And they are not a set.
4,5,6 are to go in first and are intended for the full tree and the .34 and .33 backport tree's as well. If I want to test with them I'll need them stand alone un-conflicting.
Then 1+2,3 are something else and should be done on top of these above. If they are self sustained and could be re applied on the to of the tree as patch -R, then grate. If not then a "bring them back patch" could be nice. without them we can't test any of this.
Quoted 3 lines
-->Andy
Boaz
Re: [PATCH 0/6] pnfs-submit cleanup layoutcommit for file layout
On Thu, Jun 3, 2010 at 4:10 AM, Boaz Harrosh <[email protected]> wro= te:
Quoted 2 lines · On 06/02/2010 06:54 PM, [email protected] wrote:
Quoted 1 line
This is against the pnfs-submit branch of the 2.6.34 tree. They will=
need to be
Quoted 6 lines
Quoted 6 lines
applied against the 2.6.35-rc1 tree which I can do after comments.
RFC: I would like comments, especially on 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
Remove unused layoutcommit
layoutdriver_io_operations. Will be resto=
red
Quoted 11 lines
Quoted 3 lines
in post-submit patches 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch
These two should be combined. The
cleanup_is to clean after what's done insetup_.Quoted 2 lines
0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
For example objects can do with this one only
OK - makes sense as all three get squashed into the same patch.
Quoted 6 lines
Quoted 6 lines
A cleanup, and call the async error handler. 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
This next =A0patch moves the
pnfs_layoutcommit_inodecall to nfs_wri=
te_inode,
Quoted 1 line
Quoted 1 line
and it is the only call other than in layoutreturn. (removed calls i=
n
Quoted 3 lines
Quoted 3 lines
__nfs4_close,nfs_commit_inode,nfs_wb_sync).This is fine for the file layout, and I think it's OK for the object=
and
Quoted 4 lines
Quoted 2 lines
block layouts as well.
It sounds very nice. It might have problems though. On the NFS_STABLE=
path
Quoted 1 line
again. Because of this stupid thing I found that when returning NFS_S=
TABLE
Quoted 1 line
from writes, and no commits are called, then the internal
i_sizedoes=
not
Quoted 1 line
get updated until after the layout commit has returned and the client=
detects
Quoted 1 line
a
change_attron server. (Even if it was this client that caused the =
update)
Quoted 3 lines
But this should be fixed regardless. And currently I'm running with commits on in objlayout. (Which reminds me to send the patch to Benny=
)
Quoted 2 lines
So yes I like this change a lot. It makes tons of sense to me as well=
=2E
Good.
Quoted 2 lines
Quoted 2 lines
I left the LAYOUTCOMMIT call in
nfs_write_inodea synchronous call, =
because
Quoted 5 lines
Quoted 3 lines
nfs_commit_unstable_pagessets theFLUSH_SYNCflag. Should this be an asyc LAYOUTCOMMIT call?look at the struct
writeback_control*wbc received, it has a flag whi=
ch states
Quoted 1 line
if this is sync or async do according to that flag. (Tell me if you d=
on't find it)
OK, thanks.
Quoted 2 lines
Quoted 2 lines
pnfs_layoutcommit_inodeis called afternfs_commit_unstable_pages()=
so that
Quoted 3 lines
Quoted 3 lines
if LAYOUTCOMMIT fails, the unstable pages have been processed..
The error handlers (sync and async) call
nfs4_map_errors, so unhandl=
ed
Quoted 1 line
Quoted 1 line
errors (such as
NFS4ERR_BADLAYOUT) get returned tonfs_write_ioodea=
s -EIO.
Quoted 2 lines
Quoted 2 lines
Examining the
write_inodecall paths, I could not see where the -EIO=
would
Quoted 2 lines
Quoted 2 lines
be passed back to the application. =A0Testing with pynfs which I had return
NFS4ERR_BADLAYOUTto the layout commit call, shows the -E=
IO return
Quoted 1 line
Quoted 1 line
not stopping the client nor is the error reported back to the applic=
ation.
Quoted 2 lines
Quoted 2 lines
We will add code to the error handlers for errors such as NFS4ERR_BA=
DLAYOUT
Quoted 1 line
Quoted 1 line
that require us to stop using and free the layout, and redo the I/O =
through
Quoted 9 lines
Quoted 9 lines
the MDS.
Anyway, review is much appreciated.
0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
Testing:
WithCONFIG_NFS_V4_1set
NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. No=
te: there
Quoted 2 lines
Quoted 2 lines
were exactly the same number of LAYOUTCOMMITS sent as were sent with
pnfs_layoutcommit_inodebeing called from__nfs4_close(never happen=
ed),
Quoted 3 lines
Quoted 3 lines
nfs_commit_inodeandnfs_wb_sync.Passed Connectathon general test against pynfs file layout server wi=
th
Quoted 8 lines
Quoted 2 lines
the
NFS4ERR_BADLAYOUTbeing returned on every third LAYOUTCOMMIT.Andy you got this patchset all backwards. And they are not a set.
4,5,6 are to go in first and are intended for the full tree and the .34 and .33 backport tree's as well. If I want to test with them I'll need them stand alone un-conflicting.
Sure.
Quoted 2 lines
Then 1+2,3 are something else and should be done on top of these abov=
e.
Quoted 1 line
If they are self sustained and could be re applied on the to of the t=
ree
Quoted 1 line
as patch -R, then grate. If not then a "bring them back patch" could =
be
Quoted 1 line
nice. without them we can't test any of this
Thanks for the review. I'll resend as requested so that you can test..
-->Andy
Quoted 8 lines
Quoted 4 lines
-->Andy
Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" =
in
Quoted 3 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html