Skip to contentSkip to conversation

pnfs-submit add layoutget,layoutreturn error handling version 2

PATCH In review [email protected] 4 participants 20 messages Jun 23, 2010 – Jul 1, 2010 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover pnfs-submit add layoutget,layoutreturn error handling version 2

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the pyNFS server and the testclient framework.

Still todo:

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and layoutcommit do not pass nfs_stste to the error handlers.

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

#1 lore raw

PATCH 1/10 SQUASHME: pnfs-submit remove unused filelayout_mount_type

From: Andy Adamson <[email protected]>

---
 fs/nfs/nfs4filelayout.h |    6 ------
 1 files changed, 0 insertions(+), 6 deletions(-)
fs/nfs/nfs4filelayout.h +0 -6
index 29e481d..de8391f 100644
@@ -24,8 +24,6 @@
 #define NFS4_PNFS_MAX_MULTI_CNT  64 /* 256 fit into a u8 stripe_index */
 #define NFS4_PNFS_MAX_MULTI_DS   2
 
-#define FILE_MT(inode) ((struct filelayout_mount_type *) \
-			(NFS_SERVER(inode)->pnfs_mountid->mountid))
 #define FILE_DSADDR(lseg) (container_of(lseg->deviceid, \
 					struct nfs4_file_layout_dsaddr, \
 					deviceid))
@@ -76,10 +74,6 @@ struct nfs4_filelayout {
 	u32 stripe_unit;
 };
 
-struct filelayout_mount_type {
-	struct super_block *fl_sb;
-};
-
 extern struct nfs_fh *
 nfs4_fl_select_ds_fh(struct pnfs_layout_segment *lseg, loff_t offset);
 
Signature · 1 line
1.6.6
#2 lore raw

PATCH 2/10 SQUASHME pnfs-submit: pnfs_try_to_read, write, commit using freed memory

From: Andy Adamson <[email protected]>

The nfs_read_data or nfs_write_data is freed during rpc call.

---
 fs/nfs/pnfs.h |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)
fs/nfs/pnfs.h +6 -3
index 78776f3..c60eff6 100644
@@ -105,13 +105,14 @@ static inline enum pnfs_try_status
 pnfs_try_to_read_data(struct nfs_read_data *data,
 		      const struct rpc_call_ops *call_ops)
 {
+	struct inode *inode = data->inode;
 	enum pnfs_try_status ret;
 
 	if (!data->req->wb_lseg)
 		return PNFS_NOT_ATTEMPTED;
 	ret = _pnfs_try_to_read_data(data, call_ops);
 	if (ret == PNFS_ATTEMPTED)
-		nfs_inc_stats(data->inode, NFSIOS_PNFS_READ);
+		nfs_inc_stats(inode, NFSIOS_PNFS_READ);
 	else
 		_pnfs_clear_lseg_from_pages(&data->pages);
 	return ret;
@@ -122,13 +123,14 @@ pnfs_try_to_write_data(struct nfs_write_data *data,
 		       const struct rpc_call_ops *call_ops,
 		       int how)
 {
+	struct inode *inode = data->inode;
 	enum pnfs_try_status ret;
 
 	if (!data->req->wb_lseg)
 		return PNFS_NOT_ATTEMPTED;
 	ret = _pnfs_try_to_write_data(data, call_ops, how);
 	if (ret == PNFS_ATTEMPTED)
-		nfs_inc_stats(data->inode, NFSIOS_PNFS_WRITE);
+		nfs_inc_stats(inode, NFSIOS_PNFS_WRITE);
 	else
 		_pnfs_clear_lseg_from_pages(&data->pages);
 	return ret;
@@ -139,6 +141,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
 		   const struct rpc_call_ops *call_ops,
 		   int how)
 {
+	struct inode *inode = data->inode;
 	enum pnfs_try_status ret;
 
 	/* Unlike in pnfs_try_to_write_data and pnfs_try_to_read_data,
@@ -148,7 +151,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
 	 */
 	ret = _pnfs_try_to_commit(data, call_ops, how);
 	if (ret == PNFS_ATTEMPTED)
-		nfs_inc_stats(data->inode, NFSIOS_PNFS_COMMIT);
+		nfs_inc_stats(inode, NFSIOS_PNFS_COMMIT);
 	else
 		_pnfs_clear_lseg_from_pages(&data->pages);
 	return ret;
Signature · 1 line
1.6.6
#3 lore raw

PATCH 3/10 SQUASHME pnfs-submit: clear page lseg on partial i/o

From: Andy Adamson <[email protected]>

partial reads or writes use a single page not the page list.

---
 fs/nfs/pnfs.h |   16 ++++++++++++----
 1 files changed, 12 insertions(+), 4 deletions(-)
fs/nfs/pnfs.h +12 -4
index c60eff6..70f1d97 100644
@@ -91,10 +91,18 @@ static inline int pnfs_enabled_sb(struct nfs_server *nfss)
 	return nfss->pnfs_curr_ld != NULL;
 }
 
-static inline void _pnfs_clear_lseg_from_pages(struct list_head *head)
+static inline void _pnfs_clear_lseg_from_pages(struct nfs_page *partial,
+					       struct list_head *head)
 {
 	struct nfs_page *req;
 
+	/* partial_read or partial_write */
+	if (list_empty(head)) {
+		put_lseg(partial->wb_lseg);
+		partial->wb_lseg = NULL;
+		return;
+	}
+	/* full_read or full_write */
 	list_for_each_entry(req, head, wb_list) {
 		put_lseg(req->wb_lseg);
 		req->wb_lseg = NULL;
@@ -114,7 +122,7 @@ pnfs_try_to_read_data(struct nfs_read_data *data,
 	if (ret == PNFS_ATTEMPTED)
 		nfs_inc_stats(inode, NFSIOS_PNFS_READ);
 	else
-		_pnfs_clear_lseg_from_pages(&data->pages);
+		_pnfs_clear_lseg_from_pages(data->req, &data->pages);
 	return ret;
 }
 
@@ -132,7 +140,7 @@ pnfs_try_to_write_data(struct nfs_write_data *data,
 	if (ret == PNFS_ATTEMPTED)
 		nfs_inc_stats(inode, NFSIOS_PNFS_WRITE);
 	else
-		_pnfs_clear_lseg_from_pages(&data->pages);
+		_pnfs_clear_lseg_from_pages(data->req, &data->pages);
 	return ret;
 }
 
@@ -153,7 +161,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
 	if (ret == PNFS_ATTEMPTED)
 		nfs_inc_stats(inode, NFSIOS_PNFS_COMMIT);
 	else
-		_pnfs_clear_lseg_from_pages(&data->pages);
+		_pnfs_clear_lseg_from_pages(data->req, &data->pages);
 	return ret;
 }
 
Signature · 1 line
1.6.6
#4 lore raw

PATCH 4/10 SQUASHME pnfs-submit: remove encode_layoutreturn

From: Andy Adamson <[email protected]>

Not used by the file layout

---
 fs/nfs/nfs4xdr.c          |   15 ++-------------
 include/linux/nfs4_pnfs.h |    4 ----
 2 files changed, 2 insertions(+), 17 deletions(-)
fs/nfs/nfs4xdr.c +2 -13
index a509bc1..fb50361 100644
@@ -1874,24 +1874,13 @@ encode_layoutreturn(struct xdr_stream *xdr,
 	*p++ = cpu_to_be32(args->lseg.iomode);
 	*p = cpu_to_be32(args->return_type);
 	if (args->return_type == RETURN_FILE) {
-		struct layoutdriver_io_operations *ld_io_ops =
-			NFS_SERVER(args->inode)->pnfs_curr_ld->ld_io_ops;
-
 		p = reserve_space(xdr, 16 + NFS4_STATEID_SIZE);
 		p = xdr_encode_hyper(p, args->lseg.offset);
 		p = xdr_encode_hyper(p, args->lseg.length);
 		p = xdr_encode_opaque_fixed(p, &args->stateid.u.data,
 					    NFS4_STATEID_SIZE);
-
-		dprintk("%s: call %pF\n", __func__,
-			ld_io_ops->encode_layoutreturn);
-		if (ld_io_ops->encode_layoutreturn) {
-			ld_io_ops->encode_layoutreturn(
-				&NFS_I(args->inode)->layout, xdr, args);
-		} else {
-			p = reserve_space(xdr, 4);
-			*p = cpu_to_be32(0);
-		}
+		p = reserve_space(xdr, 4);
+		*p = cpu_to_be32(0);
 	}
 	hdr->nops++;
 	hdr->replen += decode_layoutreturn_maxsz;
include/linux/nfs4_pnfs.h +0 -4
index d20b5de..782fdd9 100644
@@ -154,10 +154,6 @@ struct layoutdriver_io_operations {
 	struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
 	void (*free_lseg) (struct pnfs_layout_segment *lseg);
 
-	void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
-				struct xdr_stream *xdr,
-				const struct nfs4_pnfs_layoutreturn_arg *args);
-
 	/* Registration information for a new mounted file system
 	 */
 	int (*initialize_mountpoint) (struct nfs_client *);
Signature · 1 line
1.6.6
#5 lore raw

PATCH 5/10 SQUASHME pnfs-submit: add error handling to layout return

From: Andy Adamson <[email protected]>

---
 fs/nfs/nfs4proc.c |   37 +++++++++++++++++++++++++++----------
 1 files changed, 27 insertions(+), 10 deletions(-)
fs/nfs/nfs4proc.c +27 -10
index 5ffdfe1..0ab875f 100644
@@ -5641,6 +5641,11 @@ static void nfs4_pnfs_layoutreturn_done(struct rpc_task *task, void *calldata)
 	dprintk("--> %s\n", __func__);
 
 	nfs4_sequence_done(server, &lrp->res.seq_res, task->tk_status);
+	if (RPC_ASSASSINATED(task))
+		return;
+
+	if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
+		nfs_restart_rpc(task, server->nfs_client);
 
 	dprintk("<-- %s\n", __func__);
 }
@@ -5667,7 +5672,7 @@ static const struct rpc_call_ops nfs4_pnfs_layoutreturn_call_ops = {
 	.rpc_release = nfs4_pnfs_layoutreturn_release,
 };
 
-int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool wait)
+int _pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool issync)
 {
 	struct inode *ino = lrp->args.inode;
 	struct nfs_server *server = NFS_SERVER(ino);
@@ -5689,21 +5694,33 @@ int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool wait)
 	dprintk("--> %s\n", __func__);
 	lrp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
 	task = rpc_run_task(&task_setup_data);
-	if (IS_ERR(task)) {
-		status = PTR_ERR(task);
+	if (IS_ERR(task))
+		return PTR_ERR(task);
+	if (!issync)
 		goto out;
-	}
-	if (wait) {
-		status = nfs4_wait_for_completion_rpc_task(task);
-		if (status == 0)
-			status = task->tk_status;
-	}
-	rpc_put_task(task);
+	status = nfs4_wait_for_completion_rpc_task(task);
+	if (status != 0)
+		goto out;
+	status = task->tk_status;
 out:
 	dprintk("<-- %s\n", __func__);
+	rpc_put_task(task);
 	return status;
 }
 
+int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool issync)
+{
+	struct nfs_server *server = NFS_SERVER(lrp->args.inode);
+	struct nfs4_exception exception = { };
+	int err;
+	do {
+		err = nfs4_handle_exception(server,
+				_pnfs4_proc_layoutreturn(lrp, issync),
+				&exception);
+	} while (exception.retry);
+	return err;
+}
+
 int nfs4_pnfs_getdeviceinfo(struct nfs_server *server, struct pnfs_device *pdev)
 {
 	struct nfs4_pnfs_getdeviceinfo_arg args = {
Signature · 1 line
1.6.6
#6 lore raw

PATCH 6/10 SQUASHME pnfs-submit: handle assassinated layoutcommit

From: Andy Adamson <[email protected]>

---
 fs/nfs/nfs4proc.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
fs/nfs/nfs4proc.c +2 -0
index 0ab875f..35e8a56 100644
@@ -5539,6 +5539,8 @@ pnfs_layoutcommit_done(struct rpc_task *task, void *calldata)
 	data->status = task->tk_status;
 
 	nfs4_sequence_done(server, &data->res.seq_res, task->tk_status);
+	if (RPC_ASSASSINATED(task))
+		return;
 
 	if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
 		nfs_restart_rpc(task, server->nfs_client);
Signature · 1 line
1.6.6
#7 lore raw

PATCH 7/10 SQUASHME pnfs-submit: add error handlers to layout get

From: Andy Adamson <[email protected]>

Redo pnfs4_proc_layoutget() logic to match the other async operations that call nfs4_wait_for_completion_rpc_task.

pnfs_get_layout_done passes session and delay errors. Handle them with the generic error handlers.

---
 fs/nfs/nfs4proc.c |   45 +++++++++++++++++++++++++++++----------------
 1 files changed, 29 insertions(+), 16 deletions(-)
fs/nfs/nfs4proc.c +29 -16
index 35e8a56..6283996 100644
@@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct rpc_task *task, void *calldata)
 		return;
 
 	pnfs_get_layout_done(lgp, task->tk_status);
+
+	if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
+		nfs_restart_rpc(task, server->nfs_client);
+
 	dprintk("<-- %s\n", __func__);
 }
 
@@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_layoutget_call_ops = {
  * and deal with retries.
  * Currently we can't since we release lgp and its contents.
  */
-int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
 {
-	struct inode *ino = lgp->args.inode;
-	struct nfs_server *server = NFS_SERVER(ino);
+	struct nfs_server *server = NFS_SERVER(lgp->args.inode);
 	struct rpc_task *task;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_PNFS_LAYOUTGET],
@@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
 		.callback_data = lgp,
 		.flags = RPC_TASK_ASYNC,
 	};
-	int status;
+	int status = 0;
 
 	dprintk("--> %s\n", __func__);
 
 	lgp->res.layout.buf = (void *)__get_free_page(GFP_NOFS);
 	if (lgp->res.layout.buf == NULL) {
 		nfs4_pnfs_layoutget_release(lgp);
-		status = -ENOMEM;
-		goto out;
+		return -ENOMEM;
 	}
 
 	lgp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
 	task = rpc_run_task(&task_setup_data);
-	if (IS_ERR(task)) {
-		status = PTR_ERR(task);
-		goto out;
-	}
+	if (IS_ERR(task))
+		return PTR_ERR(task);
 	status = nfs4_wait_for_completion_rpc_task(task);
-	if (status == 0) {
-		status = lgp->status;
-		if (status == 0)
-			status = pnfs_layout_process(lgp);
-	}
-	rpc_put_task(task);
+	if (status != 0)
+		goto out;
+	status = lgp->status;
+	if (status != 0)
+		goto out;
+	status = pnfs_layout_process(lgp);
 out:
+	rpc_put_task(task);
 	dprintk("<-- %s status=%d\n", __func__, status);
 	return status;
 }
 
+int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+{
+	struct nfs_server *server = NFS_SERVER(lgp->args.inode);
+	struct nfs4_exception exception = { };
+	int err;
+	do {
+		err = nfs4_handle_exception(server, _pnfs4_proc_layoutget(lgp),
+					    &exception);
+	} while (exception.retry);
+	return err;
+}
+
 static void pnfs_layoutcommit_prepare(struct rpc_task *task, void *data)
 {
 	struct pnfs_layoutcommit_data *ldata =
Signature · 1 line
1.6.6
#8 lore raw

PATCH 8/10 pnfs-post-submit: restore encode_layoutreturn

From: Andy Adamson <[email protected]>

---
 fs/nfs/nfs4xdr.c          |   14 ++++++++++++--
 include/linux/nfs4_pnfs.h |    4 ++++
 2 files changed, 16 insertions(+), 2 deletions(-)
fs/nfs/nfs4xdr.c +12 -2
index fb50361..eeee855 100644
@@ -1874,13 +1874,23 @@ encode_layoutreturn(struct xdr_stream *xdr,
 	*p++ = cpu_to_be32(args->lseg.iomode);
 	*p = cpu_to_be32(args->return_type);
 	if (args->return_type == RETURN_FILE) {
+		struct layoutdriver_io_operations *ld_io_ops =
+			NFS_SERVER(args->inode)->pnfs_curr_ld->ld_io_ops;
+
 		p = reserve_space(xdr, 16 + NFS4_STATEID_SIZE);
 		p = xdr_encode_hyper(p, args->lseg.offset);
 		p = xdr_encode_hyper(p, args->lseg.length);
 		p = xdr_encode_opaque_fixed(p, &args->stateid.u.data,
 					    NFS4_STATEID_SIZE);
-		p = reserve_space(xdr, 4);
-		*p = cpu_to_be32(0);
+		dprintk("%s: call %pF\n", __func__,
+		ld_io_ops->encode_layoutreturn);
+		if (ld_io_ops->encode_layoutreturn) {
+			ld_io_ops->encode_layoutreturn(
+				&NFS_I(args->inode)->layout, xdr, args);
+		} else {
+			p = reserve_space(xdr, 4);
+			*p = cpu_to_be32(0);
+		}
 	}
 	hdr->nops++;
 	hdr->replen += decode_layoutreturn_maxsz;
include/linux/nfs4_pnfs.h +4 -0
index 782fdd9..d20b5de 100644
@@ -154,6 +154,10 @@ struct layoutdriver_io_operations {
 	struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
 	void (*free_lseg) (struct pnfs_layout_segment *lseg);
 
+	void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
+				struct xdr_stream *xdr,
+				const struct nfs4_pnfs_layoutreturn_arg *args);
+
 	/* Registration information for a new mounted file system
 	 */
 	int (*initialize_mountpoint) (struct nfs_client *);
Signature · 1 line
1.6.6
#9 lore raw

PATCH 9/10 SQUASHME: pnfs-submit: don't re-initialize i_lock

From: Andy Adamson <[email protected]>

Initialized in the VFS (inode_init_always)

---
 fs/nfs/inode.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
fs/nfs/inode.c +0 -1
index d6dd6e5..7989dea 100644
@@ -1420,7 +1420,6 @@ static void pnfs_init_once(struct nfs_inode *nfsi)
 {
 #ifdef CONFIG_NFS_V4_1
 	init_waitqueue_head(&nfsi->lo_waitq);
-	spin_lock_init(&nfsi->vfs_inode.i_lock);
 	seqlock_init(&nfsi->layout.seqlock);
 	INIT_LIST_HEAD(&nfsi->layout.lo_layouts);
 	INIT_LIST_HEAD(&nfsi->layout.segs);
Signature · 1 line
1.6.6
#10 lore raw

PATCH 10/10 SQUASHME pnfs-submit: remove struct nfs_server from stack

From: Andy Adamson <[email protected]>

---
 fs/nfs/nfs4filelayoutdev.c |   18 ++++++++++++------
 1 files changed, 12 insertions(+), 6 deletions(-)
fs/nfs/nfs4filelayoutdev.c +12 -6
index 5842c09..8e9d04a 100644
@@ -121,9 +121,7 @@ _data_server_lookup(u32 ip_addr, u32 port)
 static int
 nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
 {
-	struct nfs_server	tmp = {
-		.nfs_client = NULL,
-	};
+	struct nfs_server	*tmp;
 	struct sockaddr_in	sin;
 	struct rpc_clnt 	*mds_clnt = mds_srv->client;
 	struct nfs_client	*clp = mds_srv->nfs_client;
@@ -154,12 +152,18 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
 		}
 		goto out;
 	}
+
+	/* Temporay server for nfs4_set_client */
+	tmp = kzalloc(sizeof(struct nfs_server), GFP_KERNEL);
+	if (!tmp)
+		goto out;
+
 	/*
 	 * Set a retrans, timeout interval, and authflavor equual to the MDS
 	 * values. Use the MDS nfs_client cl_ipaddr field so as to use the
 	 * same co_ownerid as the MDS.
 	 */
-	err = nfs4_set_client(&tmp,
+	err = nfs4_set_client(tmp,
 			      mds_srv->nfs_client->cl_hostname,
 			      (struct sockaddr *)&sin,
 			      sizeof(struct sockaddr),
@@ -169,9 +173,9 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
 			      mds_clnt->cl_xprt->timeout,
 			      1 /* minorversion */);
 	if (err < 0)
-		goto out;
+		goto out_free;
 
-	clp = tmp.nfs_client;
+	clp = tmp->nfs_client;
 
 	/* Ask for only the EXCHGID4_FLAG_USE_PNFS_DS pNFS role */
 	dprintk("%s EXCHANGE_ID for clp %p\n", __func__, clp);
@@ -211,6 +215,8 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
 	dprintk("%s: ip=%x, port=%hu, rpcclient %p\n", __func__,
 				ntohl(ds->ds_ip_addr), ntohs(ds->ds_port),
 				clp->cl_rpcclient);
+out_free:
+	kfree(tmp);
 out:
 	dprintk("%s Returns %d\n", __func__, err);
 	return err;
Signature · 1 line
1.6.6
#11 lore raw

Re: [PATCH 07/10] SQUASHME pnfs-submit: add error handlers to layout get

In reply to [email protected]
Quoted 18 lines · On 06/23/2010 10:21 PM, [email protected] wrote:

From: Andy Adamson <[email protected]>

Redo pnfs4_proc_layoutget() logic to match the other async operations that call nfs4_wait_for_completion_rpc_task.

pnfs_get_layout_done passes session and delay errors. Handle them with the generic error handlers.

---
 fs/nfs/nfs4proc.c |   45 +++++++++++++++++++++++++++++----------------
 1 files changed, 29 insertions(+), 16 deletions(-)
fs/nfs/nfs4proc.c +0 -0
index 35e8a56..6283996 100644

fs/nfs/nfs4proc.c static void nfs4_pnfs_layoutget_done(struct rpc_task *task, void *calldata) · line 5452

Quoted 7 lines
fs/nfs/nfs4proc.c +4 -0
@@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct rpc_task *task, void *calldata)
 		return;
 
 	pnfs_get_layout_done(lgp, task->tk_status);
+
+	if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
+		nfs_restart_rpc(task, server->nfs_client);
+

Andy high.

In pnfs_get_layout_done() it is working hard to do "lgp->status = XXX"

I never understood, what gets to look at that later on?

Thanks
Boaz

Quoted 75 lines
fs/nfs/nfs4proc.c +25 -16
 	dprintk("<-- %s\n", __func__);
 }
 
@@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_layoutget_call_ops = {
  * and deal with retries.
  * Currently we can't since we release lgp and its contents.
  */
-int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
 {
-	struct inode *ino = lgp->args.inode;
-	struct nfs_server *server = NFS_SERVER(ino);
+	struct nfs_server *server = NFS_SERVER(lgp->args.inode);
 	struct rpc_task *task;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_PNFS_LAYOUTGET],
@@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
 		.callback_data = lgp,
 		.flags = RPC_TASK_ASYNC,
 	};
-	int status;
+	int status = 0;
 
 	dprintk("--> %s\n", __func__);
 
 	lgp->res.layout.buf = (void *)__get_free_page(GFP_NOFS);
 	if (lgp->res.layout.buf == NULL) {
 		nfs4_pnfs_layoutget_release(lgp);
-		status = -ENOMEM;
-		goto out;
+		return -ENOMEM;
 	}
 
 	lgp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
 	task = rpc_run_task(&task_setup_data);
-	if (IS_ERR(task)) {
-		status = PTR_ERR(task);
-		goto out;
-	}
+	if (IS_ERR(task))
+		return PTR_ERR(task);
 	status = nfs4_wait_for_completion_rpc_task(task);
-	if (status == 0) {
-		status = lgp->status;
-		if (status == 0)
-			status = pnfs_layout_process(lgp);
-	}
-	rpc_put_task(task);
+	if (status != 0)
+		goto out;
+	status = lgp->status;
+	if (status != 0)
+		goto out;
+	status = pnfs_layout_process(lgp);
 out:
+	rpc_put_task(task);
 	dprintk("<-- %s status=%d\n", __func__, status);
 	return status;
 }
 
+int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+{
+	struct nfs_server *server = NFS_SERVER(lgp->args.inode);
+	struct nfs4_exception exception = { };
+	int err;
+	do {
+		err = nfs4_handle_exception(server, _pnfs4_proc_layoutget(lgp),
+					    &exception);
+	} while (exception.retry);
+	return err;
+}
+
 static void pnfs_layoutcommit_prepare(struct rpc_task *task, void *data)
 {
 	struct pnfs_layoutcommit_data *ldata =
#12 lore raw

Re: [PATCH 07/10] SQUASHME pnfs-submit: add error handlers to layout get

On Wed, Jun 30, 2010 at 11:19 AM, Boaz Harrosh <[email protected]> w= rote:

Quoted 4 lines · On 06/23/2010 10:21 PM, [email protected] wrote:
Quoted 3 lines

From: Andy Adamson <[email protected]>

Redo pnfs4_proc_layoutget() logic to match the other async operation=

s that call

Quoted 3 lines
Quoted 3 lines

nfs4_wait_for_completion_rpc_task.

pnfs_get_layout_done passes session and delay errors. Handle them wi=

th the

Quoted 5 lines
Quoted 5 lines

generic error handlers.

---
=A0fs/nfs/nfs4proc.c | =A0 45 +++++++++++++++++++++++++++++---------=

-------

Quoted 7 lines

fs/nfs/nfs4proc.c static void nfs4_pnfs_layoutget_done(struct r=

Quoted 7 lines

=A01 files changed, 29 insertions(+), 16 deletions(-)

fs/nfs/nfs4proc.c +0 -0
index 35e8a56..6283996 100644
@@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct r=

pc_task *task, void *calldata)

Quoted 5 lines

fs/nfs/nfs4proc.c static void nfs4_pnfs_layoutget_done(struct r= · line 5450

Quoted 5 lines
fs/nfs/nfs4proc.c +2 -0
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return;
 
 =A0 =A0 =A0 pnfs_get_layout_done(lgp, task->tk_status);
+
+ =A0 =A0 if (nfs4_async_handle_error(task, server, NULL, NULL) =3D=3D=

-EAGAIN)

Quoted 6 lines

fs/nfs/nfs4proc.c static void nfs4_pnfs_layoutget_done(struct r= · line 5452

Quoted 2 lines
fs/nfs/nfs4proc.c +2 -0
+ =A0 =A0 =A0 =A0 =A0 =A0 nfs_restart_rpc(task, server->nfs_client);
+

Andy high.

In pnfs_get_layout_done() it is working hard to do "lgp->status =3D X=

XX"

Quoted 2 lines

I never understood, what gets to look at that later on?

HI Boaz

_pnfs4_proc_layoutget will return it if it is set, then send_layoutget returns it to pnfs_update_layout.

-->Andy

Quoted 8 lines

Thanks
Boaz

fs/nfs/nfs4proc.c static const struct rpc_call_ops nfs4_pnfs_la=

Quoted 4 lines
fs/nfs/nfs4proc.c +0 -0
 =A0 =A0 =A0 dprintk("<-- %s\n", __func__);
 =A0}
 
@@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_la=

youtget_call_ops =3D {

Quoted 4 lines
Quoted 4 lines

=A0 * and deal with retries.
=A0 * Currently we can't since we release lgp and its contents. =A0 */
-int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)

Quoted 8 lines
Quoted 8 lines

+static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp) =A0{
- =A0 =A0 struct inode *ino =3D lgp->args.inode;
- =A0 =A0 struct nfs_server *server =3D NFS_SERVER(ino);
+ =A0 =A0 struct nfs_server *server =3D NFS_SERVER(lgp->args.inode); =A0 =A0 =A0 struct rpc_task *task;
=A0 =A0 =A0 struct rpc_message msg =3D {
=A0 =A0 =A0 =A0 =A0 =A0 =A0 .rpc_proc =3D &nfs4_procedures[NFSPROC4_=

CLNT_PNFS_LAYOUTGET],

Quoted 1 line

fs/nfs/nfs4proc.c int pnfs4_proc_layoutget(struct nfs4_pnfs_la=

Quoted 1 line
fs/nfs/nfs4proc.c +0 -0
@@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_la=

youtget *lgp)

Quoted 9 lines
Quoted 9 lines
fs/nfs/nfs4proc.c +1 -1
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .callback_data =3D lgp,
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .flags =3D RPC_TASK_ASYNC,
 =A0 =A0 =A0 };
- =A0 =A0 int status;
+ =A0 =A0 int status =3D 0;
 
 =A0 =A0 =A0 dprintk("--> %s\n", __func__);
 

=A0 =A0 =A0 lgp->res.layout.buf =3D (void *)__get_free_page(GFP_NOFS=

);

Quoted 7 lines
Quoted 7 lines

=A0 =A0 =A0 if (lgp->res.layout.buf =3D=3D NULL) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfs4_pnfs_layoutget_release(lgp);
- =A0 =A0 =A0 =A0 =A0 =A0 status =3D -ENOMEM;
- =A0 =A0 =A0 =A0 =A0 =A0 goto out;
+ =A0 =A0 =A0 =A0 =A0 =A0 return -ENOMEM;
=A0 =A0 =A0 }

Quoted 13 lines
Quoted 5 lines

=A0 =A0 =A0 lgp->res.seq_res.sr_slotid =3D NFS4_MAX_SLOT_TABLE; =A0 =A0 =A0 task =3D rpc_run_task(&task_setup_data);
- =A0 =A0 if (IS_ERR(task)) {
- =A0 =A0 =A0 =A0 =A0 =A0 status =3D PTR_ERR(task);
- =A0 =A0 =A0 =A0 =A0 =A0 goto out;

Quoted 8 lines

- =A0 =A0 }
+ =A0 =A0 if (IS_ERR(task))
+ =A0 =A0 =A0 =A0 =A0 =A0 return PTR_ERR(task);
=A0 =A0 =A0 status =3D nfs4_wait_for_completion_rpc_task(task);
- =A0 =A0 if (status =3D=3D 0) {
- =A0 =A0 =A0 =A0 =A0 =A0 status =3D lgp->status;
- =A0 =A0 =A0 =A0 =A0 =A0 if (status =3D=3D 0)
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 status =3D pnfs_layout_pro=

cess(lgp);

Quoted 14 lines
Quoted 14 lines

- =A0 =A0 }
- =A0 =A0 rpc_put_task(task);
+ =A0 =A0 if (status !=3D 0)
+ =A0 =A0 =A0 =A0 =A0 =A0 goto out;
+ =A0 =A0 status =3D lgp->status;
+ =A0 =A0 if (status !=3D 0)
+ =A0 =A0 =A0 =A0 =A0 =A0 goto out;
+ =A0 =A0 status =3D pnfs_layout_process(lgp); =A0out:
+ =A0 =A0 rpc_put_task(task);
=A0 =A0 =A0 dprintk("<-- %s status=3D%d\n", __func__, status); =A0 =A0 =A0 return status;
=A0}

Quoted 7 lines

fs/nfs/nfs4proc.c

Quoted 7 lines
fs/nfs/nfs4proc.c +7 -0
+int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+{
+ =A0 =A0 struct nfs_server *server =3D NFS_SERVER(lgp->args.inode);
+ =A0 =A0 struct nfs4_exception exception =3D { };
+ =A0 =A0 int err;
+ =A0 =A0 do {
+ =A0 =A0 =A0 =A0 =A0 =A0 err =3D nfs4_handle_exception(server, _pnf=

s4_proc_layoutget(lgp),

Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0 =A0 =A0 &exception);

Quoted 5 lines
Quoted 5 lines

+ =A0 =A0 } while (exception.retry);
+ =A0 =A0 return err;

fs/nfs/nfs4proc.c +2 -0
+}
+

=A0static void pnfs_layoutcommit_prepare(struct rpc_task *task, void=

*data)

Quoted 5 lines
Quoted 2 lines

=A0{
=A0 =A0 =A0 struct pnfs_layoutcommit_data *ldata =3D

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" =

in

Quoted 3 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html

#13 lore raw
In reply to [email protected]
Quoted 44 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the pyNFS server and the testclient framework.

Still todo:

Quoted 8 lines

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and layoutcommit do not pass nfs_stste to the error handlers.

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit

	BUG_ON(lo->refcount <= 0);

in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING: layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/initstate
Jun 24 12:07:26 tl2 kernel: CPU 1 
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]  [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20  EFLAGS: 00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX: ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI: ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08: 0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11: 0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14: ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS:  0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3: 0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo ffff88007525c000, task ffff88007d988000)
Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8 ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18 ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release+0x43/0x68 [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>] nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>] rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/0x66 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule+0x15/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ? rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ? autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ? spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67 48 8b 82 48 03 00 00 f6 05 
Jun 24 12:07:26 tl2 kernel: RIP  [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
#14 lore raw

OK - I'll look into it.

Sorry I missed today's pNFS call.

-->Andy

Quoted 111 lines · Benny Halevy
Quoted 44 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the pyNFS server and the testclient framework.

Still todo:

Quoted 8 lines

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and layoutcommit do not pass nfs_stste to the error handlers.

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit

       BUG_ON(lo->refcount <= 0);

in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING: layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]  [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20  EFLAGS: 00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX: ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI: ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08: 0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11: 0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14: ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS:  0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3: 0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo ffff88007525c000, task ffff88007d988000)
Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8 ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18 ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release+0x43/0x68 [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>] nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>] rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/0x66 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule+0x15/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ? rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ? autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ? spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67 48 8b 82 48 03 00 00 f6 05
Jun 24 12:07:26 tl2 kernel: RIP  [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): AANLkTikJWftkWhU8TIOGxvGxo8s2_sXyMn8VIsk9caTv-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#16 lore raw

Hi Benny

I have not been able to reproduce this BUG. I've tried against the files pyNFS server with return_on_close False as well a True, and against a GFS2/pNFS cluster with write layouts turned on.

Patch 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch calls put_lseg when I/O to a DS fails. I tested this using the pyNFS files layout server and blocking the DS with iptables. I think this is the only change in this patch set that would affect the refcounting.

Are you able to reproduce the BUG?

-->Andy

Quoted 129 lines · William A. (Andy) Adamson

OK - I'll look into it.

Sorry I missed today's pNFS call.

-->Andy

Quoted 113 lines · Benny Halevy
Quoted 44 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the pyNFS server and the testclient framework.

Still todo:

Quoted 8 lines

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and layoutcommit do not pass nfs_stste to the error handlers.

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit

      BUG_ON(lo->refcount <= 0);

in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING: layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]  [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20  EFLAGS: 00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX: ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI: ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08: 0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11: 0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14: ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS:  0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3: 0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo ffff88007525c000, task ffff88007d988000)
Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8 ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18 ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release 
+0x43/0x68 [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>] nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>] rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/0x66 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task 
+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute 
+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule 
+0x15/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread 
+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ? rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ? autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ? spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread 
+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67 48 8b 82 48 03 00 00 f6 05
Jun 24 12:07:26 tl2 kernel: RIP  [<ffffffffa05d0ea4>] put_layout 
+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#17 lore raw
Quoted 6 lines · Andy Adamson

Hi Benny

I have not been able to reproduce this BUG. I've tried against the files pyNFS server with return_on_close False as well a True, and against a GFS2/pNFS cluster with write layouts turned on.

Quoted 6 lines

Patch 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch calls put_lseg when I/O to a DS fails. I tested this using the pyNFS files layout server and blocking the DS with iptables. I think this is the only change in this patch set that would affect the refcounting.

Are you able to reproduce the BUG?

The easiest way I found to reproduce this bug is running the cthon tests on a locally mounted file system exported over PNFSD_LOCAL_EXPORT. The test machine is a dual core SMP machine.
Are you testing over a VM? Is it uni-processor?

Benny

Quoted 188 lines

-->Andy

Quoted 173 lines · William A. (Andy) Adamson

OK - I'll look into it.

Sorry I missed today's pNFS call.

-->Andy

Quoted 159 lines · Benny Halevy
Quoted 51 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the- MDS.patch
was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints
the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from
NFS4ERR_DEAD_SESSION with the
pyNFS server and the testclient framework.

Still todo:

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
layoutcommit do not pass nfs_stste to the error handlers.

Quoted 5 lines

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit

      BUG_ON(lo->refcount <= 0);

in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING:  
layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/ 
git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP  
DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/ 
initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd  

exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]

Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted  
2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]   
[<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20  EFLAGS:  
00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX:  
ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI:  
ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08:  
0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11:  
0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14:  
ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS:  0000000000000000(0000)  
GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0:  
000000008005003b
Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3:  
0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1:  
0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6:  
00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo  
ffff88007525c000, task ffff88007d988000)
Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8  
ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18  
ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c  
ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release 
+0x43/0x68 [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>]  
nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>]  
rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/ 
0x66 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task 
+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute 
+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule 
+0x15/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread 
+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ?  
rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ?  
autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ?  
spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread 
+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>]  
kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ?  
kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24  
01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04  
0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67  
48 8b 82 48 03 00 00 f6 05
Jun 24 12:07:26 tl2 kernel: RIP  [<ffffffffa05d0ea4>] put_layout 
+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux- nfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#18 lore raw

On Mon, Jun 28, 2010 at 2:53 PM, Benny Halevy <[email protected]> wro= te:

Quoted 1 line

On Jun. 28, 2010, 19:44 +0300, Andy Adamson <[email protected]> wrote=

:

Quoted 9 lines
Quoted 9 lines

Hi Benny

I have not been able to reproduce this BUG. I've tried against the files pyNFS server with return_on_close False as well a True, and against a GFS2/pNFS cluster with write layouts turned on.

Patch 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch calls put_lseg when I/O to a DS fails. I tested this using the pyNFS files layout server and blocking the DS with iptables. I think this =

is

Quoted 5 lines
Quoted 3 lines

the only change in this patch set that would affect the refcounting.

Are you able to reproduce the BUG?

The easiest way I found to reproduce this bug is running the cthon te=

sts

Quoted 3 lines

on a locally mounted file system exported over PNFSD_LOCAL_EXPORT. The test machine is a dual core SMP machine.
Are you testing over a VM? =A0Is it uni-processor?

Its a VM with one processor, but with SMP support turned on in the kernel. I just added a processor and will try re-running tests.

-->Andy

Quoted 26 lines

Benny

Quoted 23 lines

-->Andy

Quoted 18 lines · William A. (Andy) Adamson

OK - I'll look into it.

Sorry I missed today's pNFS call.

-->Andy

Quoted 10 lines · Benny Halevy
Quoted 9 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-th=

e-

Quoted 53 lines
Quoted 53 lines
Quoted 53 lines
Quoted 53 lines
Quoted 42 lines

MDS.patch
was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints
the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from
NFS4ERR_DEAD_SESSION with the
pyNFS server and the testclient framework.

Still todo:

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
layoutcommit do not pass nfs_stste to the error handlers.

Quoted 5 lines

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit
=A0 =A0 =A0 BUG_ON(lo->refcount <=3D 0);
in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING:
layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy=

/

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!

Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP
DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/
initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfs=

d

Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines

exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]

Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted
2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]
[<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20 =A0EFLAGS:
00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX:
ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI:
ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08:
0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11:
0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14:
ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS: =A00000000000000000(0000)
GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS: =A00010 DS: 0000 ES: 0000 CR0:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

000000008005003b

Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3:
0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinf=

o

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

ffff88007525c000, task ffff88007d988000)

Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8
ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18
ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c
ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_relea=

se

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

+0x43/0x68 [nfs]

Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>]
nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>]
rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5=

e/

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

0x66 [sunrpc]

Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task
+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute
+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedul=

e

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

+0x15/0x17 [sunrpc]

Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread
+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ?
rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ?

autoremove_wake_function+0x0/0x39

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ?
spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread
+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>]
kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x8=

7

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ?
kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24

01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 6=

7

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

48 8b 82 48 03 00 00 f6 05

Jun 24 12:07:26 tl2 kernel: RIP =A0[<ffffffffa05d0ea4>] put_layout
+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---

--
To unsubscribe from this list: send the line "unsubscribe linux- nfs" in
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht=

ml

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs=

"

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

in
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm=

l

Quoted 3 lines
Quoted 3 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs"=

in

Quoted 4 lines
Quoted 2 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" =

in

Quoted 3 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): AANLkTilDLWK8rfwzlI8xJJUckxljCqgmblAYj9ANOMnb-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#20 lore raw
Quoted 1 line · William A. (Andy) Adamson

On Mon, Jun 28, 2010 at 2:53 PM, Benny Halevy <[email protected]> w=

rote:

Quoted 1 line
Quoted 1 line

On Jun. 28, 2010, 19:44 +0300, Andy Adamson <[email protected]> wrot=

e:

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi Benny

I have not been able to reproduce this BUG. I've tried against the files pyNFS server with return_on_close False as well a True, and against a GFS2/pNFS cluster with write layouts turned on.

Patch 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patc=

h

Quoted 1 line
Quoted 1 line
Quoted 1 line

calls put_lseg when I/O to a DS fails. I tested this using the pyNF=

S

Quoted 1 line
Quoted 1 line
Quoted 1 line

files layout server and blocking the DS with iptables. I think this=

is

Quoted 1 line
Quoted 1 line
Quoted 1 line

the only change in this patch set that would affect the refcounting=

=2E

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

Are you able to reproduce the BUG?

The easiest way I found to reproduce this bug is running the cthon t=

ests

Quoted 6 lines
Quoted 3 lines

on a locally mounted file system exported over PNFSD_LOCAL_EXPORT. The test machine is a dual core SMP machine.
Are you testing over a VM? =A0Is it uni-processor?

Its a VM with one processor, but with SMP support turned on in the kernel. I just added a processor and will try re-running tests.

Added a processor - all cthon tests succeeded. Just to be clear, I'm testing the client pnfs-submit branch.

-->Andy

Quoted 30 lines

-->Andy

Quoted 27 lines

Benny

Quoted 24 lines

-->Andy

Quoted 19 lines · William A. (Andy) Adamson

OK - I'll look into it.

Sorry I missed today's pNFS call.

-->Andy

On Thu, Jun 24, 2010 at 9:14 AM, Benny Halevy <[email protected]=

Quoted 10 lines · wrote:
Quoted 9 lines · On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-t=

he-

Quoted 53 lines
Quoted 53 lines
Quoted 53 lines
Quoted 53 lines
Quoted 53 lines
Quoted 42 lines

MDS.patch
was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints
the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from
NFS4ERR_DEAD_SESSION with the
pyNFS server and the testclient framework.

Still todo:

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
layoutcommit do not pass nfs_stste to the error handlers.

Quoted 5 lines

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Andy, I've hit
=A0 =A0 =A0 BUG_ON(lo->refcount <=3D 0);
in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING:
layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalev=

y/

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!

Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP
DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/
initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nf=

sd

Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines
Quoted 30 lines

exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]

Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted
2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]
[<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20 =A0EFLAGS:
00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX:
ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI:
ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08:
0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11:
0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14:
ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS: =A00000000000000000(0000)
GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS: =A00010 DS: 0000 ES: 0000 CR0:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

000000008005003b

Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3:
0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadin=

fo

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

ffff88007525c000, task ffff88007d988000)

Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8
ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18
ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c
ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_rele=

ase

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

+0x43/0x68 [nfs]

Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>]
nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>]
rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x=

5e/

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

0x66 [sunrpc]

Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task
+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute
+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedu=

le

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

+0x15/0x17 [sunrpc]

Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread
+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ?
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

rpc_async_schedule+0x0/0x17 [sunrpc]

Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ?
autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ?
spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread
+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x8=

7

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>]
kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x=

87

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ?
kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 2=

4

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 =

67

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

48 8b 82 48 03 00 00 f6 05

Jun 24 12:07:26 tl2 kernel: RIP =A0[<ffffffffa05d0ea4>] put_layou=

t

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

+0x2f/0xa7 [nfs]

Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---

--
To unsubscribe from this list: send the line "unsubscribe linux- nfs" in
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.h=

tml

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

--
To unsubscribe from this list: send the line "unsubscribe linux-nf=

s"

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

in
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht=

ml

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs=

" in

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm=

l

Quoted 2 lines
Quoted 2 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs"=

in

Quoted 4 lines
Quoted 3 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html

#21 lore raw
In reply to [email protected]
Quoted 5 lines · On 2010-06-23 22:21, [email protected] wrote:On Jun. 23, 2010, 22:21 +0300…

Responded to comments, added a 2 cleanup patchses

Plus some code cleanup 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch

Quoted 7 lines

and some bug fixes 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch

NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch

All but patch 3/10 (as per your request) are merged at pnfs-all-2.6.35-rc3-2010-07-01

Thanks!

Benny

Quoted 40 lines

Remove unused (by file layout) encode_layoutreturn io operation 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch

0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch

Note: pnfs4_proc_layoutget is only called by send_layout() which prints the status. 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch

Add back encode_layoutreturn io operation 0008-pnfs-post-submit-restore-encode_layoutreturn.patch

New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch

This gets rid of a frame stack warning; 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch

Testing:
---------

CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests

Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the pyNFS server and the testclient framework.

Still todo:

Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and layoutcommit do not pass nfs_stste to the error handlers.

Handle NFS4ERR_BAD_LAYOUT.

CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.

-->Andy

Signature · 9 lines
Benny Halevy
Software Architect
Panasas, Inc.
[email protected]
Tel/Fax: +972-3-647-8340
Mobile: +972-54-802-8340

Panasas: The Leader in Parallel Storage
www.panasas.com