Skip to contentSkip to conversation

nfsd41 backchannel patches for 2.6.32

PATCH Superseded Benny Halevy 8 participants 64 messages Sep 4, 2009 – Sep 17, 2009 8 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover nfsd41 backchannel patches for 2.6.32

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Benny

#1 lore raw

PATCH 1/10 nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h

Move struct rpc_buffer's definition into a sunrpc.h, a common, internal header file, in preparation for supporting the nfsv4.1 backchannel.

[nfs41: sunrpc: #include <linux/net.h> from sunrpc.h]
---
 net/sunrpc/sched.c  |    7 ++-----
 net/sunrpc/sunrpc.h |   10 ++++++++++
 2 files changed, 12 insertions(+), 5 deletions(-)
net/sunrpc/sched.c +2 -5
index 8f459ab..cef74ba 100644
@@ -21,6 +21,8 @@
 
 #include <linux/sunrpc/clnt.h>
 
+#include "sunrpc.h"
+
 #ifdef RPC_DEBUG
 #define RPCDBG_FACILITY		RPCDBG_SCHED
 #define RPC_TASK_MAGIC_ID	0xf00baa
@@ -711,11 +713,6 @@ static void rpc_async_schedule(struct work_struct *work)
 	__rpc_execute(container_of(work, struct rpc_task, u.tk_work));
 }
 
-struct rpc_buffer {
-	size_t	len;
-	char	data[];
-};
-
 /**
  * rpc_malloc - allocate an RPC buffer
  * @task: RPC task that will use this buffer
net/sunrpc/sunrpc.h +10 -0
index 5d9dd74..13171e6 100644
@@ -27,6 +27,16 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 #ifndef _NET_SUNRPC_SUNRPC_H
 #define _NET_SUNRPC_SUNRPC_H
 
+#include <linux/net.h>
+
+/*
+ * Header for dynamically allocated rpc buffers.
+ */
+struct rpc_buffer {
+	size_t	len;
+	char	data[];
+};
+
 static inline int rpc_reply_expected(struct rpc_task *task)
 {
 	return (task->tk_msg.rpc_proc != NULL) &&
Signature · 1 line
1.6.4
#2 lore raw

PATCH 2/10 nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to Benny Halevy

From: Rahul Iyer <[email protected]>

When the call direction is a reply, copy the xid and call direction into the req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns rpc_garbage.

[get rid of CONFIG_NFSD_V4_1]
[sunrpc: refactoring of svc_tcp_recvfrom]
[nfsd41: sunrpc: create common send routine for the fore and the back channels]
[nfsd41: sunrpc: Use free_page() to free server backchannel pages]
[nfsd41: sunrpc: Document server backchannel locking]
[nfsd41: sunrpc: remove bc_connect_worker()]

[nfsd41: sunrpc: Define xprt_server_backchannel()[

[nfsd41: sunrpc: remove bc_close and bc_init_auto_disconnect dummy functions]
[nfsd41: sunrpc: eliminate unneeded switch statement in xs_setup_tcp()]
[nfsd41: sunrpc: Don't auto close the server backchannel connection]
[nfsd41: sunrpc: Remove unused functions]
[nfsd41: change bc_sock to bc_xprt]
[nfsd41: sunrpc: move struct rpc_buffer def into a common header file]
[nfsd41: sunrpc: use rpc_sleep in bc_send_request so not to block on mutex]
[removed cosmetic changes]
Cc Trond Myklebust <[email protected]>
---
 include/linux/sunrpc/clnt.h     |    1 +
 include/linux/sunrpc/svc_xprt.h |    1 +
 include/linux/sunrpc/svcsock.h  |    1 +
 include/linux/sunrpc/xprt.h     |    7 ++
 net/sunrpc/clnt.c               |    1 +
 net/sunrpc/sunrpc.h             |    4 +
 net/sunrpc/svc_xprt.c           |    2 +
 net/sunrpc/svcsock.c            |  172 +++++++++++++++++++++++++++-------
 net/sunrpc/xprt.c               |   13 +++
 net/sunrpc/xprtsock.c           |  198 +++++++++++++++++++++++++++++++++++++-
 10 files changed, 359 insertions(+), 41 deletions(-)
include/linux/sunrpc/clnt.h +1 -0
index 3d02558..8ed9642 100644
@@ -114,6 +114,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
include/linux/sunrpc/svc_xprt.h +1 -0
index 2223ae0..5f4e18b 100644
@@ -65,6 +65,7 @@ struct svc_xprt {
 	size_t			xpt_locallen;	/* length of address */
 	struct sockaddr_storage	xpt_remote;	/* remote peer's address */
 	size_t			xpt_remotelen;	/* length of address */
+	struct rpc_wait_queue	xpt_bc_pending;	/* backchannel wait queue */
 };
 
 int	svc_reg_xprt_class(struct svc_xprt_class *);
include/linux/sunrpc/svcsock.h +1 -0
index 04dba23..4b854e2 100644
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt	       *sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
include/linux/sunrpc/xprt.h +7 -0
index c090df4..cfad635 100644
@@ -179,6 +179,7 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 #if defined(CONFIG_NFS_V4_1)
 	struct svc_serv		*bc_serv;       /* The RPC service which will */
 						/* process the callback */
@@ -231,6 +232,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {
@@ -366,6 +368,11 @@ static inline int xprt_test_and_set_binding(struct rpc_xprt *xprt)
 	return test_and_set_bit(XPRT_BINDING, &xprt->state);
 }
 
+static inline int xprt_server_backchannel(struct rpc_xprt *xprt)
+{
+	return xprt->bc_xprt != NULL;
+}
+
 #endif /* __KERNEL__*/
 
 #endif /* _LINUX_SUNRPC_XPRT_H */
net/sunrpc/clnt.c +1 -0
index c1e467e..7389804 100644
@@ -288,6 +288,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_xprt = args->bc_xprt,
 	};
 	char servername[48];
 
net/sunrpc/sunrpc.h +4 -0
index 13171e6..90c292e 100644
@@ -43,5 +43,9 @@ static inline int rpc_reply_expected(struct rpc_task *task)
 		(task->tk_msg.rpc_proc->p_decode != NULL);
 }
 
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset);
+
 #endif /* _NET_SUNRPC_SUNRPC_H */
 
net/sunrpc/svc_xprt.c +2 -0
index 912dea5..df124f7 100644
@@ -160,6 +160,7 @@ void svc_xprt_init(struct svc_xprt_class *xcl, struct svc_xprt *xprt,
 	mutex_init(&xprt->xpt_mutex);
 	spin_lock_init(&xprt->xpt_lock);
 	set_bit(XPT_BUSY, &xprt->xpt_flags);
+	rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending");
 }
 EXPORT_SYMBOL_GPL(svc_xprt_init);
 
@@ -810,6 +811,7 @@ int svc_send(struct svc_rqst *rqstp)
 	else
 		len = xprt->xpt_ops->xpo_sendto(rqstp);
 	mutex_unlock(&xprt->xpt_mutex);
+	rpc_wake_up(&xprt->xpt_bc_pending);
 	svc_xprt_release(rqstp);
 
 	if (len == -ECONNREFUSED || len == -ENOTCONN || len == -EAGAIN)
net/sunrpc/svcsock.c +137 -35

241 lines not on this page · raw message

net/sunrpc/xprt.c +13 -0
index f412a85..7b0cf70 100644
@@ -599,6 +599,9 @@ static void xprt_autoclose(struct work_struct *work)
 	struct rpc_xprt *xprt =
 		container_of(work, struct rpc_xprt, task_cleanup);
 
+	if (xprt_server_backchannel(xprt))
+		return;
+
 	xprt->ops->close(xprt);
 	clear_bit(XPRT_CLOSE_WAIT, &xprt->state);
 	xprt_release_write(xprt, NULL);
@@ -669,6 +672,9 @@ xprt_init_autodisconnect(unsigned long data)
 {
 	struct rpc_xprt *xprt = (struct rpc_xprt *)data;
 
+	if (xprt_server_backchannel(xprt))
+		return;
+
 	spin_lock(&xprt->transport_lock);
 	if (!list_empty(&xprt->recv) || xprt->shutdown)
 		goto out_abort;
@@ -1103,6 +1109,13 @@ found:
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
 
+	/*
+	 * Since we don't want connections for the backchannel, we set
+	 * the xprt status to connected
+	 */
+	if (args->bc_xprt)
+		xprt_set_connected(xprt);
+
 	return xprt;
 }
 
net/sunrpc/xprtsock.c +192 -6

236 lines not on this page · raw message

Signature · 1 line
1.6.4
#4 lore raw

PATCH 3/10 nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition

In reply to Benny Halevy
---
 fs/nfsd/nfs4callback.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
fs/nfsd/nfs4callback.c +1 -1
index 81d1c52..63bb384 100644
@@ -56,7 +56,7 @@
 /* Index of predefined Linux callback client operations */
 
 enum {
-        NFSPROC4_CLNT_CB_NULL = 0,
+	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
 };
 
Signature · 1 line
1.6.4
#5 lore raw

PATCH 4/10 nfsd41: Backchannel: callback infrastructure

In reply to Benny Halevy

From: Andy Adamson <[email protected]>

Keep the xprt used for create_session in cl_cb_xprt. Mark cl_callback.cb_minorversion = 1 and remember the client provided cl_callback.cb_prog rpc program number. Use it to probe the callback path.

Use the client's network address to initialize as the callback's address as expected by the xprt creation routines.

Define xdr sizes and code nfs4_cb_compound header to be able to send a null callback rpc.

[get callback minorversion from fore channel's]
[nfsd41: change bc_sock to bc_xprt]
[pulled definition for cl_cb_xprt]
[nfsd41: set up backchannel's cb_addr]
---
 fs/nfsd/nfs4callback.c     |   24 ++++++++++++++++++++++--
 fs/nfsd/nfs4state.c        |   14 ++++++++++++++
 include/linux/nfsd/state.h |    3 +++
 3 files changed, 39 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +22 -2
index 63bb384..f25ef3c 100644
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -137,11 +148,13 @@ xdr_error:                                      \
 } while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -399,6 +412,13 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	if (cb->cb_minorversion)
+		args.bc_xprt = clp->cl_cb_xprt;
+
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
+
 	/* Create RPC client */
 	client = rpc_create(&args);
 	if (IS_ERR(client)) {
fs/nfsd/nfs4state.c +14 -0
index 46e9ac5..e4c3223 100644
@@ -706,6 +706,8 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1321,6 +1323,18 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			rpc_copy_addr(
+				(struct sockaddr *)&unconf->cl_cb_conn.cb_addr,
+				sa);
+			unconf->cl_cb_conn.cb_addrlen = svc_addr_len(sa);
+			unconf->cl_cb_conn.cb_minorversion =
+				cstate->minorversion;
+			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
include/linux/nfsd/state.h +3 -0
index 70ef5f4..243277b 100644
@@ -212,6 +212,9 @@ struct nfs4_client {
 	struct nfsd4_clid_slot	cl_cs_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
+
+	/* for nfs41 callbacks */
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#6 lore raw

PATCH 5/10 nfsd41: Backchannel: Add sequence arguments to callback RPC arguments

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Follow the model we use in the client. Make the sequence arguments part of the regular RPC arguments. None of the callbacks that are soon to be implemented expect results that need to be passed back to the caller, so we don't define a separate RPC results structure. For session validation, the cb_sequence decoding will use a pointer to the sequence arguments that are part of the RPC argument.

[define struct nfsd4_cb_sequence here]
---
 fs/nfsd/nfs4callback.c     |    5 +++++
 include/linux/nfsd/state.h |    6 ++++++
 2 files changed, 11 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +5 -0
index f25ef3c..32ea3f5 100644
@@ -92,6 +92,11 @@ enum nfs_cb_opnum4 {
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
+struct nfs4_rpc_args {
+	void				*args_op;
+	struct nfsd4_cb_sequence	args_seq;
+};
+
 /*
 * Generic encode routines from fs/nfs/nfs4xdr.c
 */
include/linux/nfsd/state.h +6 -0
index 243277b..f69ea48 100644
@@ -60,6 +60,12 @@ typedef struct {
 #define si_stateownerid   si_opaque.so_stateownerid
 #define si_fileid         si_opaque.so_fileid
 
+struct nfsd4_cb_sequence {
+	/* args/res */
+	u32			cbs_minorversion;
+	struct nfs4_client	*cbs_clp;
+};
+
 struct nfs4_delegation {
 	struct list_head	dl_perfile;
 	struct list_head	dl_perclnt;
Signature · 1 line
1.6.4
#7 lore raw

PATCH 6/10 nfsd41: Backchannel: Server backchannel RPC wait queue

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

RPC callback requests will wait on this wait queue if the backchannel is out of slots.

---
 fs/nfsd/nfs4state.c        |    2 ++
 include/linux/nfsd/state.h |    4 ++++
 2 files changed, 6 insertions(+), 0 deletions(-)
fs/nfsd/nfs4state.c +2 -0
index e4c3223..957f6e5 100644
@@ -779,6 +779,8 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_delegations);
 	INIT_LIST_HEAD(&clp->cl_sessions);
 	INIT_LIST_HEAD(&clp->cl_lru);
+	clear_bit(0, &clp->cl_cb_slot_busy);
+	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
 	return clp;
 }
 
include/linux/nfsd/state.h +4 -0
index f69ea48..234e9af 100644
@@ -220,7 +220,11 @@ struct nfs4_client {
 	struct nfs4_sessionid	cl_sessionid;
 
 	/* for nfs41 callbacks */
+	/* We currently support a single back channel with a single slot */
+	unsigned long		cl_cb_slot_busy;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
+	struct rpc_wait_queue	cl_cb_waitq;	/* backchannel callers may */
+						/* wait here for slots */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#8 lore raw

PATCH 7/10 nfsd41: Backchannel: Setup sequence information

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Follows the model used by the NFS client. Setup the RPC prepare and done function pointers so that we can populate the sequence information if minorversion == 1. rpc_run_task() is then invoked directly just like existing NFS client operations do.

nfsd4_cb_prepare() determines if the sequence information needs to be setup. If the slot is in use, it adds itself to the wait queue.

nfsd4_cb_done() wakes anyone sleeping on the callback channel wait queue after our RPC reply has been received. It also sets the task message result pointer to NULL to clearly indicate we're done using it.

[define and initialize cl_cb_seq_nr here]
[pulled out unused defintion of nfsd4_cb_done]
---
 fs/nfsd/nfs4callback.c     |   62 ++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    1 +
 3 files changed, 64 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +62 -0
index 32ea3f5..da36a46 100644
@@ -521,6 +521,67 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	do_probe_callback(clp);
 }
 
+/*
+ * There's currently a single callback channel slot.
+ * If the slot is available, then mark it busy.  Otherwise, set the
+ * thread for sleeping on the callback RPC wait queue.
+ */
+static int nfsd41_cb_setup_sequence(struct nfs4_client *clp,
+		struct rpc_task *task)
+{
+	struct nfs4_rpc_args *args = task->tk_msg.rpc_argp;
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+	int status = 0;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	if (test_and_set_bit(0, &clp->cl_cb_slot_busy) != 0) {
+		rpc_sleep_on(&clp->cl_cb_waitq, task, NULL);
+		dprintk("%s slot is busy\n", __func__);
+		status = -EAGAIN;
+		goto out;
+	}
+
+	/*
+	 * We'll need the clp during XDR encoding and decoding,
+	 * and the sequence during decoding to verify the reply
+	 */
+	args->args_seq.cbs_clp = clp;
+	task->tk_msg.rpc_resp = &args->args_seq;
+
+out:
+	dprintk("%s status=%d\n", __func__, status);
+	return status;
+}
+
+/*
+ * TODO: cb_sequence should support referring call lists, cachethis, multiple
+ * slots, and mark callback channel down on communication errors.
+ */
+static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+	struct nfs4_rpc_args *args = task->tk_msg.rpc_argp;
+	u32 minorversion = clp->cl_cb_conn.cb_minorversion;
+	int status = 0;
+
+	args->args_seq.cbs_minorversion = minorversion;
+	if (minorversion) {
+		status = nfsd41_cb_setup_sequence(clp, task);
+		if (status) {
+			if (status != -EAGAIN) {
+				/* terminate rpc task */
+				task->tk_status = status;
+				task->tk_action = NULL;
+			}
+			return;
+		}
+	}
+	rpc_call_start(task);
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
@@ -560,6 +621,7 @@ static void nfsd4_cb_recall_release(void *calldata)
 }
 
 static const struct rpc_call_ops nfsd4_cb_recall_ops = {
+	.rpc_call_prepare = nfsd4_cb_prepare,
 	.rpc_call_done = nfsd4_cb_recall_done,
 	.rpc_release = nfsd4_cb_recall_release,
 };
fs/nfsd/nfs4state.c +1 -0
index 957f6e5..b2ffa3b 100644
@@ -1335,6 +1335,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			unconf->cl_cb_conn.cb_minorversion =
 				cstate->minorversion;
 			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			unconf->cl_cb_seq_nr = 1;
 			nfsd4_probe_callback(unconf);
 		}
 		conf = unconf;
include/linux/nfsd/state.h +1 -0
index 234e9af..b621428 100644
@@ -222,6 +222,7 @@ struct nfs4_client {
 	/* for nfs41 callbacks */
 	/* We currently support a single back channel with a single slot */
 	unsigned long		cl_cb_slot_busy;
+	u32			cl_cb_seq_nr;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 	struct rpc_wait_queue	cl_cb_waitq;	/* backchannel callers may */
 						/* wait here for slots */
Signature · 1 line
1.6.4
#9 lore raw

PATCH 8/10 nfsd41: Backchannel: cb_sequence callback

In reply to Benny Halevy

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Note: highest slot id and target highest slot id do not have to be 0 as was previously implemented. They can be greater than what the nfs server sent if the client supports a larger slot table on the backchannel. At this point we just ignore that.

[Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
[fixed indentation]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]

[nfsd41: fix verification of CB_SEQUENCE highest slot id[

[nfsd41: Backchannel: Remove old backchannel serialization]
[nfsd41: Backchannel: First callback sequence ID should be 1]
[nfsd41: decode_cb_sequence does not need to actually decode ignored fields]
---
 fs/nfsd/nfs4callback.c |   72 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 72 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +72 -0
index da36a46..2282594 100644
@@ -256,6 +256,27 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_delegation *dp,
 	hdr->nops++;
 }
 
+static void
+encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
+		   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	if (hdr->minorversion == 0)
+		return;
+
+	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
+
+	WRITE32(OP_CB_SEQUENCE);
+	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(args->cbs_clp->cl_cb_seq_nr);
+	WRITE32(0);		/* slotid, always 0 */
+	WRITE32(0);		/* highest slotid always 0 */
+	WRITE32(0);		/* cachethis always 0 */
+	WRITE32(0); /* FIXME: support referring_call_lists */
+	hdr->nops++;
+}
+
 static int
 nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -317,6 +338,57 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
 	return 0;
 }
 
+/*
+ * Our current back channel implmentation supports a single backchannel
+ * with a single slot.
+ */
+static int
+decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
+		   struct rpc_rqst *rqstp)
+{
+	struct nfs4_sessionid id;
+	int status;
+	u32 dummy;
+	__be32 *p;
+
+	if (res->cbs_minorversion == 0)
+		return 0;
+
+	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
+	if (status)
+		return status;
+
+	/*
+	 * If the server returns different values for sessionID, slotID or
+	 * sequence number, the server is looney tunes.
+	 */
+	status = -ESERVERFAULT;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	memcpy(id.data, p, NFS4_MAX_SESSIONID_LEN);
+	p += XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN);
+	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
+		   NFS4_MAX_SESSIONID_LEN)) {
+		dprintk("%s Invalid session id\n", __func__);
+		goto out;
+	}
+	READ32(dummy);
+	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
+		dprintk("%s Invalid sequence number\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid slotid\n", __func__);
+		goto out;
+	}
+	/* FIXME: process highest slotid and target highest slotid */
+	status = 0;
+out:
+	return status;
+}
+
+
 static int
 nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 {
Signature · 1 line
1.6.4
#10 lore raw

PATCH 9/10 nfsd41: Backchannel: Implement cb_recall over NFSv4.1

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

[nfsd41: cb_recall callback]
[Share v4.0 and v4.1 back channel xdr]
[Share v4.0 and v4.1 back channel xdr]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]
[nfsd41: conditionally decode_sequence in nfs4_xdr_dec_cb_recall]
[nfsd41: Backchannel: Add sequence arguments to callback RPC arguments]
[pulled-in definition of nfsd4_cb_done]
---
 fs/nfsd/nfs4callback.c |   52 ++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 48 insertions(+), 4 deletions(-)
fs/nfsd/nfs4callback.c +48 -4
index 2282594..7295af3 100644
@@ -288,15 +288,19 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_delegation *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_delegation *args = rpc_args->args_op;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->dl_ident,
+		.minorversion = rpc_args->args_seq.cbs_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, &rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -396,7 +400,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		struct nfsd4_cb_sequence *seq)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -406,6 +411,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	if (seq) {
+		status = decode_cb_sequence(&xdr, seq, rqstp);
+		if (status)
+			goto out;
+	}
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -654,11 +664,34 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
 	rpc_call_start(task);
 }
 
+static void nfsd4_cb_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+
+	dprintk("%s: minorversion=%d\n", __func__,
+		clp->cl_cb_conn.cb_minorversion);
+
+	if (clp->cl_cb_conn.cb_minorversion) {
+		/* No need for lock, access serialized in nfsd4_cb_prepare */
+		++clp->cl_cb_seq_nr;
+		clear_bit(0, &clp->cl_cb_slot_busy);
+		rpc_wake_up_next(&clp->cl_cb_waitq);
+		dprintk("%s: freed slot, new seqid=%d\n", __func__,
+			clp->cl_cb_seq_nr);
+
+		/* We're done looking into the sequence information */
+		task->tk_msg.rpc_resp = NULL;
+	}
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
 	struct nfs4_client *clp = dp->dl_client;
 
+	nfsd4_cb_done(task, calldata);
+
 	switch (task->tk_status) {
 	case -EIO:
 		/* Network partition? */
@@ -671,16 +704,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -706,16 +742,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
 	int status;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args) {
+		status = -ENOMEM;
+		goto out;
+	}
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
 		put_nfs4_client(clp);
 		nfs4_put_delegation(dp);
Signature · 1 line
1.6.4
#11 lore raw

PATCH 10/10 nfsd41: Refactor create_client()

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Move common initialization of 'struct nfs4_client' inside create_client().

[nfsd41: Remember the auth flavor to use for callbacks]
---
 fs/nfsd/nfs4state.c |   89 ++++++++++++++++++++++++++-------------------------
 1 files changed, 45 insertions(+), 44 deletions(-)
fs/nfsd/nfs4state.c +45 -44
index b2ffa3b..f344de2 100644
@@ -763,27 +763,6 @@ expire_client(struct nfs4_client *clp)
 	put_nfs4_client(clp);
 }
 
-static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
-{
-	struct nfs4_client *clp;
-
-	clp = alloc_client(name);
-	if (clp == NULL)
-		return NULL;
-	memcpy(clp->cl_recdir, recdir, HEXDIR_LEN);
-	atomic_set(&clp->cl_count, 1);
-	atomic_set(&clp->cl_cb_conn.cb_set, 0);
-	INIT_LIST_HEAD(&clp->cl_idhash);
-	INIT_LIST_HEAD(&clp->cl_strhash);
-	INIT_LIST_HEAD(&clp->cl_openowners);
-	INIT_LIST_HEAD(&clp->cl_delegations);
-	INIT_LIST_HEAD(&clp->cl_sessions);
-	INIT_LIST_HEAD(&clp->cl_lru);
-	clear_bit(0, &clp->cl_cb_slot_busy);
-	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
-	return clp;
-}
-
 static void copy_verf(struct nfs4_client *target, nfs4_verifier *source)
 {
 	memcpy(target->cl_verifier.data, source->data,
@@ -846,6 +825,46 @@ static void gen_confirm(struct nfs4_client *clp)
 	*p++ = i++;
 }
 
+static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir,
+		struct svc_rqst *rqstp, nfs4_verifier *verf)
+{
+	struct nfs4_client *clp;
+	struct sockaddr *sa = svc_addr(rqstp);
+	char *princ;
+
+	clp = alloc_client(name);
+	if (clp == NULL)
+		return NULL;
+
+	princ = svc_gss_principal(rqstp);
+	if (princ) {
+		clp->cl_principal = kstrdup(princ, GFP_KERNEL);
+		if (clp->cl_principal == NULL) {
+			free_client(clp);
+			return NULL;
+		}
+	}
+
+	memcpy(clp->cl_recdir, recdir, HEXDIR_LEN);
+	atomic_set(&clp->cl_count, 1);
+	atomic_set(&clp->cl_cb_conn.cb_set, 0);
+	INIT_LIST_HEAD(&clp->cl_idhash);
+	INIT_LIST_HEAD(&clp->cl_strhash);
+	INIT_LIST_HEAD(&clp->cl_openowners);
+	INIT_LIST_HEAD(&clp->cl_delegations);
+	INIT_LIST_HEAD(&clp->cl_sessions);
+	INIT_LIST_HEAD(&clp->cl_lru);
+	clear_bit(0, &clp->cl_cb_slot_busy);
+	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
+	copy_verf(clp, verf);
+	rpc_copy_addr((struct sockaddr *) &clp->cl_addr, sa);
+	clp->cl_flavor = rqstp->rq_flavor;
+	copy_cred(&clp->cl_cred, &rqstp->rq_cred);
+	gen_confirm(clp);
+
+	return clp;
+}
+
 static int check_name(struct xdr_netobj name)
 {
 	if (name.len == 0) 
@@ -1193,17 +1212,13 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 
 out_new:
 	/* Normal case */
-	new = create_client(exid->clname, dname);
+	new = create_client(exid->clname, dname, rqstp, &verf);
 	if (new == NULL) {
 		status = nfserr_serverfault;
 		goto out;
 	}
 
-	copy_verf(new, &verf);
-	copy_cred(&new->cl_cred, &rqstp->rq_cred);
-	rpc_copy_addr((struct sockaddr *) &new->cl_addr, sa);
 	gen_clid(new);
-	gen_confirm(new);
 	add_to_unconfirmed(new, strhashval);
 out_copy:
 	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
@@ -1477,7 +1492,6 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	unsigned int 		strhashval;
 	struct nfs4_client	*conf, *unconf, *new;
 	__be32 			status;
-	char			*princ;
 	char                    dname[HEXDIR_LEN];
 	
 	if (!check_name(clname))
@@ -1522,7 +1536,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 */
 		if (unconf)
 			expire_client(unconf);
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
@@ -1539,7 +1553,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			 */
 			expire_client(unconf);
 		}
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		copy_clid(new, conf);
@@ -1549,7 +1563,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 * probable client reboot; state will be removed if
 		 * confirmed.
 		 */
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
@@ -1560,24 +1574,11 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 * confirmed.
 		 */
 		expire_client(unconf);
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
 	}
-	copy_verf(new, &clverifier);
-	rpc_copy_addr((struct sockaddr *) &new->cl_addr, sa);
-	new->cl_flavor = rqstp->rq_flavor;
-	princ = svc_gss_principal(rqstp);
-	if (princ) {
-		new->cl_principal = kstrdup(princ, GFP_KERNEL);
-		if (new->cl_principal == NULL) {
-			free_client(new);
-			goto out;
-		}
-	}
-	copy_cred(&new->cl_cred, &rqstp->rq_cred);
-	gen_confirm(new);
 	gen_callback(new, setclid, rpc_get_scope_id(sa));
 	add_to_unconfirmed(new, strhashval);
 	setclid->se_clientid.cl_boot = new->cl_clientid.cl_boot;
Signature · 1 line
1.6.4
#3 lore raw

Re: [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to Benny Halevy
Quoted 12 lines · Benny Halevy

From: Rahul Iyer <[email protected]>

When the call direction is a reply, copy the xid and call direction into the req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns rpc_garbage.

Quoted 13 lines
[get rid of CONFIG_NFSD_V4_1]
[sunrpc: refactoring of svc_tcp_recvfrom]
[nfsd41: sunrpc: create common send routine for the fore and the back channels]
[nfsd41: sunrpc: Use free_page() to free server backchannel pages]

OK... Let's not go overboard with all the 'Signed-off-by's here... I'm sure this ping-ponging between Alexandros and Ricardo can be consolidated a bit. Particularly given that they work for the same company...

Quoted 96 lines
[nfsd41: sunrpc: Document server backchannel locking]
[nfsd41: sunrpc: remove bc_connect_worker()]

[nfsd41: sunrpc: Define xprt_server_backchannel()[

[nfsd41: sunrpc: remove bc_close and bc_init_auto_disconnect dummy functions]
[nfsd41: sunrpc: eliminate unneeded switch statement in xs_setup_tcp()]
[nfsd41: sunrpc: Don't auto close the server backchannel connection]
[nfsd41: sunrpc: Remove unused functions]
[nfsd41: change bc_sock to bc_xprt]
[nfsd41: sunrpc: move struct rpc_buffer def into a common header file]
[nfsd41: sunrpc: use rpc_sleep in bc_send_request so not to block on mutex]
[removed cosmetic changes]
Cc Trond Myklebust <[email protected]>
---
 include/linux/sunrpc/clnt.h     |    1 +
 include/linux/sunrpc/svc_xprt.h |    1 +
 include/linux/sunrpc/svcsock.h  |    1 +
 include/linux/sunrpc/xprt.h     |    7 ++
 net/sunrpc/clnt.c               |    1 +
 net/sunrpc/sunrpc.h             |    4 +
 net/sunrpc/svc_xprt.c           |    2 +
 net/sunrpc/svcsock.c            |  172 +++++++++++++++++++++++++++-------
 net/sunrpc/xprt.c               |   13 +++
 net/sunrpc/xprtsock.c           |  198 +++++++++++++++++++++++++++++++++++++-
 10 files changed, 359 insertions(+), 41 deletions(-)
include/linux/sunrpc/clnt.h +1 -0
index 3d02558..8ed9642 100644
@@ -114,6 +114,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
include/linux/sunrpc/svc_xprt.h +1 -0
index 2223ae0..5f4e18b 100644
@@ -65,6 +65,7 @@ struct svc_xprt {
 	size_t			xpt_locallen;	/* length of address */
 	struct sockaddr_storage	xpt_remote;	/* remote peer's address */
 	size_t			xpt_remotelen;	/* length of address */
+	struct rpc_wait_queue	xpt_bc_pending;	/* backchannel wait queue */
 };
 
 int	svc_reg_xprt_class(struct svc_xprt_class *);
include/linux/sunrpc/svcsock.h +1 -0
index 04dba23..4b854e2 100644
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt	       *sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
include/linux/sunrpc/xprt.h +2 -0
index c090df4..cfad635 100644
@@ -179,6 +179,7 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 #if defined(CONFIG_NFS_V4_1)
 	struct svc_serv		*bc_serv;       /* The RPC service which will */
 						/* process the callback */
@@ -231,6 +232,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {

include/linux/sunrpc/xprt.h static inline int xprt_test_and_set_binding(struct rpc_xprt *xprt) · line 374

Quoted 7 lines
include/linux/sunrpc/xprt.h +4 -0
@@ -366,6 +368,11 @@ static inline int xprt_test_and_set_binding(struct rpc_xprt *xprt)
 	return test_and_set_bit(XPRT_BINDING, &xprt->state);
 }
 
+static inline int xprt_server_backchannel(struct rpc_xprt *xprt)
+{
+	return xprt->bc_xprt != NULL;
+}

xprt_is_server_backchannel()? When I see a function with a name like the above, I tend to assume it will actually return a backchannel object.

Quoted 308 lines
include/linux/sunrpc/xprt.h +1 -0
+
 #endif /* __KERNEL__*/
 
 #endif /* _LINUX_SUNRPC_XPRT_H */
net/sunrpc/clnt.c +1 -0
index c1e467e..7389804 100644
@@ -288,6 +288,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_xprt = args->bc_xprt,
 	};
 	char servername[48];
 
net/sunrpc/sunrpc.h +4 -0
index 13171e6..90c292e 100644
@@ -43,5 +43,9 @@ static inline int rpc_reply_expected(struct rpc_task *task)
 		(task->tk_msg.rpc_proc->p_decode != NULL);
 }
 
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset);
+
 #endif /* _NET_SUNRPC_SUNRPC_H */
 
net/sunrpc/svc_xprt.c +2 -0
index 912dea5..df124f7 100644
@@ -160,6 +160,7 @@ void svc_xprt_init(struct svc_xprt_class *xcl, struct svc_xprt *xprt,
 	mutex_init(&xprt->xpt_mutex);
 	spin_lock_init(&xprt->xpt_lock);
 	set_bit(XPT_BUSY, &xprt->xpt_flags);
+	rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending");
 }
 EXPORT_SYMBOL_GPL(svc_xprt_init);
 
@@ -810,6 +811,7 @@ int svc_send(struct svc_rqst *rqstp)
 	else
 		len = xprt->xpt_ops->xpo_sendto(rqstp);
 	mutex_unlock(&xprt->xpt_mutex);
+	rpc_wake_up(&xprt->xpt_bc_pending);
 	svc_xprt_release(rqstp);
 
 	if (len == -ECONNREFUSED || len == -ENOTCONN || len == -EAGAIN)
net/sunrpc/svcsock.c +137 -35
index 76a380d..ccc5e83 100644
@@ -49,6 +49,7 @@
 #include <linux/sunrpc/msg_prot.h>
 #include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/stats.h>
+#include <linux/sunrpc/xprt.h>
 
 #define RPCDBG_FACILITY	RPCDBG_SVCXPRT
 
@@ -153,49 +154,27 @@ static void svc_set_cmsg_data(struct svc_rqst *rqstp, struct cmsghdr *cmh)
 }
 
 /*
- * Generic sendto routine
+ * send routine intended to be shared by the fore- and back-channel
  */
-static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset)
 {
-	struct svc_sock	*svsk =
-		container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt);
-	struct socket	*sock = svsk->sk_sock;
-	int		slen;
-	union {
-		struct cmsghdr	hdr;
-		long		all[SVC_PKTINFO_SPACE / sizeof(long)];
-	} buffer;
-	struct cmsghdr *cmh = &buffer.hdr;
-	int		len = 0;
 	int		result;
 	int		size;
 	struct page	**ppage = xdr->pages;
 	size_t		base = xdr->page_base;
 	unsigned int	pglen = xdr->page_len;
 	unsigned int	flags = MSG_MORE;
-	RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);
+	int		slen;
+	int		len = 0;
 
 	slen = xdr->len;
 
-	if (rqstp->rq_prot == IPPROTO_UDP) {
-		struct msghdr msg = {
-			.msg_name	= &rqstp->rq_addr,
-			.msg_namelen	= rqstp->rq_addrlen,
-			.msg_control	= cmh,
-			.msg_controllen	= sizeof(buffer),
-			.msg_flags	= MSG_MORE,
-		};
-
-		svc_set_cmsg_data(rqstp, cmh);
-
-		if (sock_sendmsg(sock, &msg, 0) < 0)
-			goto out;
-	}
-
 	/* send head */
 	if (slen == xdr->head[0].iov_len)
 		flags = 0;
-	len = kernel_sendpage(sock, rqstp->rq_respages[0], 0,
+	len = kernel_sendpage(sock, headpage, headoffset,
 				  xdr->head[0].iov_len, flags);
 	if (len != xdr->head[0].iov_len)
 		goto out;
@@ -219,16 +198,58 @@ static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
 		base = 0;
 		ppage++;
 	}
+
 	/* send tail */
 	if (xdr->tail[0].iov_len) {
-		result = kernel_sendpage(sock, rqstp->rq_respages[0],
-					     ((unsigned long)xdr->tail[0].iov_base)
-						& (PAGE_SIZE-1),
-					     xdr->tail[0].iov_len, 0);
-
+		result = kernel_sendpage(sock, tailpage, tailoffset,
+				   xdr->tail[0].iov_len, 0);
 		if (result > 0)
 			len += result;
 	}
+
+out:
+	return len;
+}
+
+
+/*
+ * Generic sendto routine
+ */
+static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
+{
+	struct svc_sock	*svsk =
+		container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt);
+	struct socket	*sock = svsk->sk_sock;
+	union {
+		struct cmsghdr	hdr;
+		long		all[SVC_PKTINFO_SPACE / sizeof(long)];
+	} buffer;
+	struct cmsghdr *cmh = &buffer.hdr;
+	int		len = 0;
+	unsigned long tailoff;
+	unsigned long headoff;
+	RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);
+
+	if (rqstp->rq_prot == IPPROTO_UDP) {
+		struct msghdr msg = {
+			.msg_name	= &rqstp->rq_addr,
+			.msg_namelen	= rqstp->rq_addrlen,
+			.msg_control	= cmh,
+			.msg_controllen	= sizeof(buffer),
+			.msg_flags	= MSG_MORE,
+		};
+
+		svc_set_cmsg_data(rqstp, cmh);
+
+		if (sock_sendmsg(sock, &msg, 0) < 0)
+			goto out;
+	}
+
+	tailoff = ((unsigned long)xdr->tail[0].iov_base) & (PAGE_SIZE-1);
+	headoff = 0;
+	len = svc_send_common(sock, xdr, rqstp->rq_respages[0], headoff,
+			       rqstp->rq_respages[0], tailoff);
+
 out:
 	dprintk("svc: socket %p sendto([%p %Zu... ], %d) = %d (addr %s)\n",
 		svsk, xdr->head[0].iov_base, xdr->head[0].iov_len,
@@ -951,6 +972,57 @@ static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp)
 	return -EAGAIN;
 }
 
+static int svc_process_calldir(struct svc_sock *svsk, struct svc_rqst *rqstp,
+			       struct rpc_rqst **reqpp, struct kvec *vec)
+{
+	struct rpc_rqst *req = NULL;
+	u32 *p;
+	u32 xid;
+	u32 calldir;
+	int len;
+
+	len = svc_recvfrom(rqstp, vec, 1, 8);
+	if (len < 0)
+		goto error;
+
+	p = (u32 *)rqstp->rq_arg.head[0].iov_base;
+	xid = *p++;
+	calldir = *p;
+
+	if (calldir == 0) {
+		/* REQUEST is the most common case */
+		vec[0] = rqstp->rq_arg.head[0];
+	} else {
+		/* REPLY */
+		if (svsk->sk_bc_xprt)
+			req = xprt_lookup_rqst(svsk->sk_bc_xprt, xid);
+
+		if (!req) {
+			printk(KERN_NOTICE
+				"%s: Got unrecognized reply: "
+				"calldir 0x%x sk_bc_xprt %p xid %08x\n",
+				__func__, ntohl(calldir),
+				svsk->sk_bc_xprt, xid);
+			vec[0] = rqstp->rq_arg.head[0];
+			goto out;
+		}
+
+		memcpy(&req->rq_private_buf, &req->rq_rcv_buf,
+		       sizeof(struct xdr_buf));
+		/* copy the xid and call direction */
+		memcpy(req->rq_private_buf.head[0].iov_base,
+		       rqstp->rq_arg.head[0].iov_base, 8);
+		vec[0] = req->rq_private_buf.head[0];
+	}
+ out:
+	vec[0].iov_base += 8;
+	vec[0].iov_len -= 8;
+	len = svsk->sk_reclen - 8;
+ error:
+	*reqpp = req;
+	return len;
+}
+
 /*
  * Receive data from a TCP socket.
  */
@@ -962,6 +1034,7 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	int		len;
 	struct kvec *vec;
 	int pnum, vlen;
+	struct rpc_rqst *req = NULL;
 
 	dprintk("svc: tcp_recv %p data %d conn %d close %d\n",
 		svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags),
@@ -975,9 +1048,27 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	vec = rqstp->rq_vec;
 	vec[0] = rqstp->rq_arg.head[0];
 	vlen = PAGE_SIZE;
+
+	/*
+	 * We have enough data for the whole tcp record. Let's try and read the
+	 * first 8 bytes to get the xid and the call direction. We can use this
+	 * to figure out if this is a call or a reply to a callback. If
+	 * sk_reclen is < 8 (xid and calldir), then this is a malformed packet.
+	 * In that case, don't bother with the calldir and just read the data.
+	 * It will be rejected in svc_process.
+	 */
+	if (len >= 8) {
+		len = svc_process_calldir(svsk, rqstp, &req, vec);
+		if (len < 0)
+			goto err_again;
+		vlen -= 8;
+	}
+
 	pnum = 1;
 	while (vlen < len) {
-		vec[pnum].iov_base = page_address(rqstp->rq_pages[pnum]);
+		vec[pnum].iov_base = (req) ?
+			page_address(req->rq_private_buf.pages[pnum - 1]) :
+			page_address(rqstp->rq_pages[pnum]);
 		vec[pnum].iov_len = PAGE_SIZE;
 		pnum++;
 		vlen += PAGE_SIZE;
@@ -989,6 +1080,16 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	if (len < 0)
 		goto err_again;
 
+	/*
+	 * Account for the 8 bytes we read earlier
+	 */
+	len += 8;
+
+	if (req) {
+		xprt_complete_rqst(req->rq_task, len);
+		len = 0;
+		goto out;
+	}
 	dprintk("svc: TCP complete record (%d bytes)\n", len);
 	rqstp->rq_arg.len = len;
 	rqstp->rq_arg.page_base = 0;
@@ -1002,6 +1103,7 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	rqstp->rq_xprt_ctxt   = NULL;
 	rqstp->rq_prot	      = IPPROTO_TCP;
 
+out:
 	/* Reset TCP read info */
 	svsk->sk_reclen = 0;
 	svsk->sk_tcplen = 0;
net/sunrpc/xprt.c +0 -0
index f412a85..7b0cf70 100644

net/sunrpc/xprt.c static void xprt_autoclose(struct work_struct *work) · line 603

Quoted 5 lines
net/sunrpc/xprt.c +2 -0
@@ -599,6 +599,9 @@ static void xprt_autoclose(struct work_struct *work)
 	struct rpc_xprt *xprt =
 		container_of(work, struct rpc_xprt, task_cleanup);
 
+	if (xprt_server_backchannel(xprt))
+		return;

Why do you need this? For one thing, it means that XPRT_CLOSE_WAIT never gets cleared...

net/sunrpc/xprt.c xprt_init_autodisconnect(unsigned long data) · line 676

Quoted 10 lines
net/sunrpc/xprt.c +3 -0
+
 	xprt->ops->close(xprt);
 	clear_bit(XPRT_CLOSE_WAIT, &xprt->state);
 	xprt_release_write(xprt, NULL);
@@ -669,6 +672,9 @@ xprt_init_autodisconnect(unsigned long data)
 {
 	struct rpc_xprt *xprt = (struct rpc_xprt *)data;
 
+	if (xprt_server_backchannel(xprt))
+		return;

Hmm... Do you need this? Why would you want to set up an autodisconnect timer in the first place?

Quoted 5 lines
net/sunrpc/xprt.c +1 -0
+
 	spin_lock(&xprt->transport_lock);
 	if (!list_empty(&xprt->recv) || xprt->shutdown)
 		goto out_abort;

net/sunrpc/xprt.c found: · line 1118

Quoted 10 lines
net/sunrpc/xprt.c +7 -0
@@ -1103,6 +1109,13 @@ found:
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
 
+	/*
+	 * Since we don't want connections for the backchannel, we set
+	 * the xprt status to connected
+	 */
+	if (args->bc_xprt)
+		xprt_set_connected(xprt);
+

Why don't you do this in the ->setup() callback?

Quoted 183 lines
net/sunrpc/xprt.c +0 -0
 	return xprt;
 }
 
net/sunrpc/xprtsock.c +146 -0
index 62438f3..592681c 100644
@@ -32,6 +32,7 @@
 #include <linux/tcp.h>
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/sched.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/xprtsock.h>
 #include <linux/file.h>
 #ifdef CONFIG_NFS_V4_1
@@ -43,6 +44,7 @@
 #include <net/udp.h>
 #include <net/tcp.h>
 
+#include "sunrpc.h"
 /*
  * xprtsock tunables
  */
@@ -2098,6 +2100,134 @@ static void xs_tcp_print_stats(struct rpc_xprt *xprt, struct seq_file *seq)
 			xprt->stat.bklog_u);
 }
 
+/*
+ * Allocate a bunch of pages for a scratch buffer for the rpc code. The reason
+ * we allocate pages instead doing a kmalloc like rpc_malloc is because we want
+ * to use the server side send routines.
+ */
+void *bc_malloc(struct rpc_task *task, size_t size)
+{
+	struct page *page;
+	struct rpc_buffer *buf;
+
+	BUG_ON(size > PAGE_SIZE - sizeof(struct rpc_buffer));
+	page = alloc_page(GFP_KERNEL);
+
+	if (!page)
+		return NULL;
+
+	buf = page_address(page);
+	buf->len = PAGE_SIZE;
+
+	return buf->data;
+}
+
+/*
+ * Free the space allocated in the bc_alloc routine
+ */
+void bc_free(void *buffer)
+{
+	struct rpc_buffer *buf;
+
+	if (!buffer)
+		return;
+
+	buf = container_of(buffer, struct rpc_buffer, data);
+	free_page((unsigned long)buf);
+}
+
+/*
+ * Use the svc_sock to send the callback. Must be called with svsk->sk_mutex
+ * held. Borrows heavily from svc_tcp_sendto and xs_tcp_send_request.
+ */
+static int bc_sendto(struct rpc_rqst *req)
+{
+	int len;
+	struct xdr_buf *xbufp = &req->rq_snd_buf;
+	struct rpc_xprt *xprt = req->rq_xprt;
+	struct sock_xprt *transport =
+				container_of(xprt, struct sock_xprt, xprt);
+	struct socket *sock = transport->sock;
+	unsigned long headoff;
+	unsigned long tailoff;
+
+	/*
+	 * Set up the rpc header and record marker stuff
+	 */
+	xs_encode_tcp_record_marker(xbufp);
+
+	tailoff = (unsigned long)xbufp->tail[0].iov_base & ~PAGE_MASK;
+	headoff = (unsigned long)xbufp->head[0].iov_base & ~PAGE_MASK;
+	len = svc_send_common(sock, xbufp,
+			      virt_to_page(xbufp->head[0].iov_base), headoff,
+			      xbufp->tail[0].iov_base, tailoff);
+
+	if (len != xbufp->len) {
+		printk(KERN_NOTICE "Error sending entire callback!\n");
+		len = -EAGAIN;
+	}
+
+	return len;
+}
+
+/*
+ * The send routine. Borrows from svc_send
+ */
+static int bc_send_request(struct rpc_task *task)
+{
+	struct rpc_rqst *req = task->tk_rqstp;
+	struct svc_xprt	*xprt;
+	struct svc_sock         *svsk;
+	u32                     len;
+
+	dprintk("sending request with xid: %08x\n", ntohl(req->rq_xid));
+	/*
+	 * Get the server socket associated with this callback xprt
+	 */
+	xprt = req->rq_xprt->bc_xprt;
+	svsk = container_of(xprt, struct svc_sock, sk_xprt);
+
+	/*
+	 * Grab the mutex to serialize data as the connection is shared
+	 * with the fore channel
+	 */
+	if (!mutex_trylock(&xprt->xpt_mutex)) {
+		rpc_sleep_on(&xprt->xpt_bc_pending, task, NULL);
+		if (!mutex_trylock(&xprt->xpt_mutex))
+			return -EAGAIN;
+		rpc_wake_up_queued_task(&xprt->xpt_bc_pending, task);
+	}
+	if (test_bit(XPT_DEAD, &xprt->xpt_flags))
+		len = -ENOTCONN;
+	else
+		len = bc_sendto(req);
+	mutex_unlock(&xprt->xpt_mutex);
+
+	if (len > 0)
+		len = 0;
+
+	return len;
+}
+
+/*
+ * The close routine. Since this is client initiated, we do nothing
+ */
+
+static void bc_close(struct rpc_xprt *xprt)
+{
+	return;
+}
+
+/*
+ * The xprt destroy routine. Again, because this connection is client
+ * initiated, we do nothing
+ */
+
+static void bc_destroy(struct rpc_xprt *xprt)
+{
+	return;
+}
+
 static struct rpc_xprt_ops xs_udp_ops = {
 	.set_buffer_size	= xs_udp_set_buffer_size,
 	.reserve_xprt		= xprt_reserve_xprt_cong,
@@ -2134,6 +2264,22 @@ static struct rpc_xprt_ops xs_tcp_ops = {
 	.print_stats		= xs_tcp_print_stats,
 };
 
+/*
+ * The rpc_xprt_ops for the server backchannel
+ */
+
+static struct rpc_xprt_ops bc_tcp_ops = {
+	.reserve_xprt		= xprt_reserve_xprt,
+	.release_xprt		= xprt_release_xprt,
+	.buf_alloc		= bc_malloc,
+	.buf_free		= bc_free,
+	.send_request		= bc_send_request,
+	.set_retrans_timeout	= xprt_set_retrans_timeout_def,
+	.close			= bc_close,
+	.destroy		= bc_destroy,
+	.print_stats		= xs_tcp_print_stats,
+};
+
 static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
 				      unsigned int slot_table_size)
 {
@@ -2272,14 +2418,46 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 	xprt->prot = IPPROTO_TCP;

net/sunrpc/xprtsock.c static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args) · line 2424

Quoted 10 lines
net/sunrpc/xprtsock.c +3 -4
 	xprt->tsh_size = sizeof(rpc_fraghdr) / sizeof(u32);
 	xprt->max_payload = RPC_MAX_FRAGMENT_SIZE;
+	xprt->timeout = &xs_tcp_default_timeout;
 
-	xprt->bind_timeout = XS_BIND_TO;
-	xprt->connect_timeout = XS_TCP_CONN_TO;
-	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
-	xprt->idle_timeout = XS_IDLE_DISC_TO;
+	if (args->bc_xprt) {
+		struct svc_sock *bc_sock;

Why are you embedding this inside the forward channel setup code? Just make a TCP backchannel class with its own ->setup().

Quoted 55 lines
net/sunrpc/xprtsock.c +43 -2
-	xprt->ops = &xs_tcp_ops;
-	xprt->timeout = &xs_tcp_default_timeout;
+		/* backchannel */
+		xprt_set_bound(xprt);
+		xprt->bind_timeout = 0;
+		xprt->connect_timeout = 0;
+		xprt->reestablish_timeout = 0;
+		xprt->idle_timeout = (~0);
+
+		/*
+		 * The backchannel uses the same socket connection as the
+		 * forechannel
+		 */
+		xprt->bc_xprt = args->bc_xprt;
+		bc_sock = container_of(args->bc_xprt, struct svc_sock, sk_xprt);
+		bc_sock->sk_bc_xprt = xprt;
+		transport->sock = bc_sock->sk_sock;
+		transport->inet = bc_sock->sk_sk;
+
+		xprt->ops = &bc_tcp_ops;
+
+		switch (addr->sa_family) {
+		case AF_INET:
+			xs_format_peer_addresses(xprt, "tcp",
+						 RPCBIND_NETID_TCP);
+			break;
+		case AF_INET6:
+			xs_format_peer_addresses(xprt, "tcp",
+						 RPCBIND_NETID_TCP6);
+			break;
+		default:
+			kfree(xprt);
+			return ERR_PTR(-EAFNOSUPPORT);
+		}
+
+		goto out;
+	}
 
 	switch (addr->sa_family) {
 	case AF_INET:
@@ -2303,6 +2481,14 @@ static struct rpc_xprt *xs_setup_tcp(struct xprt_create *args)
 		return ERR_PTR(-EAFNOSUPPORT);
 	}
 
+	xprt->bind_timeout = XS_BIND_TO;
+	xprt->connect_timeout = XS_TCP_CONN_TO;
+	xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
+	xprt->idle_timeout = XS_IDLE_DISC_TO;
+
+	xprt->ops = &xs_tcp_ops;
+
+out:
 	if (xprt_bound(xprt))
 		dprintk("RPC:       set up xprt to %s (port %s) via %s\n",
 				xprt->address_strings[RPC_DISPLAY_ADDR],
Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#12 lore raw

v2 cover nfsd41 backchannel patches for 2.6.32

In reply to Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

Benny

Quoted 31 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#13 lore raw

PATCH v2 1/12 nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h

Move struct rpc_buffer's definition into a sunrpc.h, a common, internal header file, in preparation for supporting the nfsv4.1 backchannel.

[nfs41: sunrpc: #include <linux/net.h> from sunrpc.h]
---
 net/sunrpc/sched.c  |    7 ++-----
 net/sunrpc/sunrpc.h |   10 ++++++++++
 2 files changed, 12 insertions(+), 5 deletions(-)
net/sunrpc/sched.c +2 -5
index 8f459ab..cef74ba 100644
@@ -21,6 +21,8 @@
 
 #include <linux/sunrpc/clnt.h>
 
+#include "sunrpc.h"
+
 #ifdef RPC_DEBUG
 #define RPCDBG_FACILITY		RPCDBG_SCHED
 #define RPC_TASK_MAGIC_ID	0xf00baa
@@ -711,11 +713,6 @@ static void rpc_async_schedule(struct work_struct *work)
 	__rpc_execute(container_of(work, struct rpc_task, u.tk_work));
 }
 
-struct rpc_buffer {
-	size_t	len;
-	char	data[];
-};
-
 /**
  * rpc_malloc - allocate an RPC buffer
  * @task: RPC task that will use this buffer
net/sunrpc/sunrpc.h +10 -0
index 5d9dd74..13171e6 100644
@@ -27,6 +27,16 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 #ifndef _NET_SUNRPC_SUNRPC_H
 #define _NET_SUNRPC_SUNRPC_H
 
+#include <linux/net.h>
+
+/*
+ * Header for dynamically allocated rpc buffers.
+ */
+struct rpc_buffer {
+	size_t	len;
+	char	data[];
+};
+
 static inline int rpc_reply_expected(struct rpc_task *task)
 {
 	return (task->tk_msg.rpc_proc != NULL) &&
Signature · 1 line
1.6.4
#15 lore raw

PATCH v2 2/12 nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to Benny Halevy

From: Rahul Iyer <[email protected]>

When the call direction is a reply, copy the xid and call direction into the req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns rpc_garbage.

[get rid of CONFIG_NFSD_V4_1]
[sunrpc: refactoring of svc_tcp_recvfrom]
[nfsd41: sunrpc: create common send routine for the fore and the back channels]
[nfsd41: sunrpc: Use free_page() to free server backchannel pages]
[nfsd41: sunrpc: Document server backchannel locking]
[nfsd41: sunrpc: remove bc_connect_worker()]

[nfsd41: sunrpc: Define xprt_server_backchannel()[

[nfsd41: sunrpc: remove bc_close and bc_init_auto_disconnect dummy functions]
[nfsd41: sunrpc: eliminate unneeded switch statement in xs_setup_tcp()]
[nfsd41: sunrpc: Don't auto close the server backchannel connection]
[nfsd41: sunrpc: Remove unused functions]
[nfsd41: change bc_sock to bc_xprt]
[nfsd41: sunrpc: move struct rpc_buffer def into a common header file]
[nfsd41: sunrpc: use rpc_sleep in bc_send_request so not to block on mutex]
[removed cosmetic changes]
[sunrpc: add new xprt class for nfsv4.1 backchannel]
[sunrpc: v2.1 change handling of auto_close and init_auto_disconnect operations for the nfsv4.1 backchannel]
[reverted more cosmetic leftovers]
[got rid of xprt_server_backchannel]
[separated "nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel"]
Cc Trond Myklebust <[email protected]>
---
 include/linux/sunrpc/svc_xprt.h |    1 +
 include/linux/sunrpc/svcsock.h  |    1 +
 include/linux/sunrpc/xprt.h     |    1 +
 net/sunrpc/sunrpc.h             |    4 +
 net/sunrpc/svc_xprt.c           |    2 +
 net/sunrpc/svcsock.c            |  172 +++++++++++++++++++++++++++++++--------
 net/sunrpc/xprt.c               |   15 +++-
 net/sunrpc/xprtsock.c           |  146 +++++++++++++++++++++++++++++++++
 8 files changed, 303 insertions(+), 39 deletions(-)
include/linux/sunrpc/svc_xprt.h +1 -0
index 2223ae0..5f4e18b 100644
@@ -65,6 +65,7 @@ struct svc_xprt {
 	size_t			xpt_locallen;	/* length of address */
 	struct sockaddr_storage	xpt_remote;	/* remote peer's address */
 	size_t			xpt_remotelen;	/* length of address */
+	struct rpc_wait_queue	xpt_bc_pending;	/* backchannel wait queue */
 };
 
 int	svc_reg_xprt_class(struct svc_xprt_class *);
include/linux/sunrpc/svcsock.h +1 -0
index 04dba23..1b353a7 100644
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt		*sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
include/linux/sunrpc/xprt.h +1 -0
index c090df4..228d694 100644
@@ -179,6 +179,7 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 #if defined(CONFIG_NFS_V4_1)
 	struct svc_serv		*bc_serv;       /* The RPC service which will */
 						/* process the callback */
net/sunrpc/sunrpc.h +4 -0
index 13171e6..90c292e 100644
@@ -43,5 +43,9 @@ static inline int rpc_reply_expected(struct rpc_task *task)
 		(task->tk_msg.rpc_proc->p_decode != NULL);
 }
 
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset);
+
 #endif /* _NET_SUNRPC_SUNRPC_H */
 
net/sunrpc/svc_xprt.c +2 -0
index 912dea5..df124f7 100644
@@ -160,6 +160,7 @@ void svc_xprt_init(struct svc_xprt_class *xcl, struct svc_xprt *xprt,
 	mutex_init(&xprt->xpt_mutex);
 	spin_lock_init(&xprt->xpt_lock);
 	set_bit(XPT_BUSY, &xprt->xpt_flags);
+	rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending");
 }
 EXPORT_SYMBOL_GPL(svc_xprt_init);
 
@@ -810,6 +811,7 @@ int svc_send(struct svc_rqst *rqstp)
 	else
 		len = xprt->xpt_ops->xpo_sendto(rqstp);
 	mutex_unlock(&xprt->xpt_mutex);
+	rpc_wake_up(&xprt->xpt_bc_pending);
 	svc_xprt_release(rqstp);
 
 	if (len == -ECONNREFUSED || len == -ENOTCONN || len == -EAGAIN)
net/sunrpc/svcsock.c +137 -35

241 lines not on this page · raw message

net/sunrpc/xprt.c +11 -4
index f412a85..f577e5a 100644
@@ -832,6 +832,11 @@ static void xprt_timer(struct rpc_task *task)
 	spin_unlock_bh(&xprt->transport_lock);
 }
 
+static inline int xprt_has_timer(struct rpc_xprt *xprt)
+{
+	return xprt->idle_timeout != (~0);
+}
+
 /**
  * xprt_prepare_transmit - reserve the transport before sending a request
  * @task: RPC task about to send a request
@@ -1013,7 +1018,7 @@ void xprt_release(struct rpc_task *task)
 	if (!list_empty(&req->rq_list))
 		list_del(&req->rq_list);
 	xprt->last_used = jiffies;
-	if (list_empty(&xprt->recv))
+	if (list_empty(&xprt->recv) && xprt_has_timer(xprt))
 		mod_timer(&xprt->timer,
 				xprt->last_used + xprt->idle_timeout);
 	spin_unlock_bh(&xprt->transport_lock);
@@ -1082,8 +1087,11 @@ found:
 #endif /* CONFIG_NFS_V4_1 */
 
 	INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
-	setup_timer(&xprt->timer, xprt_init_autodisconnect,
-			(unsigned long)xprt);
+	if (xprt_has_timer(xprt))
+		setup_timer(&xprt->timer, xprt_init_autodisconnect,
+			    (unsigned long)xprt);
+	else
+		init_timer(&xprt->timer);
 	xprt->last_used = jiffies;
 	xprt->cwnd = RPC_INITCWND;
 	xprt->bind_index = 0;
@@ -1102,7 +1110,6 @@ found:
 
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
-
 	return xprt;
 }
 
net/sunrpc/xprtsock.c +146 -0

170 lines not on this page · raw message

Signature · 1 line
1.6.4
#24 lore raw

PATCH v2 3/12 nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

In reply to Benny Halevy

From: Alexandros Batsakis <[email protected]>

---
 include/linux/sunrpc/clnt.h     |    1 +
 include/linux/sunrpc/xprt.h     |    1 +
 include/linux/sunrpc/xprtrdma.h |    5 --
 include/linux/sunrpc/xprtsock.h |    9 +++-
 net/sunrpc/clnt.c               |    1 +
 net/sunrpc/xprtsock.c           |   96 ++++++++++++++++++++++++++++++++++++++-
 6 files changed, 104 insertions(+), 9 deletions(-)
include/linux/sunrpc/clnt.h +1 -0
index 3d02558..8ed9642 100644
@@ -114,6 +114,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
include/linux/sunrpc/xprt.h +1 -0
index 228d694..7cc42af 100644
@@ -232,6 +232,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {
include/linux/sunrpc/xprtrdma.h +0 -5
index 54a379c..c2f04e1 100644
@@ -41,11 +41,6 @@
 #define _LINUX_SUNRPC_XPRTRDMA_H
 
 /*
- * RPC transport identifier for RDMA
- */
-#define XPRT_TRANSPORT_RDMA	256
-
-/*
  * rpcbind (v3+) RDMA netid.
  */
 #define RPCBIND_NETID_RDMA	"rdma"
include/linux/sunrpc/xprtsock.h +7 -2
index c2a46c4..d7c98d1 100644
@@ -20,8 +20,13 @@ void		cleanup_socket_xprt(void);
  * values. No such restriction exists for new transports, except that
  * they may not collide with these values (17 and 6, respectively).
  */
-#define XPRT_TRANSPORT_UDP	IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP	IPPROTO_TCP
+#define XPRT_TRANSPORT_BC	(1 << 31)
+enum xprt_transports {
+	XPRT_TRANSPORT_UDP 	= IPPROTO_UDP,
+	XPRT_TRANSPORT_TCP 	= IPPROTO_TCP,
+	XPRT_TRANSPORT_BC_TCP 	= IPPROTO_TCP | XPRT_TRANSPORT_BC,
+	XPRT_TRANSPORT_RDMA	= 256
+};
 
 /*
  * RPC slot table sizes for UDP, TCP transports
net/sunrpc/clnt.c +1 -0
index c1e467e..7389804 100644
@@ -288,6 +288,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_xprt = args->bc_xprt,
 	};
 	char servername[48];
 
net/sunrpc/xprtsock.c +94 -2

127 lines not on this page · raw message

Signature · 1 line
1.6.4
#31 lore raw

PATCH v2 4/12 nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition

In reply to Benny Halevy
---
 fs/nfsd/nfs4callback.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
fs/nfsd/nfs4callback.c +1 -1
index 81d1c52..63bb384 100644
@@ -56,7 +56,7 @@
 /* Index of predefined Linux callback client operations */
 
 enum {
-        NFSPROC4_CLNT_CB_NULL = 0,
+	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
 };
 
Signature · 1 line
1.6.4
#32 lore raw

PATCH v2 5/12 nfsd41: Backchannel: callback infrastructure

In reply to Benny Halevy

From: Andy Adamson <[email protected]>

Keep the xprt used for create_session in cl_cb_xprt. Mark cl_callback.cb_minorversion = 1 and remember the client provided cl_callback.cb_prog rpc program number. Use it to probe the callback path.

Use the client's network address to initialize as the callback's address as expected by the xprt creation routines.

Define xdr sizes and code nfs4_cb_compound header to be able to send a null callback rpc.

[get callback minorversion from fore channel's]
[nfsd41: change bc_sock to bc_xprt]
[pulled definition for cl_cb_xprt]
[nfsd41: set up backchannel's cb_addr]
[moved rpc_create_args init to "nfsd: modify nfsd4.1 backchannel to use new xprt class"]
---
 fs/nfsd/nfs4callback.c     |   21 +++++++++++++++++++--
 fs/nfsd/nfs4state.c        |   14 ++++++++++++++
 include/linux/nfsd/state.h |    3 +++
 3 files changed, 36 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +19 -2
index 63bb384..3e3e15b 100644
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -137,11 +148,13 @@ xdr_error:                                      \
 } while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -399,6 +412,10 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
+
 	/* Create RPC client */
 	client = rpc_create(&args);
 	if (IS_ERR(client)) {
fs/nfsd/nfs4state.c +14 -0
index 46e9ac5..e4c3223 100644
@@ -706,6 +706,8 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1321,6 +1323,18 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			rpc_copy_addr(
+				(struct sockaddr *)&unconf->cl_cb_conn.cb_addr,
+				sa);
+			unconf->cl_cb_conn.cb_addrlen = svc_addr_len(sa);
+			unconf->cl_cb_conn.cb_minorversion =
+				cstate->minorversion;
+			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
include/linux/nfsd/state.h +3 -0
index 70ef5f4..243277b 100644
@@ -212,6 +212,9 @@ struct nfs4_client {
 	struct nfsd4_clid_slot	cl_cs_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
+
+	/* for nfs41 callbacks */
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#55 lore raw

PATCH v2 6/12 nfsd41: Backchannel: Add sequence arguments to callback RPC arguments

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Follow the model we use in the client. Make the sequence arguments part of the regular RPC arguments. None of the callbacks that are soon to be implemented expect results that need to be passed back to the caller, so we don't define a separate RPC results structure. For session validation, the cb_sequence decoding will use a pointer to the sequence arguments that are part of the RPC argument.

[define struct nfsd4_cb_sequence here]
---
 fs/nfsd/nfs4callback.c     |    5 +++++
 include/linux/nfsd/state.h |    6 ++++++
 2 files changed, 11 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +5 -0
index 3e3e15b..ff59a61 100644
@@ -92,6 +92,11 @@ enum nfs_cb_opnum4 {
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
+struct nfs4_rpc_args {
+	void				*args_op;
+	struct nfsd4_cb_sequence	args_seq;
+};
+
 /*
 * Generic encode routines from fs/nfs/nfs4xdr.c
 */
include/linux/nfsd/state.h +6 -0
index 243277b..f69ea48 100644
@@ -60,6 +60,12 @@ typedef struct {
 #define si_stateownerid   si_opaque.so_stateownerid
 #define si_fileid         si_opaque.so_fileid
 
+struct nfsd4_cb_sequence {
+	/* args/res */
+	u32			cbs_minorversion;
+	struct nfs4_client	*cbs_clp;
+};
+
 struct nfs4_delegation {
 	struct list_head	dl_perfile;
 	struct list_head	dl_perclnt;
Signature · 1 line
1.6.4
#56 lore raw

PATCH v2 7/12 nfsd41: Backchannel: Server backchannel RPC wait queue

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

RPC callback requests will wait on this wait queue if the backchannel is out of slots.

---
 fs/nfsd/nfs4state.c        |    2 ++
 include/linux/nfsd/state.h |    4 ++++
 2 files changed, 6 insertions(+), 0 deletions(-)
fs/nfsd/nfs4state.c +2 -0
index e4c3223..957f6e5 100644
@@ -779,6 +779,8 @@ static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
 	INIT_LIST_HEAD(&clp->cl_delegations);
 	INIT_LIST_HEAD(&clp->cl_sessions);
 	INIT_LIST_HEAD(&clp->cl_lru);
+	clear_bit(0, &clp->cl_cb_slot_busy);
+	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
 	return clp;
 }
 
include/linux/nfsd/state.h +4 -0
index f69ea48..234e9af 100644
@@ -220,7 +220,11 @@ struct nfs4_client {
 	struct nfs4_sessionid	cl_sessionid;
 
 	/* for nfs41 callbacks */
+	/* We currently support a single back channel with a single slot */
+	unsigned long		cl_cb_slot_busy;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
+	struct rpc_wait_queue	cl_cb_waitq;	/* backchannel callers may */
+						/* wait here for slots */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#57 lore raw

PATCH v2 8/12 nfsd41: Backchannel: Setup sequence information

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Follows the model used by the NFS client. Setup the RPC prepare and done function pointers so that we can populate the sequence information if minorversion == 1. rpc_run_task() is then invoked directly just like existing NFS client operations do.

nfsd4_cb_prepare() determines if the sequence information needs to be setup. If the slot is in use, it adds itself to the wait queue.

nfsd4_cb_done() wakes anyone sleeping on the callback channel wait queue after our RPC reply has been received. It also sets the task message result pointer to NULL to clearly indicate we're done using it.

[define and initialize cl_cb_seq_nr here]
[pulled out unused defintion of nfsd4_cb_done]
---
 fs/nfsd/nfs4callback.c     |   62 ++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4state.c        |    1 +
 include/linux/nfsd/state.h |    1 +
 3 files changed, 64 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +62 -0
index ff59a61..e79e3a4 100644
@@ -518,6 +518,67 @@ nfsd4_probe_callback(struct nfs4_client *clp)
 	do_probe_callback(clp);
 }
 
+/*
+ * There's currently a single callback channel slot.
+ * If the slot is available, then mark it busy.  Otherwise, set the
+ * thread for sleeping on the callback RPC wait queue.
+ */
+static int nfsd41_cb_setup_sequence(struct nfs4_client *clp,
+		struct rpc_task *task)
+{
+	struct nfs4_rpc_args *args = task->tk_msg.rpc_argp;
+	u32 *ptr = (u32 *)clp->cl_sessionid.data;
+	int status = 0;
+
+	dprintk("%s: %u:%u:%u:%u\n", __func__,
+		ptr[0], ptr[1], ptr[2], ptr[3]);
+
+	if (test_and_set_bit(0, &clp->cl_cb_slot_busy) != 0) {
+		rpc_sleep_on(&clp->cl_cb_waitq, task, NULL);
+		dprintk("%s slot is busy\n", __func__);
+		status = -EAGAIN;
+		goto out;
+	}
+
+	/*
+	 * We'll need the clp during XDR encoding and decoding,
+	 * and the sequence during decoding to verify the reply
+	 */
+	args->args_seq.cbs_clp = clp;
+	task->tk_msg.rpc_resp = &args->args_seq;
+
+out:
+	dprintk("%s status=%d\n", __func__, status);
+	return status;
+}
+
+/*
+ * TODO: cb_sequence should support referring call lists, cachethis, multiple
+ * slots, and mark callback channel down on communication errors.
+ */
+static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+	struct nfs4_rpc_args *args = task->tk_msg.rpc_argp;
+	u32 minorversion = clp->cl_cb_conn.cb_minorversion;
+	int status = 0;
+
+	args->args_seq.cbs_minorversion = minorversion;
+	if (minorversion) {
+		status = nfsd41_cb_setup_sequence(clp, task);
+		if (status) {
+			if (status != -EAGAIN) {
+				/* terminate rpc task */
+				task->tk_status = status;
+				task->tk_action = NULL;
+			}
+			return;
+		}
+	}
+	rpc_call_start(task);
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
@@ -557,6 +618,7 @@ static void nfsd4_cb_recall_release(void *calldata)
 }
 
 static const struct rpc_call_ops nfsd4_cb_recall_ops = {
+	.rpc_call_prepare = nfsd4_cb_prepare,
 	.rpc_call_done = nfsd4_cb_recall_done,
 	.rpc_release = nfsd4_cb_recall_release,
 };
fs/nfsd/nfs4state.c +1 -0
index 957f6e5..b2ffa3b 100644
@@ -1335,6 +1335,7 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 			unconf->cl_cb_conn.cb_minorversion =
 				cstate->minorversion;
 			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			unconf->cl_cb_seq_nr = 1;
 			nfsd4_probe_callback(unconf);
 		}
 		conf = unconf;
include/linux/nfsd/state.h +1 -0
index 234e9af..b621428 100644
@@ -222,6 +222,7 @@ struct nfs4_client {
 	/* for nfs41 callbacks */
 	/* We currently support a single back channel with a single slot */
 	unsigned long		cl_cb_slot_busy;
+	u32			cl_cb_seq_nr;
 	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 	struct rpc_wait_queue	cl_cb_waitq;	/* backchannel callers may */
 						/* wait here for slots */
Signature · 1 line
1.6.4
#58 lore raw

PATCH v2 9/12 nfsd41: Backchannel: cb_sequence callback

In reply to Benny Halevy

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Note: highest slot id and target highest slot id do not have to be 0 as was previously implemented. They can be greater than what the nfs server sent if the client supports a larger slot table on the backchannel. At this point we just ignore that.

[Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
[fixed indentation]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]

[nfsd41: fix verification of CB_SEQUENCE highest slot id[

[nfsd41: Backchannel: Remove old backchannel serialization]
[nfsd41: Backchannel: First callback sequence ID should be 1]
[nfsd41: decode_cb_sequence does not need to actually decode ignored fields]
---
 fs/nfsd/nfs4callback.c |   72 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 72 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +72 -0
index e79e3a4..5e9659c 100644
@@ -256,6 +256,27 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_delegation *dp,
 	hdr->nops++;
 }
 
+static void
+encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
+		   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	if (hdr->minorversion == 0)
+		return;
+
+	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
+
+	WRITE32(OP_CB_SEQUENCE);
+	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(args->cbs_clp->cl_cb_seq_nr);
+	WRITE32(0);		/* slotid, always 0 */
+	WRITE32(0);		/* highest slotid always 0 */
+	WRITE32(0);		/* cachethis always 0 */
+	WRITE32(0); /* FIXME: support referring_call_lists */
+	hdr->nops++;
+}
+
 static int
 nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -317,6 +338,57 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
 	return 0;
 }
 
+/*
+ * Our current back channel implmentation supports a single backchannel
+ * with a single slot.
+ */
+static int
+decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
+		   struct rpc_rqst *rqstp)
+{
+	struct nfs4_sessionid id;
+	int status;
+	u32 dummy;
+	__be32 *p;
+
+	if (res->cbs_minorversion == 0)
+		return 0;
+
+	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
+	if (status)
+		return status;
+
+	/*
+	 * If the server returns different values for sessionID, slotID or
+	 * sequence number, the server is looney tunes.
+	 */
+	status = -ESERVERFAULT;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	memcpy(id.data, p, NFS4_MAX_SESSIONID_LEN);
+	p += XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN);
+	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
+		   NFS4_MAX_SESSIONID_LEN)) {
+		dprintk("%s Invalid session id\n", __func__);
+		goto out;
+	}
+	READ32(dummy);
+	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
+		dprintk("%s Invalid sequence number\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid slotid\n", __func__);
+		goto out;
+	}
+	/* FIXME: process highest slotid and target highest slotid */
+	status = 0;
+out:
+	return status;
+}
+
+
 static int
 nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 {
Signature · 1 line
1.6.4
#61 lore raw

PATCH v2 10/12 nfsd41: Backchannel: Implement cb_recall over NFSv4.1

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

[nfsd41: cb_recall callback]
[Share v4.0 and v4.1 back channel xdr]
[Share v4.0 and v4.1 back channel xdr]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]
[nfsd41: conditionally decode_sequence in nfs4_xdr_dec_cb_recall]
[nfsd41: Backchannel: Add sequence arguments to callback RPC arguments]
[pulled-in definition of nfsd4_cb_done]
---
 fs/nfsd/nfs4callback.c |   52 ++++++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 48 insertions(+), 4 deletions(-)
fs/nfsd/nfs4callback.c +48 -4
index 5e9659c..30dc375 100644
@@ -288,15 +288,19 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_delegation *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_delegation *args = rpc_args->args_op;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->dl_ident,
+		.minorversion = rpc_args->args_seq.cbs_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, &rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -396,7 +400,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		struct nfsd4_cb_sequence *seq)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -406,6 +411,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	if (seq) {
+		status = decode_cb_sequence(&xdr, seq, rqstp);
+		if (status)
+			goto out;
+	}
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -651,11 +661,34 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
 	rpc_call_start(task);
 }
 
+static void nfsd4_cb_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+
+	dprintk("%s: minorversion=%d\n", __func__,
+		clp->cl_cb_conn.cb_minorversion);
+
+	if (clp->cl_cb_conn.cb_minorversion) {
+		/* No need for lock, access serialized in nfsd4_cb_prepare */
+		++clp->cl_cb_seq_nr;
+		clear_bit(0, &clp->cl_cb_slot_busy);
+		rpc_wake_up_next(&clp->cl_cb_waitq);
+		dprintk("%s: freed slot, new seqid=%d\n", __func__,
+			clp->cl_cb_seq_nr);
+
+		/* We're done looking into the sequence information */
+		task->tk_msg.rpc_resp = NULL;
+	}
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
 	struct nfs4_client *clp = dp->dl_client;
 
+	nfsd4_cb_done(task, calldata);
+
 	switch (task->tk_status) {
 	case -EIO:
 		/* Network partition? */
@@ -668,16 +701,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -703,16 +739,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
 	int status;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args) {
+		status = -ENOMEM;
+		goto out;
+	}
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
 		put_nfs4_client(clp);
 		nfs4_put_delegation(dp);
Signature · 1 line
1.6.4
#64 lore raw

PATCH v2 11/12 nfsd41: modify nfsd4.1 backchannel to use new xprt class

In reply to Benny Halevy

From: Alexandros Batsakis <[email protected]>

This patch enables the use of the nfsv4.1 backchannel.

[initialize rpc_create_args.bc_xprt too]
---
 fs/nfsd/nfs4callback.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
fs/nfsd/nfs4callback.c +7 -1
index 30dc375..236e7ee 100644
@@ -48,6 +48,7 @@
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
 #include <linux/nfs4.h>
+#include <linux/sunrpc/xprtsock.h>
 
 #define NFSDDBG_FACILITY                NFSDDBG_PROC
 
@@ -483,7 +484,7 @@ int setup_callback_client(struct nfs4_client *clp)
 		.to_retries	= 0,
 	};
 	struct rpc_create_args args = {
-		.protocol	= IPPROTO_TCP,
+		.protocol	= XPRT_TRANSPORT_TCP,
 		.address	= (struct sockaddr *) &cb->cb_addr,
 		.addrsize	= cb->cb_addrlen,
 		.timeout	= &timeparms,
@@ -499,6 +500,11 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	if (cb->cb_minorversion) {
+		args.bc_xprt = clp->cl_cb_xprt;
+		args.protocol = XPRT_TRANSPORT_BC_TCP;
+	}
+
 	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
 		__func__, args.program->name, args.prognumber,
 		args.program->nrvers, args.version, cb->cb_minorversion);
Signature · 1 line
1.6.4
#65 lore raw

PATCH v2 12/12 nfsd41: Refactor create_client()

In reply to Benny Halevy

From: Ricardo Labiaga <[email protected]>

Move common initialization of 'struct nfs4_client' inside create_client().

[nfsd41: Remember the auth flavor to use for callbacks]
---
 fs/nfsd/nfs4state.c |   89 ++++++++++++++++++++++++++-------------------------
 1 files changed, 45 insertions(+), 44 deletions(-)
fs/nfsd/nfs4state.c +45 -44
index b2ffa3b..f344de2 100644
@@ -763,27 +763,6 @@ expire_client(struct nfs4_client *clp)
 	put_nfs4_client(clp);
 }
 
-static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir)
-{
-	struct nfs4_client *clp;
-
-	clp = alloc_client(name);
-	if (clp == NULL)
-		return NULL;
-	memcpy(clp->cl_recdir, recdir, HEXDIR_LEN);
-	atomic_set(&clp->cl_count, 1);
-	atomic_set(&clp->cl_cb_conn.cb_set, 0);
-	INIT_LIST_HEAD(&clp->cl_idhash);
-	INIT_LIST_HEAD(&clp->cl_strhash);
-	INIT_LIST_HEAD(&clp->cl_openowners);
-	INIT_LIST_HEAD(&clp->cl_delegations);
-	INIT_LIST_HEAD(&clp->cl_sessions);
-	INIT_LIST_HEAD(&clp->cl_lru);
-	clear_bit(0, &clp->cl_cb_slot_busy);
-	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
-	return clp;
-}
-
 static void copy_verf(struct nfs4_client *target, nfs4_verifier *source)
 {
 	memcpy(target->cl_verifier.data, source->data,
@@ -846,6 +825,46 @@ static void gen_confirm(struct nfs4_client *clp)
 	*p++ = i++;
 }
 
+static struct nfs4_client *create_client(struct xdr_netobj name, char *recdir,
+		struct svc_rqst *rqstp, nfs4_verifier *verf)
+{
+	struct nfs4_client *clp;
+	struct sockaddr *sa = svc_addr(rqstp);
+	char *princ;
+
+	clp = alloc_client(name);
+	if (clp == NULL)
+		return NULL;
+
+	princ = svc_gss_principal(rqstp);
+	if (princ) {
+		clp->cl_principal = kstrdup(princ, GFP_KERNEL);
+		if (clp->cl_principal == NULL) {
+			free_client(clp);
+			return NULL;
+		}
+	}
+
+	memcpy(clp->cl_recdir, recdir, HEXDIR_LEN);
+	atomic_set(&clp->cl_count, 1);
+	atomic_set(&clp->cl_cb_conn.cb_set, 0);
+	INIT_LIST_HEAD(&clp->cl_idhash);
+	INIT_LIST_HEAD(&clp->cl_strhash);
+	INIT_LIST_HEAD(&clp->cl_openowners);
+	INIT_LIST_HEAD(&clp->cl_delegations);
+	INIT_LIST_HEAD(&clp->cl_sessions);
+	INIT_LIST_HEAD(&clp->cl_lru);
+	clear_bit(0, &clp->cl_cb_slot_busy);
+	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
+	copy_verf(clp, verf);
+	rpc_copy_addr((struct sockaddr *) &clp->cl_addr, sa);
+	clp->cl_flavor = rqstp->rq_flavor;
+	copy_cred(&clp->cl_cred, &rqstp->rq_cred);
+	gen_confirm(clp);
+
+	return clp;
+}
+
 static int check_name(struct xdr_netobj name)
 {
 	if (name.len == 0) 
@@ -1193,17 +1212,13 @@ nfsd4_exchange_id(struct svc_rqst *rqstp,
 
 out_new:
 	/* Normal case */
-	new = create_client(exid->clname, dname);
+	new = create_client(exid->clname, dname, rqstp, &verf);
 	if (new == NULL) {
 		status = nfserr_serverfault;
 		goto out;
 	}
 
-	copy_verf(new, &verf);
-	copy_cred(&new->cl_cred, &rqstp->rq_cred);
-	rpc_copy_addr((struct sockaddr *) &new->cl_addr, sa);
 	gen_clid(new);
-	gen_confirm(new);
 	add_to_unconfirmed(new, strhashval);
 out_copy:
 	exid->clientid.cl_boot = new->cl_clientid.cl_boot;
@@ -1477,7 +1492,6 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	unsigned int 		strhashval;
 	struct nfs4_client	*conf, *unconf, *new;
 	__be32 			status;
-	char			*princ;
 	char                    dname[HEXDIR_LEN];
 	
 	if (!check_name(clname))
@@ -1522,7 +1536,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 */
 		if (unconf)
 			expire_client(unconf);
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
@@ -1539,7 +1553,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			 */
 			expire_client(unconf);
 		}
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		copy_clid(new, conf);
@@ -1549,7 +1563,7 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 * probable client reboot; state will be removed if
 		 * confirmed.
 		 */
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
@@ -1560,24 +1574,11 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		 * confirmed.
 		 */
 		expire_client(unconf);
-		new = create_client(clname, dname);
+		new = create_client(clname, dname, rqstp, &clverifier);
 		if (new == NULL)
 			goto out;
 		gen_clid(new);
 	}
-	copy_verf(new, &clverifier);
-	rpc_copy_addr((struct sockaddr *) &new->cl_addr, sa);
-	new->cl_flavor = rqstp->rq_flavor;
-	princ = svc_gss_principal(rqstp);
-	if (princ) {
-		new->cl_principal = kstrdup(princ, GFP_KERNEL);
-		if (new->cl_principal == NULL) {
-			free_client(new);
-			goto out;
-		}
-	}
-	copy_cred(&new->cl_cred, &rqstp->rq_cred);
-	gen_confirm(new);
 	gen_callback(new, setclid, rpc_get_scope_id(sa));
 	add_to_unconfirmed(new, strhashval);
 	setclid->se_clientid.cl_boot = new->cl_clientid.cl_boot;
Signature · 1 line
1.6.4
#16 lore raw

Re: [PATCH v2 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to Benny Halevy
Quoted 379 lines · Benny Halevy

From: Rahul Iyer <[email protected]>

When the call direction is a reply, copy the xid and call direction into the req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns rpc_garbage.

[get rid of CONFIG_NFSD_V4_1]
[sunrpc: refactoring of svc_tcp_recvfrom]
[nfsd41: sunrpc: create common send routine for the fore and the back channels]
[nfsd41: sunrpc: Use free_page() to free server backchannel pages]
[nfsd41: sunrpc: Document server backchannel locking]
[nfsd41: sunrpc: remove bc_connect_worker()]

[nfsd41: sunrpc: Define xprt_server_backchannel()[

[nfsd41: sunrpc: remove bc_close and bc_init_auto_disconnect dummy functions]
[nfsd41: sunrpc: eliminate unneeded switch statement in xs_setup_tcp()]
[nfsd41: sunrpc: Don't auto close the server backchannel connection]
[nfsd41: sunrpc: Remove unused functions]
[nfsd41: change bc_sock to bc_xprt]
[nfsd41: sunrpc: move struct rpc_buffer def into a common header file]
[nfsd41: sunrpc: use rpc_sleep in bc_send_request so not to block on mutex]
[removed cosmetic changes]
[sunrpc: add new xprt class for nfsv4.1 backchannel]
[sunrpc: v2.1 change handling of auto_close and init_auto_disconnect operations for the nfsv4.1 backchannel]
[reverted more cosmetic leftovers]
[got rid of xprt_server_backchannel]
[separated "nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel"]
Cc Trond Myklebust <[email protected]>
---
 include/linux/sunrpc/svc_xprt.h |    1 +
 include/linux/sunrpc/svcsock.h  |    1 +
 include/linux/sunrpc/xprt.h     |    1 +
 net/sunrpc/sunrpc.h             |    4 +
 net/sunrpc/svc_xprt.c           |    2 +
 net/sunrpc/svcsock.c            |  172 +++++++++++++++++++++++++++++++--------
 net/sunrpc/xprt.c               |   15 +++-
 net/sunrpc/xprtsock.c           |  146 +++++++++++++++++++++++++++++++++
 8 files changed, 303 insertions(+), 39 deletions(-)
include/linux/sunrpc/svc_xprt.h +1 -0
index 2223ae0..5f4e18b 100644
@@ -65,6 +65,7 @@ struct svc_xprt {
 	size_t			xpt_locallen;	/* length of address */
 	struct sockaddr_storage	xpt_remote;	/* remote peer's address */
 	size_t			xpt_remotelen;	/* length of address */
+	struct rpc_wait_queue	xpt_bc_pending;	/* backchannel wait queue */
 };
 
 int	svc_reg_xprt_class(struct svc_xprt_class *);
include/linux/sunrpc/svcsock.h +1 -0
index 04dba23..1b353a7 100644
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt		*sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
include/linux/sunrpc/xprt.h +1 -0
index c090df4..228d694 100644
@@ -179,6 +179,7 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 #if defined(CONFIG_NFS_V4_1)
 	struct svc_serv		*bc_serv;       /* The RPC service which will */
 						/* process the callback */
net/sunrpc/sunrpc.h +4 -0
index 13171e6..90c292e 100644
@@ -43,5 +43,9 @@ static inline int rpc_reply_expected(struct rpc_task *task)
 		(task->tk_msg.rpc_proc->p_decode != NULL);
 }
 
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset);
+
 #endif /* _NET_SUNRPC_SUNRPC_H */
 
net/sunrpc/svc_xprt.c +2 -0
index 912dea5..df124f7 100644
@@ -160,6 +160,7 @@ void svc_xprt_init(struct svc_xprt_class *xcl, struct svc_xprt *xprt,
 	mutex_init(&xprt->xpt_mutex);
 	spin_lock_init(&xprt->xpt_lock);
 	set_bit(XPT_BUSY, &xprt->xpt_flags);
+	rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending");
 }
 EXPORT_SYMBOL_GPL(svc_xprt_init);
 
@@ -810,6 +811,7 @@ int svc_send(struct svc_rqst *rqstp)
 	else
 		len = xprt->xpt_ops->xpo_sendto(rqstp);
 	mutex_unlock(&xprt->xpt_mutex);
+	rpc_wake_up(&xprt->xpt_bc_pending);
 	svc_xprt_release(rqstp);
 
 	if (len == -ECONNREFUSED || len == -ENOTCONN || len == -EAGAIN)
net/sunrpc/svcsock.c +137 -35
index 76a380d..ccc5e83 100644
@@ -49,6 +49,7 @@
 #include <linux/sunrpc/msg_prot.h>
 #include <linux/sunrpc/svcsock.h>
 #include <linux/sunrpc/stats.h>
+#include <linux/sunrpc/xprt.h>
 
 #define RPCDBG_FACILITY	RPCDBG_SVCXPRT
 
@@ -153,49 +154,27 @@ static void svc_set_cmsg_data(struct svc_rqst *rqstp, struct cmsghdr *cmh)
 }
 
 /*
- * Generic sendto routine
+ * send routine intended to be shared by the fore- and back-channel
  */
-static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset)
 {
-	struct svc_sock	*svsk =
-		container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt);
-	struct socket	*sock = svsk->sk_sock;
-	int		slen;
-	union {
-		struct cmsghdr	hdr;
-		long		all[SVC_PKTINFO_SPACE / sizeof(long)];
-	} buffer;
-	struct cmsghdr *cmh = &buffer.hdr;
-	int		len = 0;
 	int		result;
 	int		size;
 	struct page	**ppage = xdr->pages;
 	size_t		base = xdr->page_base;
 	unsigned int	pglen = xdr->page_len;
 	unsigned int	flags = MSG_MORE;
-	RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);
+	int		slen;
+	int		len = 0;
 
 	slen = xdr->len;
 
-	if (rqstp->rq_prot == IPPROTO_UDP) {
-		struct msghdr msg = {
-			.msg_name	= &rqstp->rq_addr,
-			.msg_namelen	= rqstp->rq_addrlen,
-			.msg_control	= cmh,
-			.msg_controllen	= sizeof(buffer),
-			.msg_flags	= MSG_MORE,
-		};
-
-		svc_set_cmsg_data(rqstp, cmh);
-
-		if (sock_sendmsg(sock, &msg, 0) < 0)
-			goto out;
-	}
-
 	/* send head */
 	if (slen == xdr->head[0].iov_len)
 		flags = 0;
-	len = kernel_sendpage(sock, rqstp->rq_respages[0], 0,
+	len = kernel_sendpage(sock, headpage, headoffset,
 				  xdr->head[0].iov_len, flags);
 	if (len != xdr->head[0].iov_len)
 		goto out;
@@ -219,16 +198,58 @@ static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
 		base = 0;
 		ppage++;
 	}
+
 	/* send tail */
 	if (xdr->tail[0].iov_len) {
-		result = kernel_sendpage(sock, rqstp->rq_respages[0],
-					     ((unsigned long)xdr->tail[0].iov_base)
-						& (PAGE_SIZE-1),
-					     xdr->tail[0].iov_len, 0);
-
+		result = kernel_sendpage(sock, tailpage, tailoffset,
+				   xdr->tail[0].iov_len, 0);
 		if (result > 0)
 			len += result;
 	}
+
+out:
+	return len;
+}
+
+
+/*
+ * Generic sendto routine
+ */
+static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr)
+{
+	struct svc_sock	*svsk =
+		container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt);
+	struct socket	*sock = svsk->sk_sock;
+	union {
+		struct cmsghdr	hdr;
+		long		all[SVC_PKTINFO_SPACE / sizeof(long)];
+	} buffer;
+	struct cmsghdr *cmh = &buffer.hdr;
+	int		len = 0;
+	unsigned long tailoff;
+	unsigned long headoff;
+	RPC_IFDEBUG(char buf[RPC_MAX_ADDRBUFLEN]);
+
+	if (rqstp->rq_prot == IPPROTO_UDP) {
+		struct msghdr msg = {
+			.msg_name	= &rqstp->rq_addr,
+			.msg_namelen	= rqstp->rq_addrlen,
+			.msg_control	= cmh,
+			.msg_controllen	= sizeof(buffer),
+			.msg_flags	= MSG_MORE,
+		};
+
+		svc_set_cmsg_data(rqstp, cmh);
+
+		if (sock_sendmsg(sock, &msg, 0) < 0)
+			goto out;
+	}
+
+	tailoff = ((unsigned long)xdr->tail[0].iov_base) & (PAGE_SIZE-1);
+	headoff = 0;
+	len = svc_send_common(sock, xdr, rqstp->rq_respages[0], headoff,
+			       rqstp->rq_respages[0], tailoff);
+
 out:
 	dprintk("svc: socket %p sendto([%p %Zu... ], %d) = %d (addr %s)\n",
 		svsk, xdr->head[0].iov_base, xdr->head[0].iov_len,
@@ -951,6 +972,57 @@ static int svc_tcp_recv_record(struct svc_sock *svsk, struct svc_rqst *rqstp)
 	return -EAGAIN;
 }
 
+static int svc_process_calldir(struct svc_sock *svsk, struct svc_rqst *rqstp,
+			       struct rpc_rqst **reqpp, struct kvec *vec)
+{
+	struct rpc_rqst *req = NULL;
+	u32 *p;
+	u32 xid;
+	u32 calldir;
+	int len;
+
+	len = svc_recvfrom(rqstp, vec, 1, 8);
+	if (len < 0)
+		goto error;
+
+	p = (u32 *)rqstp->rq_arg.head[0].iov_base;
+	xid = *p++;
+	calldir = *p;
+
+	if (calldir == 0) {
+		/* REQUEST is the most common case */
+		vec[0] = rqstp->rq_arg.head[0];
+	} else {
+		/* REPLY */
+		if (svsk->sk_bc_xprt)
+			req = xprt_lookup_rqst(svsk->sk_bc_xprt, xid);
+
+		if (!req) {
+			printk(KERN_NOTICE
+				"%s: Got unrecognized reply: "
+				"calldir 0x%x sk_bc_xprt %p xid %08x\n",
+				__func__, ntohl(calldir),
+				svsk->sk_bc_xprt, xid);
+			vec[0] = rqstp->rq_arg.head[0];
+			goto out;
+		}
+
+		memcpy(&req->rq_private_buf, &req->rq_rcv_buf,
+		       sizeof(struct xdr_buf));
+		/* copy the xid and call direction */
+		memcpy(req->rq_private_buf.head[0].iov_base,
+		       rqstp->rq_arg.head[0].iov_base, 8);
+		vec[0] = req->rq_private_buf.head[0];
+	}
+ out:
+	vec[0].iov_base += 8;
+	vec[0].iov_len -= 8;
+	len = svsk->sk_reclen - 8;
+ error:
+	*reqpp = req;
+	return len;
+}
+
 /*
  * Receive data from a TCP socket.
  */
@@ -962,6 +1034,7 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	int		len;
 	struct kvec *vec;
 	int pnum, vlen;
+	struct rpc_rqst *req = NULL;
 
 	dprintk("svc: tcp_recv %p data %d conn %d close %d\n",
 		svsk, test_bit(XPT_DATA, &svsk->sk_xprt.xpt_flags),
@@ -975,9 +1048,27 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	vec = rqstp->rq_vec;
 	vec[0] = rqstp->rq_arg.head[0];
 	vlen = PAGE_SIZE;
+
+	/*
+	 * We have enough data for the whole tcp record. Let's try and read the
+	 * first 8 bytes to get the xid and the call direction. We can use this
+	 * to figure out if this is a call or a reply to a callback. If
+	 * sk_reclen is < 8 (xid and calldir), then this is a malformed packet.
+	 * In that case, don't bother with the calldir and just read the data.
+	 * It will be rejected in svc_process.
+	 */
+	if (len >= 8) {
+		len = svc_process_calldir(svsk, rqstp, &req, vec);
+		if (len < 0)
+			goto err_again;
+		vlen -= 8;
+	}
+
 	pnum = 1;
 	while (vlen < len) {
-		vec[pnum].iov_base = page_address(rqstp->rq_pages[pnum]);
+		vec[pnum].iov_base = (req) ?
+			page_address(req->rq_private_buf.pages[pnum - 1]) :
+			page_address(rqstp->rq_pages[pnum]);
 		vec[pnum].iov_len = PAGE_SIZE;
 		pnum++;
 		vlen += PAGE_SIZE;
@@ -989,6 +1080,16 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	if (len < 0)
 		goto err_again;
 
+	/*
+	 * Account for the 8 bytes we read earlier
+	 */
+	len += 8;
+
+	if (req) {
+		xprt_complete_rqst(req->rq_task, len);
+		len = 0;
+		goto out;
+	}
 	dprintk("svc: TCP complete record (%d bytes)\n", len);
 	rqstp->rq_arg.len = len;
 	rqstp->rq_arg.page_base = 0;
@@ -1002,6 +1103,7 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp)
 	rqstp->rq_xprt_ctxt   = NULL;
 	rqstp->rq_prot	      = IPPROTO_TCP;
 
+out:
 	/* Reset TCP read info */
 	svsk->sk_reclen = 0;
 	svsk->sk_tcplen = 0;
net/sunrpc/xprt.c +0 -0
index f412a85..f577e5a 100644

net/sunrpc/xprt.c static void xprt_timer(struct rpc_task *task) · line 838

Quoted 7 lines
net/sunrpc/xprt.c +4 -0
@@ -832,6 +832,11 @@ static void xprt_timer(struct rpc_task *task)
 	spin_unlock_bh(&xprt->transport_lock);
 }
 
+static inline int xprt_has_timer(struct rpc_xprt *xprt)
+{
+	return xprt->idle_timeout != (~0);
+}

Why did this change again?

It's a disconnect timer, and the idle_timeout sets the timeout period. A test for whether or not that period is 0 therefore makes sense (a zero timeout being a nonsense value for a timer).

Testing for arbitrary non-zero values is more dubious, and forces the backchannel to explicitly set a non-zero value. What value does that add?

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#18 lore raw

Re: [PATCH v2 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to a message not in the archive
Quoted 15 lines · Trond Myklebust
Quoted 8 lines
net/sunrpc/xprt.c +0 -0
Quoted 7 lines

index f412a85..f577e5a 100644

net/sunrpc/xprt.c +0 -0
@@ -832,6 +832,11 @@ static void xprt_timer(struct rpc_task *task)
 	spin_unlock_bh(&xprt->transport_lock);
 }
 
Quoted 4 lines

net/sunrpc/xprt.c static void xprt_timer(struct rpc_task *task) · line 838

Quoted 4 lines
net/sunrpc/xprt.c +4 -0
+static inline int xprt_has_timer(struct rpc_xprt *xprt)
+{
+	return xprt->idle_timeout != (~0);
+}

Why did this change again?

Quoted 8 lines

It's a disconnect timer, and the idle_timeout sets the timeout period. A test for whether or not that period is 0 therefore makes sense (a zero timeout being a nonsense value for a timer).

Testing for arbitrary non-zero values is more dubious, and forces the backchannel to explicitly set a non-zero value. What value does that add?

Good question. I agree with your direction.

Alexandros, why was this != 0 in PATCH 3/3 v2:
http://linux-nfs.org/pipermail/pnfs/2009-September/009057.html
but changed back to ~0 in PATCH 3/3 v2.1?
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

With this in mind, xs_setup_bc_tcp can simply initialize idle_timeout to zero, right?
xprt->bind_timeout = 0;
xprt->connect_timeout = 0; xprt->reestablish_timeout = 0;
- xprt->idle_timeout = (~0);
+ xprt->idle_timeout = 0;

/*
* The backchannel uses the same socket connection as the

Benny

Quoted 9 lines
Signature · 7 lines
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#19 lore raw

Re: [pnfs] [PATCH v2 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling

Trond,

you are right. It was changed to be similar to the pre-existing code, but there is no need. The timeout value should to be set explicitly to 0 as Benny suggested.

-alexandros

On Thu, Sep 10, 2009 at 5:33 AM, Benny Halevy <[email protected]> wro= te:

Quoted 1 line

On Sep. 10, 2009, 14:49 +0300, Trond Myklebust <Trond.Myklebust@netap=

Quoted 5 lines · p.com> wrote:
Quoted 5 lines
Quoted 5 lines
net/sunrpc/xprt.c +0 -0

net/sunrpc/xprt.c static void xprt_timer(struct rpc_task *tas=

Quoted 4 lines

index f412a85..f577e5a 100644

net/sunrpc/xprt.c +0 -0
@@ -832,6 +832,11 @@ static void xprt_timer(struct rpc_task *tas=

k)

Quoted 11 lines
Quoted 11 lines
Quoted 7 lines

net/sunrpc/xprt.c static void xprt_timer(struct rpc_task *task) · line 838

Quoted 7 lines
net/sunrpc/xprt.c +4 -0
 =A0 spin_unlock_bh(&xprt->transport_lock);
 =A0}
 
+static inline int xprt_has_timer(struct rpc_xprt *xprt)
+{
+ return xprt->idle_timeout !=3D (~0);
+}

Why did this change again?

It's a disconnect timer, and the idle_timeout sets the timeout perio=

d. A

Quoted 1 line
Quoted 1 line

test for whether or not that period is 0 therefore makes sense (a ze=

ro

Quoted 3 lines
Quoted 3 lines

timeout being a nonsense value for a timer).

Testing for arbitrary non-zero values is more dubious, and forces th=

e

Quoted 19 lines
Quoted 3 lines

backchannel to explicitly set a non-zero value. What value does that add?

Good question. I agree with your direction.

Alexandros, why was this !=3D 0 in PATCH 3/3 v2:
http://linux-nfs.org/pipermail/pnfs/2009-September/009057.html
but changed back to ~0 in PATCH 3/3 v2.1?
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

With this in mind, xs_setup_bc_tcp can simply initialize idle_timeout to zero, right?
=A0 =A0 =A0 =A0xprt->bind_timeout =3D 0;
=A0 =A0 =A0 =A0xprt->connect_timeout =3D 0;
=A0 =A0 =A0 =A0xprt->reestablish_timeout =3D 0;
- =A0 =A0 =A0 xprt->idle_timeout =3D (~0);
+ =A0 =A0 =A0 xprt->idle_timeout =3D 0;

Quoted 2 lines

=A0 =A0 =A0 =A0/*
=A0 =A0 =A0 =A0 * The backchannel uses the same socket connection as =

the

Quoted 17 lines

Benny

Quoted 9 lines

--
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#23 lore raw

PATCH v3 2/12 nfsd41: sunrpc: Added rpc server-side backchannel handling

In reply to a message not in the archive

From: Rahul Iyer <[email protected]>

When the call direction is a reply, copy the xid and call direction into the req->rq_private_buf.head[0].iov_base otherwise rpc_verify_header returns rpc_garbage.

[get rid of CONFIG_NFSD_V4_1]
[sunrpc: refactoring of svc_tcp_recvfrom]
[nfsd41: sunrpc: create common send routine for the fore and the back channels]
[nfsd41: sunrpc: Use free_page() to free server backchannel pages]
[nfsd41: sunrpc: Document server backchannel locking]
[nfsd41: sunrpc: remove bc_connect_worker()]

[nfsd41: sunrpc: Define xprt_server_backchannel()[

[nfsd41: sunrpc: remove bc_close and bc_init_auto_disconnect dummy functions]
[nfsd41: sunrpc: eliminate unneeded switch statement in xs_setup_tcp()]
[nfsd41: sunrpc: Don't auto close the server backchannel connection]
[nfsd41: sunrpc: Remove unused functions]
[nfsd41: change bc_sock to bc_xprt]
[nfsd41: sunrpc: move struct rpc_buffer def into a common header file]
[nfsd41: sunrpc: use rpc_sleep in bc_send_request so not to block on mutex]
[removed cosmetic changes]
[sunrpc: add new xprt class for nfsv4.1 backchannel]
[sunrpc: v2.1 change handling of auto_close and init_auto_disconnect operations for the nfsv4.1 backchannel]
[reverted more cosmetic leftovers]
[got rid of xprt_server_backchannel]
[separated "nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel"]
Cc Trond Myklebust <[email protected]>
[sunrpc: change idle timeout value for the backchannel]

---

Changes from v2:
xprt_has_timer checks for idle_timeout == 0 rather than ~0

---
 include/linux/sunrpc/svc_xprt.h |    1 +
 include/linux/sunrpc/svcsock.h  |    1 +
 include/linux/sunrpc/xprt.h     |    1 +
 net/sunrpc/sunrpc.h             |    4 +
 net/sunrpc/svc_xprt.c           |    2 +
 net/sunrpc/svcsock.c            |  172 +++++++++++++++++++++++++++++++--------
 net/sunrpc/xprt.c               |   15 +++-
 net/sunrpc/xprtsock.c           |  146 +++++++++++++++++++++++++++++++++
 8 files changed, 303 insertions(+), 39 deletions(-)
include/linux/sunrpc/svc_xprt.h +1 -0
index 2223ae0..5f4e18b 100644
@@ -65,6 +65,7 @@ struct svc_xprt {
 	size_t			xpt_locallen;	/* length of address */
 	struct sockaddr_storage	xpt_remote;	/* remote peer's address */
 	size_t			xpt_remotelen;	/* length of address */
+	struct rpc_wait_queue	xpt_bc_pending;	/* backchannel wait queue */
 };
 
 int	svc_reg_xprt_class(struct svc_xprt_class *);
include/linux/sunrpc/svcsock.h +1 -0
index 04dba23..1b353a7 100644
@@ -28,6 +28,7 @@ struct svc_sock {
 	/* private TCP part */
 	u32			sk_reclen;	/* length of record */
 	u32			sk_tcplen;	/* current read length */
+	struct rpc_xprt		*sk_bc_xprt;	/* NFSv4.1 backchannel xprt */
 };
 
 /*
include/linux/sunrpc/xprt.h +1 -0
index c090df4..228d694 100644
@@ -179,6 +179,7 @@ struct rpc_xprt {
 	spinlock_t		reserve_lock;	/* lock slot table */
 	u32			xid;		/* Next XID value to use */
 	struct rpc_task *	snd_task;	/* Task blocked in send */
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 #if defined(CONFIG_NFS_V4_1)
 	struct svc_serv		*bc_serv;       /* The RPC service which will */
 						/* process the callback */
net/sunrpc/sunrpc.h +4 -0
index 13171e6..90c292e 100644
@@ -43,5 +43,9 @@ static inline int rpc_reply_expected(struct rpc_task *task)
 		(task->tk_msg.rpc_proc->p_decode != NULL);
 }
 
+int svc_send_common(struct socket *sock, struct xdr_buf *xdr,
+		    struct page *headpage, unsigned long headoffset,
+		    struct page *tailpage, unsigned long tailoffset);
+
 #endif /* _NET_SUNRPC_SUNRPC_H */
 
net/sunrpc/svc_xprt.c +2 -0
index 912dea5..df124f7 100644
@@ -160,6 +160,7 @@ void svc_xprt_init(struct svc_xprt_class *xcl, struct svc_xprt *xprt,
 	mutex_init(&xprt->xpt_mutex);
 	spin_lock_init(&xprt->xpt_lock);
 	set_bit(XPT_BUSY, &xprt->xpt_flags);
+	rpc_init_wait_queue(&xprt->xpt_bc_pending, "xpt_bc_pending");
 }
 EXPORT_SYMBOL_GPL(svc_xprt_init);
 
@@ -810,6 +811,7 @@ int svc_send(struct svc_rqst *rqstp)
 	else
 		len = xprt->xpt_ops->xpo_sendto(rqstp);
 	mutex_unlock(&xprt->xpt_mutex);
+	rpc_wake_up(&xprt->xpt_bc_pending);
 	svc_xprt_release(rqstp);
 
 	if (len == -ECONNREFUSED || len == -ENOTCONN || len == -EAGAIN)
net/sunrpc/svcsock.c +137 -35

241 lines not on this page · raw message

net/sunrpc/xprt.c +11 -4
index f412a85..fd46d42 100644
@@ -832,6 +832,11 @@ static void xprt_timer(struct rpc_task *task)
 	spin_unlock_bh(&xprt->transport_lock);
 }
 
+static inline int xprt_has_timer(struct rpc_xprt *xprt)
+{
+	return xprt->idle_timeout != 0;
+}
+
 /**
  * xprt_prepare_transmit - reserve the transport before sending a request
  * @task: RPC task about to send a request
@@ -1013,7 +1018,7 @@ void xprt_release(struct rpc_task *task)
 	if (!list_empty(&req->rq_list))
 		list_del(&req->rq_list);
 	xprt->last_used = jiffies;
-	if (list_empty(&xprt->recv))
+	if (list_empty(&xprt->recv) && xprt_has_timer(xprt))
 		mod_timer(&xprt->timer,
 				xprt->last_used + xprt->idle_timeout);
 	spin_unlock_bh(&xprt->transport_lock);
@@ -1082,8 +1087,11 @@ found:
 #endif /* CONFIG_NFS_V4_1 */
 
 	INIT_WORK(&xprt->task_cleanup, xprt_autoclose);
-	setup_timer(&xprt->timer, xprt_init_autodisconnect,
-			(unsigned long)xprt);
+	if (xprt_has_timer(xprt))
+		setup_timer(&xprt->timer, xprt_init_autodisconnect,
+			    (unsigned long)xprt);
+	else
+		init_timer(&xprt->timer);
 	xprt->last_used = jiffies;
 	xprt->cwnd = RPC_INITCWND;
 	xprt->bind_index = 0;
@@ -1102,7 +1110,6 @@ found:
 
 	dprintk("RPC:       created transport %p with %u slots\n", xprt,
 			xprt->max_reqs);
-
 	return xprt;
 }
 
net/sunrpc/xprtsock.c +146 -0

170 lines not on this page · raw message

Signature · 1 line
1.6.4
#25 lore raw

PATCH v3 3/12 nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

In reply to Benny Halevy

From: Alexandros Batsakis <[email protected]>

[sunrpc: change idle timeout value for the backchannel]

---

Changes from v2:

xs_setup_bc_tcp sets cprt->idle_timeout = 0 rather than ~0

---
 include/linux/sunrpc/clnt.h     |    1 +
 include/linux/sunrpc/xprt.h     |    1 +
 include/linux/sunrpc/xprtrdma.h |    5 --
 include/linux/sunrpc/xprtsock.h |    9 +++-
 net/sunrpc/clnt.c               |    1 +
 net/sunrpc/xprtsock.c           |   96 ++++++++++++++++++++++++++++++++++++++-
 6 files changed, 104 insertions(+), 9 deletions(-)
include/linux/sunrpc/clnt.h +1 -0
index 3d02558..8ed9642 100644
@@ -114,6 +114,7 @@ struct rpc_create_args {
 	rpc_authflavor_t	authflavor;
 	unsigned long		flags;
 	char			*client_name;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 /* Values for "flags" field */
include/linux/sunrpc/xprt.h +1 -0
index 228d694..7cc42af 100644
@@ -232,6 +232,7 @@ struct xprt_create {
 	struct sockaddr *	srcaddr;	/* optional local address */
 	struct sockaddr *	dstaddr;	/* remote peer address */
 	size_t			addrlen;
+	struct svc_xprt		*bc_xprt;	/* NFSv4.1 backchannel */
 };
 
 struct xprt_class {
include/linux/sunrpc/xprtrdma.h +0 -5
index 54a379c..c2f04e1 100644
@@ -41,11 +41,6 @@
 #define _LINUX_SUNRPC_XPRTRDMA_H
 
 /*
- * RPC transport identifier for RDMA
- */
-#define XPRT_TRANSPORT_RDMA	256
-
-/*
  * rpcbind (v3+) RDMA netid.
  */
 #define RPCBIND_NETID_RDMA	"rdma"
include/linux/sunrpc/xprtsock.h +7 -2
index c2a46c4..d7c98d1 100644
@@ -20,8 +20,13 @@ void		cleanup_socket_xprt(void);
  * values. No such restriction exists for new transports, except that
  * they may not collide with these values (17 and 6, respectively).
  */
-#define XPRT_TRANSPORT_UDP	IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP	IPPROTO_TCP
+#define XPRT_TRANSPORT_BC	(1 << 31)
+enum xprt_transports {
+	XPRT_TRANSPORT_UDP 	= IPPROTO_UDP,
+	XPRT_TRANSPORT_TCP 	= IPPROTO_TCP,
+	XPRT_TRANSPORT_BC_TCP 	= IPPROTO_TCP | XPRT_TRANSPORT_BC,
+	XPRT_TRANSPORT_RDMA	= 256
+};
 
 /*
  * RPC slot table sizes for UDP, TCP transports
net/sunrpc/clnt.c +1 -0
index c1e467e..7389804 100644
@@ -288,6 +288,7 @@ struct rpc_clnt *rpc_create(struct rpc_create_args *args)
 		.srcaddr = args->saddress,
 		.dstaddr = args->address,
 		.addrlen = args->addrsize,
+		.bc_xprt = args->bc_xprt,
 	};
 	char servername[48];
 
net/sunrpc/xprtsock.c +94 -2

127 lines not on this page · raw message

Signature · 1 line
1.6.4
#66 lore raw
In reply to Benny Halevy

Bruce, I've updated patches 2 and 3 based on Trond's comment to PATCH v2 02/12 and squashed Alexandros fix for it.

Here's the latest list of patches:
[PATCH v2 01/12] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH v3 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel [PATCH v2 04/12] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure
[PATCH v2 06/12] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH v2 07/12] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH v2 08/12] nfsd41: Backchannel: Setup sequence information
[PATCH v2 09/12] nfsd41: Backchannel: cb_sequence callback
[PATCH v2 10/12] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH v2 11/12] nfsd41: modify nfsd4.1 backchannel to use new xprt class [PATCH v2 12/12] nfsd41: Refactor create_client()

Thanks,

Benny

Quoted 53 lines · Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

Benny

Quoted 26 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Quoted 5 lines

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#67 lore raw
Quoted 2 lines · Benny Halevy

Bruce, I've updated patches 2 and 3 based on Trond's comment to PATCH v2 02/12 and squashed Alexandros fix for it.

I forgot to mention that this patchset is also available on
git://linux-nfs.org/~bhalevy/linux-pnfs.git nfsd41-for-2.6.32

Benny

Quoted 77 lines

Here's the latest list of patches:
[PATCH v2 01/12] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH v3 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel [PATCH v2 04/12] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure
[PATCH v2 06/12] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH v2 07/12] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH v2 08/12] nfsd41: Backchannel: Setup sequence information
[PATCH v2 09/12] nfsd41: Backchannel: cb_sequence callback
[PATCH v2 10/12] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH v2 11/12] nfsd41: modify nfsd4.1 backchannel to use new xprt class [PATCH v2 12/12] nfsd41: Refactor create_client()

Thanks,

Benny

Quoted 49 lines · Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

Benny

Quoted 26 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Quoted 5 lines

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

Quoted 4 lines

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#14 lore raw

Re: [PATCH v2 01/12] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h

In reply to Benny Halevy
Quoted 2 lines · Benny Halevy

Move struct rpc_buffer's definition into a sunrpc.h, a common, internal header file, in preparation for supporting the nfsv4.1 backchannel.

Applied, thanks.

--b.

Quoted 58 lines
[nfs41: sunrpc: #include <linux/net.h> from sunrpc.h]
---
 net/sunrpc/sched.c  |    7 ++-----
 net/sunrpc/sunrpc.h |   10 ++++++++++
 2 files changed, 12 insertions(+), 5 deletions(-)
net/sunrpc/sched.c +2 -5
index 8f459ab..cef74ba 100644
@@ -21,6 +21,8 @@
 
 #include <linux/sunrpc/clnt.h>
 
+#include "sunrpc.h"
+
 #ifdef RPC_DEBUG
 #define RPCDBG_FACILITY		RPCDBG_SCHED
 #define RPC_TASK_MAGIC_ID	0xf00baa
@@ -711,11 +713,6 @@ static void rpc_async_schedule(struct work_struct *work)
 	__rpc_execute(container_of(work, struct rpc_task, u.tk_work));
 }
 
-struct rpc_buffer {
-	size_t	len;
-	char	data[];
-};
-
 /**
  * rpc_malloc - allocate an RPC buffer
  * @task: RPC task that will use this buffer
net/sunrpc/sunrpc.h +10 -0
index 5d9dd74..13171e6 100644
@@ -27,6 +27,16 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 #ifndef _NET_SUNRPC_SUNRPC_H
 #define _NET_SUNRPC_SUNRPC_H
 
+#include <linux/net.h>
+
+/*
+ * Header for dynamically allocated rpc buffers.
+ */
+struct rpc_buffer {
+	size_t	len;
+	char	data[];
+};
+
 static inline int rpc_reply_expected(struct rpc_task *task)
 {
 	return (task->tk_msg.rpc_proc != NULL) &&
Signature · 1 line
1.6.4
#68 lore raw
In reply to Benny Halevy
Quoted 2 lines · Benny Halevy

Bruce, I've updated patches 2 and 3 based on Trond's comment to PATCH v2 02/12 and squashed Alexandros fix for it.

Thanks! Trond, did you have any more comments?

--b.

Quoted 74 lines

Here's the latest list of patches:
[PATCH v2 01/12] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH v3 02/12] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel [PATCH v2 04/12] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure
[PATCH v2 06/12] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH v2 07/12] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH v2 08/12] nfsd41: Backchannel: Setup sequence information
[PATCH v2 09/12] nfsd41: Backchannel: cb_sequence callback
[PATCH v2 10/12] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH v2 11/12] nfsd41: modify nfsd4.1 backchannel to use new xprt class [PATCH v2 12/12] nfsd41: Refactor create_client()

Thanks,

Benny

Quoted 53 lines · Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

Benny

Quoted 26 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Quoted 5 lines

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#69 lore raw
Quoted 5 lines · J. Bruce Fields
Quoted 2 lines · Benny Halevy

Bruce, I've updated patches 2 and 3 based on Trond's comment to PATCH v2 02/12 and squashed Alexandros fix for it.

Thanks! Trond, did you have any more comments?

No.

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#26 lore raw

Re: [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

In reply to Benny Halevy
Quoted 23 lines · Benny Halevy
include/linux/sunrpc/xprtrdma.h +0 -5
index 54a379c..c2f04e1 100644
@@ -41,11 +41,6 @@
 #define _LINUX_SUNRPC_XPRTRDMA_H
 
 /*
- * RPC transport identifier for RDMA
- */
-#define XPRT_TRANSPORT_RDMA	256
-
-/*
  * rpcbind (v3+) RDMA netid.
  */
 #define RPCBIND_NETID_RDMA	"rdma"
include/linux/sunrpc/xprtsock.h +0 -0
index c2a46c4..d7c98d1 100644
@@ -20,8 +20,13 @@ void		cleanup_socket_xprt(void);
  * values. No such restriction exists for new transports, except that
  * they may not collide with these values (17 and 6, respectively).

include/linux/sunrpc/xprtsock.h void cleanup_socket_xprt(void); · line 29

Quoted 10 lines
include/linux/sunrpc/xprtsock.h +7 -2
  */
-#define XPRT_TRANSPORT_UDP	IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP	IPPROTO_TCP
+#define XPRT_TRANSPORT_BC	(1 << 31)
+enum xprt_transports {
+	XPRT_TRANSPORT_UDP 	= IPPROTO_UDP,
+	XPRT_TRANSPORT_TCP 	= IPPROTO_TCP,
+	XPRT_TRANSPORT_BC_TCP 	= IPPROTO_TCP | XPRT_TRANSPORT_BC,
+	XPRT_TRANSPORT_RDMA	= 256
+};

This fails to compile when CONFIG_SUNRPC_XPRT_RDMA is set.

A minimal fix might be:

	--- a/net/sunrpc/xprtrdma/transport.c
	+++ b/net/sunrpc/xprtrdma/transport.c
	@@ -50,6 +50,8 @@
	 #include <linux/module.h>
	 #include <linux/init.h>
	 #include <linux/seq_file.h>
	+#include <linux/in.h>
	+#include <linux/sunrpc/xprtsock.h>
 
	 #include "xprt_rdma.h"

Or maybe just ditch the enum and leave these as they were before.

--b.

#27 lore raw

Re: [pnfs] [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

Quoted 2 lines · On Fri, Sep 11, 2009 at 1:58 PM, J. Bruce Fields <[email protected]>= wrote:

include/linux/sunrpc/=

Quoted 1 line · Benny Halevy
include/linux/sunrpc/= renamed from include/linux/sunrpc/xprtrdma.h +0 -0

xprtrdma.h

Quoted 6 lines
Quoted 6 lines

index 54a379c..c2f04e1 100644

include/linux/sunrpc/xprtrdma.h +0 -0
@@ -41,11 +41,6 @@

=A0#define _LINUX_SUNRPC_XPRTRDMA_H

Quoted 10 lines

include/linux/sunrpc/=

Quoted 10 lines

=A0/*
- * RPC transport identifier for RDMA
- */

include/linux/sunrpc/xprtrdma.h +0 -3
-#define XPRT_TRANSPORT_RDMA =A0256
-
-/*

=A0 * rpcbind (v3+) RDMA netid.
=A0 */
=A0#define RPCBIND_NETID_RDMA =A0 "rdma"

include/linux/sunrpc/= renamed from include/linux/sunrpc/xprtsock.h +0 -0

xprtsock.h

Quoted 4 lines

include/linux/sunrpc/xprtsock.h void =A0 =A0 =A0 =A0 =A0 =A0 =A0 cleanup_socket_x=

Quoted 4 lines

index c2a46c4..d7c98d1 100644

include/linux/sunrpc/xprtsock.h +0 -0
@@ -20,8 +20,13 @@ void =A0 =A0 =A0 =A0 =A0 =A0 =A0 cleanup_socket_x=

prt(void);

Quoted 1 line
Quoted 1 line

=A0 * values. No such restriction exists for new transports, except =

that

Quoted 1 line
Quoted 1 line

=A0 * they may not collide with these values (17 and 6, respectively=

).

Quoted 8 lines

include/linux/sunrpc/xprtsock.h

Quoted 8 lines

=A0 */

include/linux/sunrpc/xprtsock.h +5 -2
-#define XPRT_TRANSPORT_UDP =A0 IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP =A0 IPPROTO_TCP
+#define XPRT_TRANSPORT_BC =A0 =A0(1 << 31)
+enum xprt_transports {
+ =A0 =A0 XPRT_TRANSPORT_UDP =A0 =A0 =A0=3D IPPROTO_UDP,
+ =A0 =A0 XPRT_TRANSPORT_TCP =A0 =A0 =A0=3D IPPROTO_TCP,
+ =A0 =A0 XPRT_TRANSPORT_BC_TCP =A0 =3D IPPROTO_TCP | XPRT_TRANSPORT=

_BC,

Quoted 16 lines
Quoted 2 lines

+ =A0 =A0 XPRT_TRANSPORT_RDMA =A0 =A0 =3D 256
+};

This fails to compile when CONFIG_SUNRPC_XPRT_RDMA is set.

A minimal fix might be:

=A0 =A0 =A0 =A0--- a/net/sunrpc/xprtrdma/transport.c =A0 =A0 =A0 =A0+++ b/net/sunrpc/xprtrdma/transport.c =A0 =A0 =A0 =A0@@ -50,6 +50,8 @@
=A0 =A0 =A0 =A0 #include <linux/module.h> =A0 =A0 =A0 =A0 #include <linux/init.h>
=A0 =A0 =A0 =A0 #include <linux/seq_file.h> =A0 =A0 =A0 =A0+#include <linux/in.h>
=A0 =A0 =A0 =A0+#include <linux/sunrpc/xprtsock.h>

Quoted 4 lines

=A0 =A0 =A0 =A0 #include "xprt_rdma.h"

Or maybe just ditch the enum and leave these as they were before.

The incentive here was to have all the transports definitions together so that nobody re-uses a number by mistake (not very likely of course), so I think the fix you suggested is appropriate.

-alexandros

Quoted 6 lines

--b.
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#20 lore raw

Build error of latest Linux-pnfs 2.6.31

In reply to Benny Halevy

I got the following error from include/linux/stats.h. It looks
obviously that the line 481 and line 499 are duplicated. If I remove
line 481 from that file, then I got error in the end "ERROR: "nfs4_reset_lease" [fs/nfsd/nfsd.ko] undefined!".

Could you tell me how to fix that? I downloaded the package using git
by doing:

git clone git://git.linux-nfs.org/projects/bhalevy/linux-pnfs.git

On 09/11/2009. So I got the latest patch.

When Make started, I got error from

Thanks for help

Peixing

CC [M] fs/nfsd/nfs4proc.o
In file included from fs/nfsd/nfs4proc.c:47:
include/linux/nfsd/state.h:483: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:483: warning: its scope is only this definition or declaration, which is probably not what you want include/linux/nfsd/state.h:484: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:485: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:487: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:518: error: static declaration of release_pnfs_ds_dev_list follows non-static declaration include/linux/nfsd/state.h:481: error: previous declaration of release_pnfs_ds_dev_list was here
make[2]: *** [fs/nfsd/nfs4proc.o] Error 1

#22 lore raw

Re: Build error of latest Linux-pnfs 2.6.31

In reply to a message not in the archive
Quoted 7 lines · On 2009-09-12 01:29, [email protected] wrote:

I got the following error from include/linux/stats.h. It looks
obviously that the line 481 and line 499 are duplicated. If I remove
line 481 from that file, then I got error in the end "ERROR: "nfs4_reset_lease" [fs/nfsd/nfsd.ko] undefined!".

Could you tell me how to fix that? I downloaded the package using git
by doing:

It looks like CONFIG_PNFSD is configured as 'n' isn't it? Anyway, does the following patch help?

Quoted 1 line

From 9a0a09a7cb3116dcb2f1610eb7579e06e80a60f4 Mon Sep 17 00:00:00 2001

From: Benny Halevy <[email protected]>
Date: Sun, 13 Sep 2009 12:26:31 +0300
Subject: [PATCH] SQUASHME: pnfsd: fix compiler warnings when CONFIG_PNFSD is not defined

---
 include/linux/nfsd/state.h |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)
include/linux/nfsd/state.h +4 -4
index 9c0df5b..28e491d 100644
@@ -478,14 +478,14 @@ extern struct nfs4_stateid * find_stateid(stateid_t *stid, int flags);
 extern struct nfs4_delegation * find_delegation_stateid(struct inode *ino,
 		stateid_t *stid);
 extern __be32 nfs4_check_stateid(stateid_t *stateid);
+extern void expire_client_lock(struct nfs4_client *clp);
+#if defined(CONFIG_PNFSD)
 extern void release_pnfs_ds_dev_list(struct nfs4_stateid *stp);
 extern void nfs4_pnfs_state_init(void);
 extern int put_layoutrecall(struct nfs4_layoutrecall *);
 extern void nomatching_layout(struct nfs4_layoutrecall *);
 extern void *layoutrecall_done(struct nfs4_layoutrecall *);
-extern void expire_client_lock(struct nfs4_client *clp);
 extern int nfsd4_cb_layout(struct nfs4_layoutrecall *lp);
-#if defined(CONFIG_PNFSD)
 extern void nfsd4_free_pnfs_slabs(void);
 extern void nfsd4_free_slab(struct kmem_cache **slab);
 extern int nfsd4_init_pnfs_slabs(void);
@@ -515,8 +515,8 @@ extern void pnfs_expire_client(struct nfs4_client *clp);
 #else /* CONFIG_PNFSD */
 static inline void nfsd4_free_pnfs_slabs(void) {}
 static inline int nfsd4_init_pnfs_slabs(void) { return 0; }
-static void release_pnfs_ds_dev_list(struct nfs4_stateid *stp) {}
-static void pnfs_expire_client(struct nfs4_client *clp) {}
+static inline void release_pnfs_ds_dev_list(struct nfs4_stateid *stp) {}
+static inline void pnfs_expire_client(struct nfs4_client *clp) {}
 #endif /* CONFIG_PNFSD */
 
 static inline void
Signature · 1 line
1.6.4
Quoted 28 lines

git clone git://git.linux-nfs.org/projects/bhalevy/linux-pnfs.git

On 09/11/2009. So I got the latest patch.

When Make started, I got error from

Thanks for help

Peixing

CC [M] fs/nfsd/nfs4proc.o
In file included from fs/nfsd/nfs4proc.c:47:
include/linux/nfsd/state.h:483: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:483: warning: its scope is only this definition or declaration, which is probably not what you want include/linux/nfsd/state.h:484: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:485: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:487: warning: struct nfs4_layoutrecall declared inside parameter list include/linux/nfsd/state.h:518: error: static declaration of release_pnfs_ds_dev_list follows non-static declaration include/linux/nfsd/state.h:481: error: previous declaration of release_pnfs_ds_dev_list was here
make[2]: *** [fs/nfsd/nfs4proc.o] Error 1

#59 lore raw

Re: [PATCH v2 09/12] nfsd41: Backchannel: cb_sequence callback

In reply to Benny Halevy
Quoted 6 lines · Benny Halevy

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Note: highest slot id and target highest slot id do not have to be 0 as was previously implemented. They can be greater than what the nfs server sent if the client supports a larger slot table on the backchannel. At this point we just ignore that.

Minor point (applying as is), but, in future: a changelog that says how this version of the patch differs from a previous version won't be useful to someone reading the git history (and lacking the previous versions). If you think the above mistake is one that someone might risk making again, a comment in the appropriate spot in the code might be more useful.

--b.

Quoted 116 lines
[Rework the back channel xdr using the shared v4.0 and v4.1 framework.]
[fixed indentation]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]

[nfsd41: fix verification of CB_SEQUENCE highest slot id[

[nfsd41: Backchannel: Remove old backchannel serialization]
[nfsd41: Backchannel: First callback sequence ID should be 1]
[nfsd41: decode_cb_sequence does not need to actually decode ignored fields]
---
 fs/nfsd/nfs4callback.c |   72 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 72 insertions(+), 0 deletions(-)
fs/nfsd/nfs4callback.c +72 -0
index e79e3a4..5e9659c 100644
@@ -256,6 +256,27 @@ encode_cb_recall(struct xdr_stream *xdr, struct nfs4_delegation *dp,
 	hdr->nops++;
 }
 
+static void
+encode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *args,
+		   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	if (hdr->minorversion == 0)
+		return;
+
+	RESERVE_SPACE(1 + NFS4_MAX_SESSIONID_LEN + 20);
+
+	WRITE32(OP_CB_SEQUENCE);
+	WRITEMEM(args->cbs_clp->cl_sessionid.data, NFS4_MAX_SESSIONID_LEN);
+	WRITE32(args->cbs_clp->cl_cb_seq_nr);
+	WRITE32(0);		/* slotid, always 0 */
+	WRITE32(0);		/* highest slotid always 0 */
+	WRITE32(0);		/* cachethis always 0 */
+	WRITE32(0); /* FIXME: support referring_call_lists */
+	hdr->nops++;
+}
+
 static int
 nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 {
@@ -317,6 +338,57 @@ decode_cb_op_hdr(struct xdr_stream *xdr, enum nfs_opnum4 expected)
 	return 0;
 }
 
+/*
+ * Our current back channel implmentation supports a single backchannel
+ * with a single slot.
+ */
+static int
+decode_cb_sequence(struct xdr_stream *xdr, struct nfsd4_cb_sequence *res,
+		   struct rpc_rqst *rqstp)
+{
+	struct nfs4_sessionid id;
+	int status;
+	u32 dummy;
+	__be32 *p;
+
+	if (res->cbs_minorversion == 0)
+		return 0;
+
+	status = decode_cb_op_hdr(xdr, OP_CB_SEQUENCE);
+	if (status)
+		return status;
+
+	/*
+	 * If the server returns different values for sessionID, slotID or
+	 * sequence number, the server is looney tunes.
+	 */
+	status = -ESERVERFAULT;
+
+	READ_BUF(NFS4_MAX_SESSIONID_LEN + 16);
+	memcpy(id.data, p, NFS4_MAX_SESSIONID_LEN);
+	p += XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN);
+	if (memcmp(id.data, res->cbs_clp->cl_sessionid.data,
+		   NFS4_MAX_SESSIONID_LEN)) {
+		dprintk("%s Invalid session id\n", __func__);
+		goto out;
+	}
+	READ32(dummy);
+	if (dummy != res->cbs_clp->cl_cb_seq_nr) {
+		dprintk("%s Invalid sequence number\n", __func__);
+		goto out;
+	}
+	READ32(dummy); 	/* slotid must be 0 */
+	if (dummy != 0) {
+		dprintk("%s Invalid slotid\n", __func__);
+		goto out;
+	}
+	/* FIXME: process highest slotid and target highest slotid */
+	status = 0;
+out:
+	return status;
+}
+
+
 static int
 nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 {
Signature · 1 line
1.6.4
#29 lore raw

Re: [pnfs] [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

In reply to a message not in the archive
Quoted 1 line · Alexandros Batsakis

On Fri, Sep 11, 2009 at 1:58 PM, J. Bruce Fields <bfields-uC3wQj2KruMpug/[email protected]=

Quoted 2 lines · g> wrote:
Quoted 2 lines · Benny Halevy

include/linux/sunrp=

Quoted 1 line
include/linux/sunrp= renamed from include/linux/sunrpc/xprtrdma.h +0 -0

c/xprtrdma.h

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

index 54a379c..c2f04e1 100644

include/linux/sunrpc/xprtrdma.h +0 -0
@@ -41,11 +41,6 @@

=C2=A0#define _LINUX_SUNRPC_XPRTRDMA_H

Quoted 10 lines
Quoted 10 lines

include/linux/sunrp=

Quoted 10 lines

=C2=A0/*
- * RPC transport identifier for RDMA
- */

include/linux/sunrpc/xprtrdma.h +0 -3
-#define XPRT_TRANSPORT_RDMA =C2=A0256
-
-/*

=C2=A0 * rpcbind (v3+) RDMA netid.
=C2=A0 */
=C2=A0#define RPCBIND_NETID_RDMA =C2=A0 "rdma"

include/linux/sunrp= renamed from include/linux/sunrpc/xprtsock.h +0 -0

c/xprtsock.h

Quoted 4 lines
Quoted 4 lines

include/linux/sunrpc/xprtsock.h void =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =

Quoted 4 lines

index c2a46c4..d7c98d1 100644

include/linux/sunrpc/xprtsock.h +0 -0
@@ -20,8 +20,13 @@ void =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =

=C2=A0 cleanup_socket_xprt(void);

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 * values. No such restriction exists for new transports, ex=

cept that

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 * they may not collide with these values (17 and 6, respect=

ively).

Quoted 6 lines
Quoted 6 lines

include/linux/sunrpc/xprtsock.h

Quoted 6 lines

=C2=A0 */

include/linux/sunrpc/xprtsock.h +3 -2
-#define XPRT_TRANSPORT_UDP =C2=A0 IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP =C2=A0 IPPROTO_TCP
+#define XPRT_TRANSPORT_BC =C2=A0 =C2=A0(1 << 31)
+enum xprt_transports {
+ =C2=A0 =C2=A0 XPRT_TRANSPORT_UDP =C2=A0 =C2=A0 =C2=A0=3D IPPROTO=

_UDP,

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 XPRT_TRANSPORT_TCP =C2=A0 =C2=A0 =C2=A0=3D IPPROTO=

_TCP,

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 XPRT_TRANSPORT_BC_TCP =C2=A0 =3D IPPROTO_TCP | XPR=

T_TRANSPORT_BC,

Quoted 16 lines
Quoted 16 lines
Quoted 2 lines

+ =C2=A0 =C2=A0 XPRT_TRANSPORT_RDMA =C2=A0 =C2=A0 =3D 256 +};

This fails to compile when CONFIG_SUNRPC_XPRT_RDMA is set.

A minimal fix might be:

=C2=A0 =C2=A0 =C2=A0 =C2=A0--- a/net/sunrpc/xprtrdma/transport.c =C2=A0 =C2=A0 =C2=A0 =C2=A0+++ b/net/sunrpc/xprtrdma/transport.c =C2=A0 =C2=A0 =C2=A0 =C2=A0@@ -50,6 +50,8 @@
=C2=A0 =C2=A0 =C2=A0 =C2=A0 #include <linux/module.h> =C2=A0 =C2=A0 =C2=A0 =C2=A0 #include <linux/init.h> =C2=A0 =C2=A0 =C2=A0 =C2=A0 #include <linux/seq_file.h> =C2=A0 =C2=A0 =C2=A0 =C2=A0+#include <linux/in.h> =C2=A0 =C2=A0 =C2=A0 =C2=A0+#include <linux/sunrpc/xprtsock.h>

Quoted 6 lines
Quoted 4 lines

=C2=A0 =C2=A0 =C2=A0 =C2=A0 #include "xprt_rdma.h"

Or maybe just ditch the enum and leave these as they were before.

=20
The incentive here was to have all the transports definitions togethe=

r

Quoted 2 lines

so that nobody re-uses a number by mistake (not very likely of course), so I think the fix you suggested is appropriate.

OK. Actually, rather than adding a socket-specific include to an rdma-specific file, how about the below? Applying with the following change assuming no objections.

--b.

include/linux/sunrpc/xprt.h +0 -0
index 7cc42af..6f9457a 100644
@@ -124,6 +124,23 @@ struct rpc_xprt_ops {
 	void		(*print_stats)(struct rpc_xprt *xprt, struct seq_file *seq);
 };

=20
+/*
+ * RPC transport identifiers
+ *
+ * To preserve compatibility with the historical use of raw IP protoco= l
+ * id's for transport selection, UDP and TCP identifiers are specified
+ * with the previous values. No such restriction exists for new transp= orts,
+ * except that they may not collide with these values (17 and 6,
+ * respectively).
+ */
+#define XPRT_TRANSPORT_BC (1 << 31)
+enum xprt_transports {
+ XPRT_TRANSPORT_UDP =3D IPPROTO_UDP,
+ XPRT_TRANSPORT_TCP =3D IPPROTO_TCP,
+ XPRT_TRANSPORT_BC_TCP =3D IPPROTO_TCP | XPRT_TRANSPORT_BC,
+ XPRT_TRANSPORT_RDMA =3D 256
+};
+
struct rpc_xprt {
struct kref kref; /* Reference count */
struct rpc_xprt_ops * ops; /* transport methods */

include/linux/sunrpc/xpr= renamed from include/linux/sunrpc/xprtsock.h +0 -0

tsock.h
index d7c98d1..3f14a02 100644

include/linux/sunrpc/xprtsock.h +0 -0
@@ -13,22 +13,6 @@ int		init_socket_xprt(void);
 void		cleanup_socket_xprt(void);

=20
/*
- * RPC transport identifiers for UDP, TCP
- *
- * To preserve compatibility with the historical use of raw IP protoco= l
- * id's for transport selection, these are specified with the previous
- * values. No such restriction exists for new transports, except that
- * they may not collide with these values (17 and 6, respectively).
- */
-#define XPRT_TRANSPORT_BC (1 << 31)
-enum xprt_transports {
- XPRT_TRANSPORT_UDP =3D IPPROTO_UDP,
- XPRT_TRANSPORT_TCP =3D IPPROTO_TCP,
- XPRT_TRANSPORT_BC_TCP =3D IPPROTO_TCP | XPRT_TRANSPORT_BC,
- XPRT_TRANSPORT_RDMA =3D 256
-};
-
-/*
* RPC slot table sizes for UDP, TCP transports
*/
extern unsigned int xprt_udp_slot_table_entries;

#62 lore raw

Re: [PATCH v2 10/12] nfsd41: Backchannel: Implement cb_recall over NFSv4.1

In reply to Benny Halevy
Quoted 50 lines · Benny Halevy
(quoted diff) +13 -2
@@ -668,16 +701,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -703,16 +739,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
 	int status;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args) {
+		status = -ENOMEM;
+		goto out;
+	}
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
 		put_nfs4_client(clp);
 		nfs4_put_delegation(dp);
Signature · 1 line
1.6.4

Tracing down through rpc_call_async.... It doesn't look to me like it will call rpc_call_done on returning an error, so you're leaking args in that case. Probably just need a kfree(args); applying as below.

--b.

commit 18d8774a149c28b8c9ac66592873e78bbc66330a

Author: Ricardo Labiaga <[email protected]>
Date:   Thu Sep 10 12:27:04 2009 +0300

nfsd41: Backchannel: Implement cb_recall over NFSv4.1

[nfsd41: cb_recall callback]
[Share v4.0 and v4.1 back channel xdr]
[Share v4.0 and v4.1 back channel xdr]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]
[nfsd41: conditionally decode_sequence in nfs4_xdr_dec_cb_recall]
[nfsd41: Backchannel: Add sequence arguments to callback RPC arguments]
[pulled-in definition of nfsd4_cb_done]
fs/nfsd/nfs4callback.c +48 -5
index 74c8b33..fabfe12 100644
@@ -267,15 +267,19 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_delegation *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_delegation *args = rpc_args->args_op;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->dl_ident,
+		.minorversion = rpc_args->args_seq.cbs_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, &rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -324,7 +328,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		struct nfsd4_cb_sequence *seq)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -334,6 +339,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	if (seq) {
+		status = decode_cb_sequence(&xdr, seq, rqstp);
+		if (status)
+			goto out;
+	}
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -575,11 +585,34 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
 	rpc_call_start(task);
 }
 
+static void nfsd4_cb_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+
+	dprintk("%s: minorversion=%d\n", __func__,
+		clp->cl_cb_conn.cb_minorversion);
+
+	if (clp->cl_cb_conn.cb_minorversion) {
+		/* No need for lock, access serialized in nfsd4_cb_prepare */
+		++clp->cl_cb_seq_nr;
+		clear_bit(0, &clp->cl_cb_slot_busy);
+		rpc_wake_up_next(&clp->cl_cb_waitq);
+		dprintk("%s: freed slot, new seqid=%d\n", __func__,
+			clp->cl_cb_seq_nr);
+
+		/* We're done looking into the sequence information */
+		task->tk_msg.rpc_resp = NULL;
+	}
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
 	struct nfs4_client *clp = dp->dl_client;
 
+	nfsd4_cb_done(task, calldata);
+
 	switch (task->tk_status) {
 	case -EIO:
 		/* Network partition? */
@@ -592,16 +625,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -627,17 +663,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
-	int status;
+	int status = -ENOMEM;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args)
+		goto out;
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
+		kfree(args);
 		put_nfs4_client(clp);
 		nfs4_put_delegation(dp);
 	}
#60 lore raw

Re: [pnfs] [PATCH v2 09/12] nfsd41: Backchannel: cb_sequence callback

In reply to J. Bruce Fields
Quoted 8 lines · J. Bruce Fields
Quoted 6 lines · Benny Halevy

Implement the cb_sequence callback conforming to draft-ietf-nfsv4-minorversion1

Note: highest slot id and target highest slot id do not have to be 0 as was previously implemented. They can be greater than what the nfs server sent if the client supports a larger slot table on the backchannel. At this point we just ignore that.

Quoted 9 lines

Minor point (applying as is), but, in future: a changelog that says how this version of the patch differs from a previous version won't be useful to someone reading the git history (and lacking the previous versions). If you think the above mistake is one that someone might risk making again, a comment in the appropriate spot in the code might be more useful.

--b.

I disagree. The paragraph above is perfectly understandable if you just remove the "as was previously implemented". There is no missing information. Then if so, the "as was previously implemented" is useful extra information that says that "we tried that before and it was bad". This is the kind of information that is important to a reader, even without reading the old version.

Just my $0.017

Boaz

#30 lore raw

Re: [pnfs] [PATCH v3 03/12] nfsd41: sunrpc: add new xprt class for nfsv4.1 backchannel

In reply to J. Bruce Fields
Quoted 53 lines · J. Bruce Fields
Quoted 52 lines · Alexandros Batsakis
Quoted 51 lines · J. Bruce Fields
Quoted 23 lines · Benny Halevy
include/linux/sunrpc/xprtrdma.h +0 -5
index 54a379c..c2f04e1 100644
@@ -41,11 +41,6 @@
 #define _LINUX_SUNRPC_XPRTRDMA_H
 
 /*
- * RPC transport identifier for RDMA
- */
-#define XPRT_TRANSPORT_RDMA  256
-
-/*
  * rpcbind (v3+) RDMA netid.
  */
 #define RPCBIND_NETID_RDMA   "rdma"
include/linux/sunrpc/xprtsock.h +0 -0
index c2a46c4..d7c98d1 100644
@@ -20,8 +20,13 @@ void               cleanup_socket_xprt(void);
  * values. No such restriction exists for new transports, except that
  * they may not collide with these values (17 and 6, respectively).

include/linux/sunrpc/xprtsock.h void cleanup_socket_xprt(void); · line 29

Quoted 10 lines
include/linux/sunrpc/xprtsock.h +7 -2
  */
-#define XPRT_TRANSPORT_UDP   IPPROTO_UDP
-#define XPRT_TRANSPORT_TCP   IPPROTO_TCP
+#define XPRT_TRANSPORT_BC    (1 << 31)
+enum xprt_transports {
+     XPRT_TRANSPORT_UDP      = IPPROTO_UDP,
+     XPRT_TRANSPORT_TCP      = IPPROTO_TCP,
+     XPRT_TRANSPORT_BC_TCP   = IPPROTO_TCP | XPRT_TRANSPORT_BC,
+     XPRT_TRANSPORT_RDMA     = 256
+};

This fails to compile when CONFIG_SUNRPC_XPRT_RDMA is set.

A minimal fix might be:

       --- a/net/sunrpc/xprtrdma/transport.c
       +++ b/net/sunrpc/xprtrdma/transport.c
       @@ -50,6 +50,8 @@
        #include <linux/module.h>
        #include <linux/init.h>
        #include <linux/seq_file.h>
       +#include <linux/in.h>
       +#include <linux/sunrpc/xprtsock.h>

        #include "xprt_rdma.h"

Or maybe just ditch the enum and leave these as they were before.

Quoted 7 lines
Quoted 3 lines

The incentive here was to have all the transports definitions together so that nobody re-uses a number by mistake (not very likely of course), so I think the fix you suggested is appropriate.

OK. Actually, rather than adding a socket-specific include to an rdma-specific file, how about the below? Applying with the following change assuming no objections.

Ack

Benny

Quoted 58 lines

--b.

include/linux/sunrpc/xprt.h +17 -0
index 7cc42af..6f9457a 100644
@@ -124,6 +124,23 @@ struct rpc_xprt_ops {
 	void		(*print_stats)(struct rpc_xprt *xprt, struct seq_file *seq);
 };
 
+/*
+ * RPC transport identifiers
+ *
+ * To preserve compatibility with the historical use of raw IP protocol
+ * id's for transport selection, UDP and TCP identifiers are specified
+ * with the previous values. No such restriction exists for new transports,
+ * except that they may not collide with these values (17 and 6,
+ * respectively).
+ */
+#define XPRT_TRANSPORT_BC       (1 << 31)
+enum xprt_transports {
+	XPRT_TRANSPORT_UDP	= IPPROTO_UDP,
+	XPRT_TRANSPORT_TCP	= IPPROTO_TCP,
+	XPRT_TRANSPORT_BC_TCP	= IPPROTO_TCP | XPRT_TRANSPORT_BC,
+	XPRT_TRANSPORT_RDMA	= 256
+};
+
 struct rpc_xprt {
 	struct kref		kref;		/* Reference count */
 	struct rpc_xprt_ops *	ops;		/* transport methods */
include/linux/sunrpc/xprtsock.h +0 -16
index d7c98d1..3f14a02 100644
@@ -13,22 +13,6 @@ int		init_socket_xprt(void);
 void		cleanup_socket_xprt(void);
 
 /*
- * RPC transport identifiers for UDP, TCP
- *
- * To preserve compatibility with the historical use of raw IP protocol
- * id's for transport selection, these are specified with the previous
- * values. No such restriction exists for new transports, except that
- * they may not collide with these values (17 and 6, respectively).
- */
-#define XPRT_TRANSPORT_BC	(1 << 31)
-enum xprt_transports {
-	XPRT_TRANSPORT_UDP 	= IPPROTO_UDP,
-	XPRT_TRANSPORT_TCP 	= IPPROTO_TCP,
-	XPRT_TRANSPORT_BC_TCP 	= IPPROTO_TCP | XPRT_TRANSPORT_BC,
-	XPRT_TRANSPORT_RDMA	= 256
-};
-
-/*
  * RPC slot table sizes for UDP, TCP transports
  */
 extern unsigned int xprt_udp_slot_table_entries;
#63 lore raw

Re: [PATCH v2 10/12] nfsd41: Backchannel: Implement cb_recall over NFSv4.1

In reply to J. Bruce Fields
Quoted 47 lines · J. Bruce Fields
Quoted 46 lines · Benny Halevy
fs/nfsd/nfs4callback.c +13 -2
@@ -668,16 +701,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -703,16 +739,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
 	int status;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args) {
+		status = -ENOMEM;
+		goto out;
+	}
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
 		put_nfs4_client(clp);
Quoted 8 lines
Quoted 4 lines
fs/nfsd/nfs4callback.c +0 -0
 		nfs4_put_delegation(dp);
Signature · 1 line
1.6.4

Tracing down through rpc_call_async.... It doesn't look to me like it will call rpc_call_done on returning an error, so you're leaking args in that case. Probably just need a kfree(args); applying as below.

Right on the spot. Thanks!
(We could do that in the rpc_release method but then the calldata would have to be a struct nfs4_rpc_args which is somewhat cumbersome...)

Benny

Quoted 157 lines

--b.

commit 18d8774a149c28b8c9ac66592873e78bbc66330a

Author: Ricardo Labiaga <[email protected]>
Date:   Thu Sep 10 12:27:04 2009 +0300

nfsd41: Backchannel: Implement cb_recall over NFSv4.1

[nfsd41: cb_recall callback]
[Share v4.0 and v4.1 back channel xdr]
[Share v4.0 and v4.1 back channel xdr]
[nfsd41: use nfsd4_cb_sequence for callback minorversion]
[nfsd41: conditionally decode_sequence in nfs4_xdr_dec_cb_recall]
[nfsd41: Backchannel: Add sequence arguments to callback RPC arguments]
[pulled-in definition of nfsd4_cb_done]
fs/nfsd/nfs4callback.c +48 -5
index 74c8b33..fabfe12 100644
@@ -267,15 +267,19 @@ nfs4_xdr_enc_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p, struct nfs4_delegation *args)
+nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, __be32 *p,
+		struct nfs4_rpc_args *rpc_args)
 {
 	struct xdr_stream xdr;
+	struct nfs4_delegation *args = rpc_args->args_op;
 	struct nfs4_cb_compound_hdr hdr = {
 		.ident = args->dl_ident,
+		.minorversion = rpc_args->args_seq.cbs_minorversion,
 	};
 
 	xdr_init_encode(&xdr, &req->rq_snd_buf, p);
 	encode_cb_compound_hdr(&xdr, &hdr);
+	encode_cb_sequence(&xdr, &rpc_args->args_seq, &hdr);
 	encode_cb_recall(&xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 	return 0;
@@ -324,7 +328,8 @@ nfs4_xdr_dec_cb_null(struct rpc_rqst *req, __be32 *p)
 }
 
 static int
-nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
+nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p,
+		struct nfsd4_cb_sequence *seq)
 {
 	struct xdr_stream xdr;
 	struct nfs4_cb_compound_hdr hdr;
@@ -334,6 +339,11 @@ nfs4_xdr_dec_cb_recall(struct rpc_rqst *rqstp, __be32 *p)
 	status = decode_cb_compound_hdr(&xdr, &hdr);
 	if (status)
 		goto out;
+	if (seq) {
+		status = decode_cb_sequence(&xdr, seq, rqstp);
+		if (status)
+			goto out;
+	}
 	status = decode_cb_op_hdr(&xdr, OP_CB_RECALL);
 out:
 	return status;
@@ -575,11 +585,34 @@ static void nfsd4_cb_prepare(struct rpc_task *task, void *calldata)
 	rpc_call_start(task);
 }
 
+static void nfsd4_cb_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_delegation *dp = calldata;
+	struct nfs4_client *clp = dp->dl_client;
+
+	dprintk("%s: minorversion=%d\n", __func__,
+		clp->cl_cb_conn.cb_minorversion);
+
+	if (clp->cl_cb_conn.cb_minorversion) {
+		/* No need for lock, access serialized in nfsd4_cb_prepare */
+		++clp->cl_cb_seq_nr;
+		clear_bit(0, &clp->cl_cb_slot_busy);
+		rpc_wake_up_next(&clp->cl_cb_waitq);
+		dprintk("%s: freed slot, new seqid=%d\n", __func__,
+			clp->cl_cb_seq_nr);
+
+		/* We're done looking into the sequence information */
+		task->tk_msg.rpc_resp = NULL;
+	}
+}
+
 static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_delegation *dp = calldata;
 	struct nfs4_client *clp = dp->dl_client;
 
+	nfsd4_cb_done(task, calldata);
+
 	switch (task->tk_status) {
 	case -EIO:
 		/* Network partition? */
@@ -592,16 +625,19 @@ static void nfsd4_cb_recall_done(struct rpc_task *task, void *calldata)
 		break;
 	default:
 		/* success, or error we can't handle */
-		return;
+		goto done;
 	}
 	if (dp->dl_retries--) {
 		rpc_delay(task, 2*HZ);
 		task->tk_status = 0;
 		rpc_restart_call(task);
+		return;
 	} else {
 		atomic_set(&clp->cl_cb_conn.cb_set, 0);
 		warn_no_callback_path(clp, task->tk_status);
 	}
+done:
+	kfree(task->tk_msg.rpc_argp);
 }
 
 static void nfsd4_cb_recall_release(void *calldata)
@@ -627,17 +663,24 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 {
 	struct nfs4_client *clp = dp->dl_client;
 	struct rpc_clnt *clnt = clp->cl_cb_conn.cb_client;
+	struct nfs4_rpc_args *args;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
-		.rpc_argp = dp,
 		.rpc_cred = clp->cl_cb_conn.cb_cred
 	};
-	int status;
+	int status = -ENOMEM;
 
+	args = kzalloc(sizeof(*args), GFP_KERNEL);
+	if (!args)
+		goto out;
+	args->args_op = dp;
+	msg.rpc_argp = args;
 	dp->dl_retries = 1;
 	status = rpc_call_async(clnt, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_recall_ops, dp);
+out:
 	if (status) {
+		kfree(args);
 		put_nfs4_client(clp);
 		nfs4_put_delegation(dp);
 	}
#33 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to Benny Halevy
Quoted 132 lines · Benny Halevy

From: Andy Adamson <[email protected]>

Keep the xprt used for create_session in cl_cb_xprt. Mark cl_callback.cb_minorversion = 1 and remember the client provided cl_callback.cb_prog rpc program number. Use it to probe the callback path.

Use the client's network address to initialize as the callback's address as expected by the xprt creation routines.

Define xdr sizes and code nfs4_cb_compound header to be able to send a null callback rpc.

[get callback minorversion from fore channel's]
[nfsd41: change bc_sock to bc_xprt]
[pulled definition for cl_cb_xprt]
[nfsd41: set up backchannel's cb_addr]
[moved rpc_create_args init to "nfsd: modify nfsd4.1 backchannel to use new xprt class"]
---
 fs/nfsd/nfs4callback.c     |   21 +++++++++++++++++++--
 fs/nfsd/nfs4state.c        |   14 ++++++++++++++
 include/linux/nfsd/state.h |    3 +++
 3 files changed, 36 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +19 -2
index 63bb384..3e3e15b 100644
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -137,11 +148,13 @@ xdr_error:                                      \
 } while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -399,6 +412,10 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
+
 	/* Create RPC client */
 	client = rpc_create(&args);
 	if (IS_ERR(client)) {
fs/nfsd/nfs4state.c +3 -0
index 46e9ac5..e4c3223 100644
@@ -706,6 +706,8 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1321,6 +1323,18 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {

fs/nfsd/nfs4state.c nfsd4_create_session(struct svc_rqst *rqstp, · line 1336

Quoted 10 lines
fs/nfsd/nfs4state.c +10 -0
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			rpc_copy_addr(
+				(struct sockaddr *)&unconf->cl_cb_conn.cb_addr,
+				sa);
+			unconf->cl_cb_conn.cb_addrlen = svc_addr_len(sa);
+			unconf->cl_cb_conn.cb_minorversion =
+				cstate->minorversion;
+			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);

This results in a NULL deference in rpcauth_lookup_credcache()--probably some callback parameters that aren't set up right yet.

--b.

Quoted 21 lines
fs/nfsd/nfs4state.c +1 -0
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
include/linux/nfsd/state.h +3 -0
index 70ef5f4..243277b 100644
@@ -212,6 +212,9 @@ struct nfs4_client {
 	struct nfsd4_clid_slot	cl_cs_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
+
+	/* for nfs41 callbacks */
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#34 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 132 lines · bfields
Quoted 131 lines · Benny Halevy

From: Andy Adamson <[email protected]>

Keep the xprt used for create_session in cl_cb_xprt. Mark cl_callback.cb_minorversion = 1 and remember the client provided cl_callback.cb_prog rpc program number. Use it to probe the callback path.

Use the client's network address to initialize as the callback's address as expected by the xprt creation routines.

Define xdr sizes and code nfs4_cb_compound header to be able to send a null callback rpc.

[get callback minorversion from fore channel's]
[nfsd41: change bc_sock to bc_xprt]
[pulled definition for cl_cb_xprt]
[nfsd41: set up backchannel's cb_addr]
[moved rpc_create_args init to "nfsd: modify nfsd4.1 backchannel to use new xprt class"]
---
 fs/nfsd/nfs4callback.c     |   21 +++++++++++++++++++--
 fs/nfsd/nfs4state.c        |   14 ++++++++++++++
 include/linux/nfsd/state.h |    3 +++
 3 files changed, 36 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +19 -2
index 63bb384..3e3e15b 100644
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -137,11 +148,13 @@ xdr_error:                                      \
 } while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -399,6 +412,10 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
+
 	/* Create RPC client */
 	client = rpc_create(&args);
 	if (IS_ERR(client)) {
fs/nfsd/nfs4state.c +2 -0
index 46e9ac5..e4c3223 100644
@@ -706,6 +706,8 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1321,6 +1323,18 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
Quoted 14 lines

fs/nfsd/nfs4state.c nfsd4_create_session(struct svc_rqst *rqstp, · line 1336

Quoted 11 lines
fs/nfsd/nfs4state.c +11 -0
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			rpc_copy_addr(
+				(struct sockaddr *)&unconf->cl_cb_conn.cb_addr,
+				sa);
+			unconf->cl_cb_conn.cb_addrlen = svc_addr_len(sa);
+			unconf->cl_cb_conn.cb_minorversion =
+				cstate->minorversion;
+			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);

This results in a NULL deference in rpcauth_lookup_credcache()--probably some callback parameters that aren't set up right yet.

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

--b.

Quoted 24 lines

--b.

Quoted 21 lines
fs/nfsd/nfs4state.c +1 -0
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
include/linux/nfsd/state.h +3 -0
index 70ef5f4..243277b 100644
@@ -212,6 +212,9 @@ struct nfs4_client {
 	struct nfsd4_clid_slot	cl_cs_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
+
+	/* for nfs41 callbacks */
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#35 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 133 lines · J. Bruce Fields
Quoted 132 lines · bfields
Quoted 131 lines · Benny Halevy

From: Andy Adamson <[email protected]>

Keep the xprt used for create_session in cl_cb_xprt. Mark cl_callback.cb_minorversion = 1 and remember the client provided cl_callback.cb_prog rpc program number. Use it to probe the callback path.

Use the client's network address to initialize as the callback's address as expected by the xprt creation routines.

Define xdr sizes and code nfs4_cb_compound header to be able to send a null callback rpc.

[get callback minorversion from fore channel's]
[nfsd41: change bc_sock to bc_xprt]
[pulled definition for cl_cb_xprt]
[nfsd41: set up backchannel's cb_addr]
[moved rpc_create_args init to "nfsd: modify nfsd4.1 backchannel to use new xprt class"]
---
 fs/nfsd/nfs4callback.c     |   21 +++++++++++++++++++--
 fs/nfsd/nfs4state.c        |   14 ++++++++++++++
 include/linux/nfsd/state.h |    3 +++
 3 files changed, 36 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +19 -2
index 63bb384..3e3e15b 100644
@@ -43,6 +43,7 @@
 #include <linux/sunrpc/xdr.h>
 #include <linux/sunrpc/svc.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/svcsock.h>
 #include <linux/nfsd/nfsd.h>
 #include <linux/nfsd/state.h>
 #include <linux/sunrpc/sched.h>
@@ -52,16 +53,19 @@
 
 #define NFSPROC4_CB_NULL 0
 #define NFSPROC4_CB_COMPOUND 1
+#define NFS4_STATEID_SIZE 16
 
 /* Index of predefined Linux callback client operations */
 
 enum {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
 enum nfs_cb_opnum4 {
 	OP_CB_RECALL            = 4,
+	OP_CB_SEQUENCE          = 11,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -70,15 +74,22 @@ enum nfs_cb_opnum4 {
 #define NFS4_dec_cb_null_sz		0
 #define cb_compound_enc_hdr_sz		4
 #define cb_compound_dec_hdr_sz		(3 + (NFS4_MAXTAGLEN >> 2))
+#define sessionid_sz			(NFS4_MAX_SESSIONID_LEN >> 2)
+#define cb_sequence_enc_sz		(sessionid_sz + 4 +             \
+					1 /* no referring calls list yet */)
+#define cb_sequence_dec_sz		(op_dec_sz + sessionid_sz + 4)
+
 #define op_enc_sz			1
 #define op_dec_sz			2
 #define enc_nfs4_fh_sz			(1 + (NFS4_FHSIZE >> 2))
 #define enc_stateid_sz			(NFS4_STATEID_SIZE >> 2)
 #define NFS4_enc_cb_recall_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
 					1 + enc_stateid_sz +            \
 					enc_nfs4_fh_sz)
 
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
 					op_dec_sz)
 
 /*
@@ -137,11 +148,13 @@ xdr_error:                                      \
 } while (0)
 
 struct nfs4_cb_compound_hdr {
-	int		status;
-	u32		ident;
+	/* args */
+	u32		ident;	/* minorversion 0 only */
 	u32		nops;
 	__be32		*nops_p;
 	u32		minorversion;
+	/* res */
+	int		status;
 	u32		taglen;
 	char		*tag;
 };
@@ -399,6 +412,10 @@ int setup_callback_client(struct nfs4_client *clp)
 	if (!clp->cl_principal && (clp->cl_flavor >= RPC_AUTH_GSS_KRB5))
 		return -EINVAL;
 
+	dprintk("%s: program %s 0x%x nrvers %u version %u minorversion %u\n",
+		__func__, args.program->name, args.prognumber,
+		args.program->nrvers, args.version, cb->cb_minorversion);
+
 	/* Create RPC client */
 	client = rpc_create(&args);
 	if (IS_ERR(client)) {
fs/nfsd/nfs4state.c +2 -0
index 46e9ac5..e4c3223 100644
@@ -706,6 +706,8 @@ static inline void
 free_client(struct nfs4_client *clp)
 {
 	shutdown_callback_client(clp);
+	if (clp->cl_cb_xprt)
+		svc_xprt_put(clp->cl_cb_xprt);
 	if (clp->cl_cred.cr_group_info)
 		put_group_info(clp->cl_cred.cr_group_info);
 	kfree(clp->cl_principal);
@@ -1321,6 +1323,18 @@ nfsd4_create_session(struct svc_rqst *rqstp,
 		cr_ses->flags &= ~SESSION4_PERSIST;
 		cr_ses->flags &= ~SESSION4_RDMA;
 
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +5 -0
+		if (cr_ses->flags & SESSION4_BACK_CHAN) {
+			unconf->cl_cb_xprt = rqstp->rq_xprt;
+			svc_xprt_get(unconf->cl_cb_xprt);
+			rpc_copy_addr(
+				(struct sockaddr *)&unconf->cl_cb_conn.cb_addr,
Quoted 8 lines

fs/nfsd/nfs4state.c nfsd4_create_session(struct svc_rqst *rqstp, · line 1336

Quoted 6 lines
fs/nfsd/nfs4state.c +6 -0
+				sa);
+			unconf->cl_cb_conn.cb_addrlen = svc_addr_len(sa);
+			unconf->cl_cb_conn.cb_minorversion =
+				cstate->minorversion;
+			unconf->cl_cb_conn.cb_prog = cr_ses->callback_prog;
+			nfsd4_probe_callback(unconf);

This results in a NULL deference in rpcauth_lookup_credcache()--probably some callback parameters that aren't set up right yet.

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Benny

Quoted 26 lines

--b.

Quoted 23 lines

--b.

Quoted 21 lines
fs/nfsd/nfs4state.c +1 -0
+		}
 		conf = unconf;
 	} else {
 		status = nfserr_stale_clientid;
include/linux/nfsd/state.h +3 -0
index 70ef5f4..243277b 100644
@@ -212,6 +212,9 @@ struct nfs4_client {
 	struct nfsd4_clid_slot	cl_cs_slot;	/* create_session slot */
 	u32			cl_exchange_flags;
 	struct nfs4_sessionid	cl_sessionid;
+
+	/* for nfs41 callbacks */
+	struct svc_xprt		*cl_cb_xprt;	/* 4.1 callback transport */
 };
 
 /* struct nfs4_client_reset
Signature · 1 line
1.6.4
#36 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

--b.

#37 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 13 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Quoted 4 lines

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

There shouldn't be a need for an auth_null hashtable. It isn't a credential...

Trond

#39 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to a message not in the archive
Quoted 14 lines · Trond Myklebust
Quoted 13 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Quoted 7 lines
Quoted 4 lines

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

There shouldn't be a need for an auth_null hashtable. It isn't a credential...

Got it. Looking at the code: I guess rpcauth_lookup_credcache is really meant to be used only by auth types implementing their own cred lookups.

Err, maybe I want the following?

Or add a "lookup_machine_cred" helper to auth.c. Or use the auth_generic stuff?

--b.

fs/nfsd/nfs4callback.c +2 -2
index 77a6d10..b91cc8e 100644
@@ -450,6 +450,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	struct auth_cred acred = {
 		.machine_cred = 1
 	};
+	struct rpc_auth *auth = cb->cb_client->cl_auth;
 
 	/*
 	 * Note in the gss case this doesn't actually have to wait for a
@@ -457,8 +458,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	 * non-uptodate cred which the rpc state machine will fill in with
 	 * a refresh_upcall later.
 	 */
-	return rpcauth_lookup_credcache(cb->cb_client->cl_auth, &acred,
-							RPCAUTH_LOOKUP_NEW);
+	return auth->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
 }
 
 void do_probe_callback(struct nfs4_client *clp)
#40 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 46 lines · J. Bruce Fields
Quoted 14 lines · Trond Myklebust
Quoted 13 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Quoted 7 lines
Quoted 4 lines

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

There shouldn't be a need for an auth_null hashtable. It isn't a credential...

Got it. Looking at the code: I guess rpcauth_lookup_credcache is really meant to be used only by auth types implementing their own cred lookups.

Err, maybe I want the following?

Or add a "lookup_machine_cred" helper to auth.c. Or use the auth_generic stuff?

--b.

fs/nfsd/nfs4callback.c +1 -0
index 77a6d10..b91cc8e 100644
@@ -450,6 +450,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	struct auth_cred acred = {
 		.machine_cred = 1
 	};
+	struct rpc_auth *auth = cb->cb_client->cl_auth;
 
 	/*
 	 * Note in the gss case this doesn't actually have to wait for a

fs/nfsd/nfs4callback.c static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb) · line 464

Quoted 9 lines
fs/nfsd/nfs4callback.c +1 -2
@@ -457,8 +458,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	 * non-uptodate cred which the rpc state machine will fill in with
 	 * a refresh_upcall later.
 	 */
-	return rpcauth_lookup_credcache(cb->cb_client->cl_auth, &acred,
-							RPCAUTH_LOOKUP_NEW);
+	return auth->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
 }
 
 void do_probe_callback(struct nfs4_client *clp)

That looks better. There is no guarantee that auth flavours will implement a credcache...

However, is there any reason why you can't just call rpc_lookup_machine_cred()?

#42 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to a message not in the archive
Quoted 55 lines · Trond Myklebust
Quoted 54 lines · J. Bruce Fields
Quoted 14 lines · Trond Myklebust
Quoted 13 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Quoted 7 lines
Quoted 4 lines

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

There shouldn't be a need for an auth_null hashtable. It isn't a credential...

Got it. Looking at the code: I guess rpcauth_lookup_credcache is really meant to be used only by auth types implementing their own cred lookups.

Err, maybe I want the following?

Or add a "lookup_machine_cred" helper to auth.c. Or use the auth_generic stuff?

--b.

fs/nfsd/nfs4callback.c +2 -2
index 77a6d10..b91cc8e 100644
@@ -450,6 +450,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	struct auth_cred acred = {
 		.machine_cred = 1
 	};
+	struct rpc_auth *auth = cb->cb_client->cl_auth;
 
 	/*
 	 * Note in the gss case this doesn't actually have to wait for a
@@ -457,8 +458,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	 * non-uptodate cred which the rpc state machine will fill in with
 	 * a refresh_upcall later.
 	 */
-	return rpcauth_lookup_credcache(cb->cb_client->cl_auth, &acred,
-							RPCAUTH_LOOKUP_NEW);
+	return auth->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
 }
 
Quoted 7 lines

fs/nfsd/nfs4callback.c static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb) · line 464

Quoted 1 line
fs/nfsd/nfs4callback.c +0 -0
 void do_probe_callback(struct nfs4_client *clp)

That looks better. There is no guarantee that auth flavours will implement a credcache...

However, is there any reason why you can't just call rpc_lookup_machine_cred()?

I don't want to sleep; could we add a flags argument to rpc_lookup_machine_cred()?

--b.

#43 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 61 lines · J. Bruce Fields
Quoted 60 lines · Trond Myklebust
Quoted 46 lines · J. Bruce Fields
Quoted 14 lines · Trond Myklebust
Quoted 13 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Where exactly is the NULL deref?

Quoted 3 lines

Note--that's fixed 7 patches later in fsd41: Refactor create_client(), but I don't actually understand how yet.

unconf's cl_flavor initialization was moved in the latter patch from nfsd4_setclientid to create_client so maybe this could be the culprit (though, assuming it is initialized to 0 it will choosing implicitly authnull_ops in rpcauth_create() which _should_ work...)

Quoted 7 lines
Quoted 4 lines

Oog, yes, turns out auth_null doesn't initialize the cred hashtable. So also reproduceable by mounting with "mount -tnfs4 -osec=null", then touching a file. So either we should be using some other interface, or rpcauth_lookupcred should be checking au_credcache, or something.

There shouldn't be a need for an auth_null hashtable. It isn't a credential...

Got it. Looking at the code: I guess rpcauth_lookup_credcache is really meant to be used only by auth types implementing their own cred lookups.

Err, maybe I want the following?

Or add a "lookup_machine_cred" helper to auth.c. Or use the auth_generic stuff?

--b.

fs/nfsd/nfs4callback.c +1 -0
index 77a6d10..b91cc8e 100644
@@ -450,6 +450,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	struct auth_cred acred = {
 		.machine_cred = 1
 	};
+	struct rpc_auth *auth = cb->cb_client->cl_auth;
 
 	/*
 	 * Note in the gss case this doesn't actually have to wait for a

fs/nfsd/nfs4callback.c static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb) · line 464

Quoted 9 lines
fs/nfsd/nfs4callback.c +1 -2
@@ -457,8 +458,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	 * non-uptodate cred which the rpc state machine will fill in with
 	 * a refresh_upcall later.
 	 */
-	return rpcauth_lookup_credcache(cb->cb_client->cl_auth, &acred,
-							RPCAUTH_LOOKUP_NEW);
+	return auth->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
 }
 
 void do_probe_callback(struct nfs4_client *clp)

That looks better. There is no guarantee that auth flavours will implement a credcache...

Quoted 5 lines
Quoted 2 lines

However, is there any reason why you can't just call rpc_lookup_machine_cred()?

I don't want to sleep; could we add a flags argument to rpc_lookup_machine_cred()?

I suppose we could. Alternatively, why not just call it once and for all when you start the NFSv4 server? The one cred will work on all rpc_clients. You just have to remember to bump the reference count before using it in an RPC call...

#45 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to a message not in the archive
Quoted 3 lines · Trond Myklebust

I suppose we could. Alternatively, why not just call it once and for all when you start the NFSv4 server? The one cred will work on all rpc_clients.

(This is correct...)

Quoted 2 lines

You just have to remember to bump the reference count before using it in an RPC call...

Oops! Sorry, this is incorrect. The RPC engine will take care of bumping the ref count for you if necessary...

Cheers
Trond

#47 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to a message not in the archive
Quoted 6 lines · Trond Myklebust
Quoted 3 lines · Trond Myklebust

I suppose we could. Alternatively, why not just call it once and for all when you start the NFSv4 server? The one cred will work on all rpc_clients.

Quoted 7 lines

(This is correct...)

Quoted 2 lines

You just have to remember to bump the reference count before using it in an RPC call...

Oops! Sorry, this is incorrect. The RPC engine will take care of bumping the ref count for you if necessary...

OK, on another look: I create a generic cred with rpc_lookup_machine_cred(). The gss cred then gets created by something like:

	rpc_call_async
	  rpc_run_task
	    rpc_new_task
	      rpc_init_task
	        rpcauth_bindcred
		  cred->cr_ops->crbind == generic_bind_cred
		    gss_auth->au_ops->lookup_cred(auth, acred, 0)
		      rpcauth_lookup_credcache(gss_auth, acred, 0)

But rpcauth_lookup_credcache ends up calling cr_init (gss_cred_init) and waiting synchronously for the gss upcall. I want to avoid that. Should I be setting up the cred differently, or could the rpc task initialization process be tweaked somehow? (Or am I just misunderstanding the code?)

--b.

#48 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

Quoted 28 lines · J. Bruce Fields
Quoted 6 lines · Trond Myklebust
Quoted 3 lines · Trond Myklebust

I suppose we could. Alternatively, why not just call it once and for all when you start the NFSv4 server? The one cred will work on all rpc_clients.

Quoted 7 lines

(This is correct...)

Quoted 2 lines

You just have to remember to bump the reference count before using it in an RPC call...

Oops! Sorry, this is incorrect. The RPC engine will take care of bumping the ref count for you if necessary...

OK, on another look: I create a generic cred with rpc_lookup_machine_cred(). The gss cred then gets created by something like:

	rpc_call_async
	  rpc_run_task
	    rpc_new_task
	      rpc_init_task
	        rpcauth_bindcred
		  cred->cr_ops->crbind == generic_bind_cred
		    gss_auth->au_ops->lookup_cred(auth, acred, 0)
		      rpcauth_lookup_credcache(gss_auth, acred, 0)
Quoted 5 lines

But rpcauth_lookup_credcache ends up calling cr_init (gss_cred_init) and waiting synchronously for the gss upcall. I want to avoid that. Should I be setting up the cred differently, or could the rpc task initialization process be tweaked somehow? (Or am I just misunderstanding the code?)

How about something like the following?

------------------------------------------------------------------------
SUNRPC: Defer the auth_gss upcall when the RPC call is asynchronous

From: Trond Myklebust <[email protected]>

Otherwise, the upcall is going to be synchronous, which may not be what the caller wants...

---

 include/linux/sunrpc/auth.h |    4 ++--
 net/sunrpc/auth.c           |   20 ++++++++++++--------
 net/sunrpc/auth_generic.c   |    4 ++--
 3 files changed, 16 insertions(+), 12 deletions(-)
include/linux/sunrpc/auth.h +2 -2
index 3f63218..996df4d 100644
@@ -111,7 +111,7 @@ struct rpc_credops {
 	void			(*crdestroy)(struct rpc_cred *);
 
 	int			(*crmatch)(struct auth_cred *, struct rpc_cred *, int);
-	void			(*crbind)(struct rpc_task *, struct rpc_cred *);
+	void			(*crbind)(struct rpc_task *, struct rpc_cred *, int);
 	__be32 *		(*crmarshal)(struct rpc_task *, __be32 *);
 	int			(*crrefresh)(struct rpc_task *);
 	__be32 *		(*crvalidate)(struct rpc_task *, __be32 *);
@@ -140,7 +140,7 @@ struct rpc_cred *	rpcauth_lookup_credcache(struct rpc_auth *, struct auth_cred *
 void			rpcauth_init_cred(struct rpc_cred *, const struct auth_cred *, struct rpc_auth *, const struct rpc_credops *);
 struct rpc_cred *	rpcauth_lookupcred(struct rpc_auth *, int);
 void			rpcauth_bindcred(struct rpc_task *, struct rpc_cred *, int);
-void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *);
+void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *, int);
 void			put_rpccred(struct rpc_cred *);
 void			rpcauth_unbindcred(struct rpc_task *);
 __be32 *		rpcauth_marshcred(struct rpc_task *, __be32 *);
net/sunrpc/auth.c +12 -8
index 0c431c2..54a4e04 100644
@@ -385,7 +385,7 @@ rpcauth_init_cred(struct rpc_cred *cred, const struct auth_cred *acred,
 EXPORT_SYMBOL_GPL(rpcauth_init_cred);
 
 void
-rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	task->tk_msg.rpc_cred = get_rpccred(cred);
 	dprintk("RPC: %5u holding %s cred %p\n", task->tk_pid,
@@ -394,7 +394,7 @@ rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
 EXPORT_SYMBOL_GPL(rpcauth_generic_bind_cred);
 
 static void
-rpcauth_bind_root_cred(struct rpc_task *task)
+rpcauth_bind_root_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred acred = {
@@ -405,7 +405,7 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, task->tk_client->cl_auth->au_ops->au_name);
-	ret = auth->au_ops->lookup_cred(auth, &acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, &acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -413,14 +413,14 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 }
 
 static void
-rpcauth_bind_new_cred(struct rpc_task *task)
+rpcauth_bind_new_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct rpc_cred *ret;
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, auth->au_ops->au_name);
-	ret = rpcauth_lookupcred(auth, 0);
+	ret = rpcauth_lookupcred(auth, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -430,12 +430,16 @@ rpcauth_bind_new_cred(struct rpc_task *task)
 void
 rpcauth_bindcred(struct rpc_task *task, struct rpc_cred *cred, int flags)
 {
+	int lookupflags = 0;
+
+	if (flags & RPC_TASK_ASYNC)
+		lookupflags |= RPCAUTH_LOOKUP_NEW;
 	if (cred != NULL)
-		cred->cr_ops->crbind(task, cred);
+		cred->cr_ops->crbind(task, cred, lookupflags);
 	else if (flags & RPC_TASK_ROOTCREDS)
-		rpcauth_bind_root_cred(task);
+		rpcauth_bind_root_cred(task, lookupflags);
 	else
-		rpcauth_bind_new_cred(task);
+		rpcauth_bind_new_cred(task, lookupflags);
 }
 
 void
net/sunrpc/auth_generic.c +2 -2
index 4028502..bf88bf8 100644
@@ -55,13 +55,13 @@ struct rpc_cred *rpc_lookup_machine_cred(void)
 EXPORT_SYMBOL_GPL(rpc_lookup_machine_cred);
 
 static void
-generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred *acred = &container_of(cred, struct generic_cred, gc_base)->acred;
 	struct rpc_cred *ret;
 
-	ret = auth->au_ops->lookup_cred(auth, acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
#50 lore raw

Re: [PATCH v2 05/12] nfsd41: Backchannel: callback infrastructure

In reply to a message not in the archive
Quoted 29 lines · Trond Myklebust
Quoted 28 lines · J. Bruce Fields
Quoted 6 lines · Trond Myklebust
Quoted 3 lines · Trond Myklebust

I suppose we could. Alternatively, why not just call it once and for all when you start the NFSv4 server? The one cred will work on all rpc_clients.

Quoted 7 lines

(This is correct...)

Quoted 2 lines

You just have to remember to bump the reference count before using it in an RPC call...

Oops! Sorry, this is incorrect. The RPC engine will take care of bumping the ref count for you if necessary...

OK, on another look: I create a generic cred with rpc_lookup_machine_cred(). The gss cred then gets created by something like:

	rpc_call_async
	  rpc_run_task
	    rpc_new_task
	      rpc_init_task
	        rpcauth_bindcred
		  cred->cr_ops->crbind == generic_bind_cred
		    gss_auth->au_ops->lookup_cred(auth, acred, 0)
		      rpcauth_lookup_credcache(gss_auth, acred, 0)
Quoted 7 lines
Quoted 5 lines

But rpcauth_lookup_credcache ends up calling cr_init (gss_cred_init) and waiting synchronously for the gss upcall. I want to avoid that. Should I be setting up the cred differently, or could the rpc task initialization process be tweaked somehow? (Or am I just misunderstanding the code?)

How about something like the following?

Looks perfect, thanks! Any objections to my submitting the following 4 patches for 2.6.32? I'm leaving in the minimal bugfix first mainly as something simple and nonobtrusive for stable kernels.

--b.

Quoted 130 lines

------------------------------------------------------------------------
SUNRPC: Defer the auth_gss upcall when the RPC call is asynchronous

From: Trond Myklebust <[email protected]>

Otherwise, the upcall is going to be synchronous, which may not be what the caller wants...

---

 include/linux/sunrpc/auth.h |    4 ++--
 net/sunrpc/auth.c           |   20 ++++++++++++--------
 net/sunrpc/auth_generic.c   |    4 ++--
 3 files changed, 16 insertions(+), 12 deletions(-)
include/linux/sunrpc/auth.h +2 -2
index 3f63218..996df4d 100644
@@ -111,7 +111,7 @@ struct rpc_credops {
 	void			(*crdestroy)(struct rpc_cred *);
 
 	int			(*crmatch)(struct auth_cred *, struct rpc_cred *, int);
-	void			(*crbind)(struct rpc_task *, struct rpc_cred *);
+	void			(*crbind)(struct rpc_task *, struct rpc_cred *, int);
 	__be32 *		(*crmarshal)(struct rpc_task *, __be32 *);
 	int			(*crrefresh)(struct rpc_task *);
 	__be32 *		(*crvalidate)(struct rpc_task *, __be32 *);
@@ -140,7 +140,7 @@ struct rpc_cred *	rpcauth_lookup_credcache(struct rpc_auth *, struct auth_cred *
 void			rpcauth_init_cred(struct rpc_cred *, const struct auth_cred *, struct rpc_auth *, const struct rpc_credops *);
 struct rpc_cred *	rpcauth_lookupcred(struct rpc_auth *, int);
 void			rpcauth_bindcred(struct rpc_task *, struct rpc_cred *, int);
-void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *);
+void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *, int);
 void			put_rpccred(struct rpc_cred *);
 void			rpcauth_unbindcred(struct rpc_task *);
 __be32 *		rpcauth_marshcred(struct rpc_task *, __be32 *);
net/sunrpc/auth.c +12 -8
index 0c431c2..54a4e04 100644
@@ -385,7 +385,7 @@ rpcauth_init_cred(struct rpc_cred *cred, const struct auth_cred *acred,
 EXPORT_SYMBOL_GPL(rpcauth_init_cred);
 
 void
-rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	task->tk_msg.rpc_cred = get_rpccred(cred);
 	dprintk("RPC: %5u holding %s cred %p\n", task->tk_pid,
@@ -394,7 +394,7 @@ rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
 EXPORT_SYMBOL_GPL(rpcauth_generic_bind_cred);
 
 static void
-rpcauth_bind_root_cred(struct rpc_task *task)
+rpcauth_bind_root_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred acred = {
@@ -405,7 +405,7 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, task->tk_client->cl_auth->au_ops->au_name);
-	ret = auth->au_ops->lookup_cred(auth, &acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, &acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -413,14 +413,14 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 }
 
 static void
-rpcauth_bind_new_cred(struct rpc_task *task)
+rpcauth_bind_new_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct rpc_cred *ret;
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, auth->au_ops->au_name);
-	ret = rpcauth_lookupcred(auth, 0);
+	ret = rpcauth_lookupcred(auth, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -430,12 +430,16 @@ rpcauth_bind_new_cred(struct rpc_task *task)
 void
 rpcauth_bindcred(struct rpc_task *task, struct rpc_cred *cred, int flags)
 {
+	int lookupflags = 0;
+
+	if (flags & RPC_TASK_ASYNC)
+		lookupflags |= RPCAUTH_LOOKUP_NEW;
 	if (cred != NULL)
-		cred->cr_ops->crbind(task, cred);
+		cred->cr_ops->crbind(task, cred, lookupflags);
 	else if (flags & RPC_TASK_ROOTCREDS)
-		rpcauth_bind_root_cred(task);
+		rpcauth_bind_root_cred(task, lookupflags);
 	else
-		rpcauth_bind_new_cred(task);
+		rpcauth_bind_new_cred(task, lookupflags);
 }
 
 void
net/sunrpc/auth_generic.c +2 -2
index 4028502..bf88bf8 100644
@@ -55,13 +55,13 @@ struct rpc_cred *rpc_lookup_machine_cred(void)
 EXPORT_SYMBOL_GPL(rpc_lookup_machine_cred);
 
 static void
-generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred *acred = &container_of(cred, struct generic_cred, gc_base)->acred;
 	struct rpc_cred *ret;
 
-	ret = auth->au_ops->lookup_cred(auth, acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
#51 lore raw

PATCH 1/4 nfsd4: fix null dereference creating nfsv4 callback client

On setting up the callback to the client, we attempt to use the same authentication flavor the client did. We find an rpc cred to use by calling rpcauth_lookup_credcache(), which assumes that the given authentication flavor has a credentials cache. However, this is not required to be true--in particular, auth_null does not use one. Instead, we should call the auth's lookup_cred() method.

Without this, a client attempting to mount using nfsv4 and auth_null triggers a null dereference.

---
 fs/nfsd/nfs4callback.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfs4callback.c +2 -2
index 63bb384..4abb882 100644
@@ -437,6 +437,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	struct auth_cred acred = {
 		.machine_cred = 1
 	};
+	struct rpc_auth *auth = cb->cb_client->cl_auth;
 
 	/*
 	 * Note in the gss case this doesn't actually have to wait for a
@@ -444,8 +445,7 @@ static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
 	 * non-uptodate cred which the rpc state machine will fill in with
 	 * a refresh_upcall later.
 	 */
-	return rpcauth_lookup_credcache(cb->cb_client->cl_auth, &acred,
-							RPCAUTH_LOOKUP_NEW);
+	return auth->au_ops->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
 }
 
 void do_probe_callback(struct nfs4_client *clp)
Signature · 1 line
1.6.0.4
#52 lore raw

PATCH 2/4 SUNRPC: Defer the auth_gss upcall when the RPC call is asynchronous

From: Trond Myklebust <[email protected]>

Otherwise, the upcall is going to be synchronous, which may not be what the caller wants...

---
 include/linux/sunrpc/auth.h |    4 ++--
 net/sunrpc/auth.c           |   20 ++++++++++++--------
 net/sunrpc/auth_generic.c   |    4 ++--
 3 files changed, 16 insertions(+), 12 deletions(-)
include/linux/sunrpc/auth.h +2 -2
index 3f63218..996df4d 100644
@@ -111,7 +111,7 @@ struct rpc_credops {
 	void			(*crdestroy)(struct rpc_cred *);
 
 	int			(*crmatch)(struct auth_cred *, struct rpc_cred *, int);
-	void			(*crbind)(struct rpc_task *, struct rpc_cred *);
+	void			(*crbind)(struct rpc_task *, struct rpc_cred *, int);
 	__be32 *		(*crmarshal)(struct rpc_task *, __be32 *);
 	int			(*crrefresh)(struct rpc_task *);
 	__be32 *		(*crvalidate)(struct rpc_task *, __be32 *);
@@ -140,7 +140,7 @@ struct rpc_cred *	rpcauth_lookup_credcache(struct rpc_auth *, struct auth_cred *
 void			rpcauth_init_cred(struct rpc_cred *, const struct auth_cred *, struct rpc_auth *, const struct rpc_credops *);
 struct rpc_cred *	rpcauth_lookupcred(struct rpc_auth *, int);
 void			rpcauth_bindcred(struct rpc_task *, struct rpc_cred *, int);
-void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *);
+void			rpcauth_generic_bind_cred(struct rpc_task *, struct rpc_cred *, int);
 void			put_rpccred(struct rpc_cred *);
 void			rpcauth_unbindcred(struct rpc_task *);
 __be32 *		rpcauth_marshcred(struct rpc_task *, __be32 *);
net/sunrpc/auth.c +12 -8
index 0c431c2..54a4e04 100644
@@ -385,7 +385,7 @@ rpcauth_init_cred(struct rpc_cred *cred, const struct auth_cred *acred,
 EXPORT_SYMBOL_GPL(rpcauth_init_cred);
 
 void
-rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	task->tk_msg.rpc_cred = get_rpccred(cred);
 	dprintk("RPC: %5u holding %s cred %p\n", task->tk_pid,
@@ -394,7 +394,7 @@ rpcauth_generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
 EXPORT_SYMBOL_GPL(rpcauth_generic_bind_cred);
 
 static void
-rpcauth_bind_root_cred(struct rpc_task *task)
+rpcauth_bind_root_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred acred = {
@@ -405,7 +405,7 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, task->tk_client->cl_auth->au_ops->au_name);
-	ret = auth->au_ops->lookup_cred(auth, &acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, &acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -413,14 +413,14 @@ rpcauth_bind_root_cred(struct rpc_task *task)
 }
 
 static void
-rpcauth_bind_new_cred(struct rpc_task *task)
+rpcauth_bind_new_cred(struct rpc_task *task, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct rpc_cred *ret;
 
 	dprintk("RPC: %5u looking up %s cred\n",
 		task->tk_pid, auth->au_ops->au_name);
-	ret = rpcauth_lookupcred(auth, 0);
+	ret = rpcauth_lookupcred(auth, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
@@ -430,12 +430,16 @@ rpcauth_bind_new_cred(struct rpc_task *task)
 void
 rpcauth_bindcred(struct rpc_task *task, struct rpc_cred *cred, int flags)
 {
+	int lookupflags = 0;
+
+	if (flags & RPC_TASK_ASYNC)
+		lookupflags |= RPCAUTH_LOOKUP_NEW;
 	if (cred != NULL)
-		cred->cr_ops->crbind(task, cred);
+		cred->cr_ops->crbind(task, cred, lookupflags);
 	else if (flags & RPC_TASK_ROOTCREDS)
-		rpcauth_bind_root_cred(task);
+		rpcauth_bind_root_cred(task, lookupflags);
 	else
-		rpcauth_bind_new_cred(task);
+		rpcauth_bind_new_cred(task, lookupflags);
 }
 
 void
net/sunrpc/auth_generic.c +2 -2
index 4028502..bf88bf8 100644
@@ -55,13 +55,13 @@ struct rpc_cred *rpc_lookup_machine_cred(void)
 EXPORT_SYMBOL_GPL(rpc_lookup_machine_cred);
 
 static void
-generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred)
+generic_bind_cred(struct rpc_task *task, struct rpc_cred *cred, int lookupflags)
 {
 	struct rpc_auth *auth = task->tk_client->cl_auth;
 	struct auth_cred *acred = &container_of(cred, struct generic_cred, gc_base)->acred;
 	struct rpc_cred *ret;
 
-	ret = auth->au_ops->lookup_cred(auth, acred, 0);
+	ret = auth->au_ops->lookup_cred(auth, acred, lookupflags);
 	if (!IS_ERR(ret))
 		task->tk_msg.rpc_cred = ret;
 	else
Signature · 1 line
1.6.0.4
#53 lore raw

PATCH 3/4 nfsd4: allow nfs4 state startup to fail

The failure here is pretty unlikely, but we should handle it anyway.

---
 fs/nfsd/nfs4state.c       |   17 ++++++++++++-----
 fs/nfsd/nfssvc.c          |    4 +++-
 include/linux/nfsd/nfsd.h |    4 ++--
 3 files changed, 17 insertions(+), 8 deletions(-)
fs/nfsd/nfs4state.c +12 -5
index 46e9ac5..11db40c 100644
@@ -4004,7 +4004,7 @@ set_max_delegations(void)
 
 /* initialization to perform when the nfsd service is started: */
 
-static void
+static int
 __nfs4_state_start(void)
 {
 	unsigned long grace_time;
@@ -4016,19 +4016,26 @@ __nfs4_state_start(void)
 	printk(KERN_INFO "NFSD: starting %ld-second grace period\n",
 	       grace_time/HZ);
 	laundry_wq = create_singlethread_workqueue("nfsd4");
+	if (laundry_wq == NULL)
+		return -ENOMEM;
 	queue_delayed_work(laundry_wq, &laundromat_work, grace_time);
 	set_max_delegations();
+	return 0;
 }
 
-void
+int
 nfs4_state_start(void)
 {
+	int ret;
+
 	if (nfs4_init)
-		return;
+		return 0;
 	nfsd4_load_reboot_recovery_data();
-	__nfs4_state_start();
+	ret = __nfs4_state_start();
+	if (ret)
+		return ret;
 	nfs4_init = 1;
-	return;
+	return 0;
 }
 
 time_t
fs/nfsd/nfssvc.c +3 -1
index 4472449..fcc0010 100644
@@ -411,7 +411,9 @@ nfsd_svc(unsigned short port, int nrservs)
 	error =	nfsd_racache_init(2*nrservs);
 	if (error<0)
 		goto out;
-	nfs4_state_start();
+	error = nfs4_state_start();
+	if (error)
+		goto out;
 
 	nfsd_reset_versions();
 
include/linux/nfsd/nfsd.h +2 -2
index 2812ed5..24fdf89 100644
@@ -166,7 +166,7 @@ extern int nfsd_max_blksize;
 extern unsigned int max_delegations;
 int nfs4_state_init(void);
 void nfsd4_free_slabs(void);
-void nfs4_state_start(void);
+int nfs4_state_start(void);
 void nfs4_state_shutdown(void);
 time_t nfs4_lease_time(void);
 void nfs4_reset_lease(time_t leasetime);
@@ -174,7 +174,7 @@ int nfs4_reset_recoverydir(char *recdir);
 #else
 static inline int nfs4_state_init(void) { return 0; }
 static inline void nfsd4_free_slabs(void) { }
-static inline void nfs4_state_start(void) { }
+static inline int nfs4_state_start(void) { }
 static inline void nfs4_state_shutdown(void) { }
 static inline time_t nfs4_lease_time(void) { return 0; }
 static inline void nfs4_reset_lease(time_t leasetime) { }
Signature · 1 line
1.6.0.4
#54 lore raw

PATCH 4/4 nfsd4: use common rpc_cred for all callbacks

Callbacks are always made using the machine's identity, so we can use a single auth_generic credential shared among callbacks to all clients and let the rpc code take care of the rest.

---
 fs/nfsd/nfs4callback.c     |   33 ++++++++++-----------------------
 fs/nfsd/nfs4state.c        |    6 +-----
 include/linux/nfsd/state.h |    2 +-
 3 files changed, 12 insertions(+), 29 deletions(-)
fs/nfsd/nfs4callback.c +10 -23
index 4abb882..1285197 100644
@@ -432,42 +432,29 @@ static const struct rpc_call_ops nfsd4_cb_probe_ops = {
 	.rpc_call_done = nfsd4_cb_probe_done,
 };
 
-static struct rpc_cred *lookup_cb_cred(struct nfs4_cb_conn *cb)
+static struct rpc_cred *callback_cred;
+
+int set_callback_cred(void)
 {
-	struct auth_cred acred = {
-		.machine_cred = 1
-	};
-	struct rpc_auth *auth = cb->cb_client->cl_auth;
-
-	/*
-	 * Note in the gss case this doesn't actually have to wait for a
-	 * gss upcall (or any calls to the client); this just creates a
-	 * non-uptodate cred which the rpc state machine will fill in with
-	 * a refresh_upcall later.
-	 */
-	return auth->au_ops->lookup_cred(auth, &acred, RPCAUTH_LOOKUP_NEW);
+	callback_cred = rpc_lookup_machine_cred();
+	if (!callback_cred)
+		return -ENOMEM;
+	return 0;
 }
 
+
 void do_probe_callback(struct nfs4_client *clp)
 {
 	struct nfs4_cb_conn *cb = &clp->cl_cb_conn;
 	struct rpc_message msg = {
 		.rpc_proc       = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_NULL],
 		.rpc_argp       = clp,
+		.rpc_cred	= callback_cred
 	};
-	struct rpc_cred *cred;
 	int status;
 
-	cred = lookup_cb_cred(cb);
-	if (IS_ERR(cred)) {
-		status = PTR_ERR(cred);
-		goto out;
-	}
-	cb->cb_cred = cred;
-	msg.rpc_cred = cb->cb_cred;
 	status = rpc_call_async(cb->cb_client, &msg, RPC_TASK_SOFT,
 				&nfsd4_cb_probe_ops, (void *)clp);
-out:
 	if (status) {
 		warn_no_callback_path(clp, status);
 		put_nfs4_client(clp);
@@ -550,7 +537,7 @@ nfsd4_cb_recall(struct nfs4_delegation *dp)
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_RECALL],
 		.rpc_argp = dp,
-		.rpc_cred = clp->cl_cb_conn.cb_cred
+		.rpc_cred = callback_cred
 	};
 	int status;
 
fs/nfsd/nfs4state.c +1 -5
index 11db40c..0445192 100644
@@ -696,10 +696,6 @@ shutdown_callback_client(struct nfs4_client *clp)
 		clp->cl_cb_conn.cb_client = NULL;
 		rpc_shutdown_client(clnt);
 	}
-	if (clp->cl_cb_conn.cb_cred) {
-		put_rpccred(clp->cl_cb_conn.cb_cred);
-		clp->cl_cb_conn.cb_cred = NULL;
-	}
 }
 
 static inline void
@@ -4020,7 +4016,7 @@ __nfs4_state_start(void)
 		return -ENOMEM;
 	queue_delayed_work(laundry_wq, &laundromat_work, grace_time);
 	set_max_delegations();
-	return 0;
+	return set_callback_cred();
 }
 
 int
include/linux/nfsd/state.h +1 -1
index 70ef5f4..9bf3aa8 100644
@@ -89,7 +89,6 @@ struct nfs4_cb_conn {
 	/* RPC client info */
 	atomic_t		cb_set;     /* successful CB_NULL call */
 	struct rpc_clnt *       cb_client;
-	struct rpc_cred	*	cb_cred;
 };
 
 /* Maximum number of slots per session. 160 is useful for long haul TCP */
@@ -362,6 +361,7 @@ extern int nfs4_in_grace(void);
 extern __be32 nfs4_check_open_reclaim(clientid_t *clid);
 extern void put_nfs4_client(struct nfs4_client *clp);
 extern void nfs4_free_stateowner(struct kref *kref);
+extern int set_callback_cred(void);
 extern void nfsd4_probe_callback(struct nfs4_client *clp);
 extern void nfsd4_cb_recall(struct nfs4_delegation *dp);
 extern void nfs4_put_delegation(struct nfs4_delegation *dp);
Signature · 1 line
1.6.0.4
#70 lore raw
In reply to Benny Halevy
Quoted 8 lines · Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

Quoted 6 lines

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

These should all be in my for-2.6.32 branch; any testing would be appreciated to make sure I didn't introduce any problems.

--b.

Quoted 36 lines

Benny

Quoted 31 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#71 lore raw
Quoted 12 lines · J. Bruce Fields
Quoted 11 lines · Benny Halevy

Bruce,

This version incorporates the latest fixes from Alexandros that address Trond's comments:
http://linux-nfs.org/pipermail/pnfs/2009-September/009052.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009053.html
http://linux-nfs.org/pipermail/pnfs/2009-September/009059.html

This version introduces a new xprt class for the nfsv4.1 backchannel with its own setup routine to keep xs_setup_tcp clean.

Quoted 6 lines
Quoted 3 lines

I also cleaned up the patches a little further by removing a bit of dead code and fixing checkpatch whitespace related warnings in Alexandros squashme patches.

These should all be in my for-2.6.32 branch; any testing would be appreciated to make sure I didn't introduce any problems.

Thanks!
I'll sync up with your branch and start pounding on it.

Benny

Quoted 37 lines

--b.

Quoted 34 lines

Benny

Quoted 31 lines · Benny Halevy

Bruce,

Here's the updated patchset implementing the nfs41 backchannel for the nfs server.

Changes from previous version:
- Rebase onto git://git.linux-nfs.org/~bfields/linux.git for-2.6.32

- bc_send_request does not block on the xpt_mutex
but rather uses the rpc_sleep_on to wait on it.

- nfsd4_create_session initializes unconf->cl_cb_conn.cb_addr.

- cosmetic-only changes cleaned up.

[PATCH 01/10] nfsd41: sunrpc: move struct rpc_buffer def into sunrpc.h [PATCH 02/10] nfsd41: sunrpc: Added rpc server-side backchannel handling [PATCH 03/10] nfsd4: fix whitespace in NFSPROC4_CLNT_CB_NULL definition [PATCH 04/10] nfsd41: Backchannel: callback infrastructure
[PATCH 05/10] nfsd41: Backchannel: Add sequence arguments to callback RPC arguments [PATCH 06/10] nfsd41: Backchannel: Server backchannel RPC wait queue [PATCH 07/10] nfsd41: Backchannel: Setup sequence information
[PATCH 08/10] nfsd41: Backchannel: cb_sequence callback
[PATCH 09/10] nfsd41: Backchannel: Implement cb_recall over NFSv4.1 [PATCH 10/10] nfsd41: Refactor create_client()

Benny
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs