Skip to contentSkip to conversation

NFSv4.1 DRC rewrite version 6

PATCH Applied [email protected] 4 participants 23 messages Aug 27, 2009 – Sep 8, 2009 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover NFSv4.1 DRC rewrite version 6

NFSv4.1 DRC Rewrite Version 6

These patches apply against git://linux-nfs.org/~bfields/linux: for-2.6.32 branch and continue to rewrite the NFSv4.1 Sessions DRC. Besides some bug fixes:

1) The bound on the fore channel per-session DRC size are rewritten. Instead of just using a maximum number of slots to bound the size, The server gives the client the number of ca_maxresponsesize_cached slots it requests bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.

This allows clients to tailor a session to usage.
For example, an I/O session (READ/WRITE/COMMIT only) can have a much smaller ca_maxresponsesize_cached (for only WRITE/COMMIt compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

2) the page-based DRC is replaced with a buffer based DRC with each slot table entry (struct nfsd4_slot + cache) allocated separately. This allocation prepares us for slot size re-negotiation via the SEQUENCE operation target and high slot id arguments.

Testing:

NFSv4.1 mount: pynfs tests - including the SEQUENCE replay cache tests. connectathon tests.

NFSv4.0 mount: connectathon tests.

-->Andy

0001-nfsd41-expand-solo-sequence-check.patch 0002-nfsd41-bound-forechannel-drc-size-by-memory-usage.patch 0003-nfsd41-use-session-maxreqs-for-sequence-target-and.patch 0004-nfsd41-replace-nfserr_resource-in-pure-nfs41-respon.patch 0005-nfsd41-replace-page-based-DRC-with-buffer-based-DRC.patch

#1 lore raw

PATCH 1/5 nfsd41: expand solo sequence check

From: Andy Adamson <[email protected]>

Failed sequenece == solo sequence.
Remove redundant sequence operation cache checks.

---
 fs/nfsd/nfs4state.c       |    9 ---------
 include/linux/nfsd/xdr4.h |    2 +-
 2 files changed, 1 insertions(+), 10 deletions(-)
fs/nfsd/nfs4state.c +0 -9
index d2a0524..ddfd36f 100644
@@ -991,16 +991,10 @@ nfsd4_store_cache_entry(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	struct svc_rqst *rqstp = resp->rqstp;
-	struct nfsd4_compoundargs *args = rqstp->rq_argp;
-	struct nfsd4_op *op = &args->ops[resp->opcnt];
 	struct kvec *resv = &rqstp->rq_res.head[0];
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
-	/* Don't cache a failed OP_SEQUENCE. */
-	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
-		return;
-
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
 	entry->ce_opcnt = resp->opcnt;
 	entry->ce_status = resp->cstate.status;
@@ -1490,9 +1484,6 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
 	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
-	/* Always set the cache entry cachethis for solo sequence */
-	if (nfsd4_is_solo_sequence(resp))
-		slot->sl_cache_entry.ce_cachethis = 1;
 
 	cstate->slot = slot;
 	cstate->session = session;
include/linux/nfsd/xdr4.h +1 -1
index 5e4beb0..3f71660 100644
@@ -467,7 +467,7 @@ struct nfsd4_compoundres {
 static inline bool nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
-	return args->opcnt == 1;
+	return resp->opcnt == 1 && args->ops[0].opnum == OP_SEQUENCE;
 }
 
 static inline bool nfsd4_not_cached(struct nfsd4_compoundres *resp)
Signature · 1 line
1.6.2.5
#2 lore raw

PATCH 2/5 nfsd41: bound forechannel drc size by memory usage

From: Andy Adamson <[email protected]>

By using the requested ca_maxresponsesize_cached * ca_maxresponses to bound a forechannel drc request size, clients can tailor a session to usage.

For example, an I/O session (READ/WRITE only) can have a much smaller ca_maxresponsesize_cached (for only WRITE compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

---
 fs/nfsd/nfs4state.c        |   60 +++++++++++++++++++++++++++++++------------
 include/linux/nfsd/state.h |    8 ++++-
 2 files changed, 49 insertions(+), 19 deletions(-)
fs/nfsd/nfs4state.c +43 -17
index ddfd36f..a691139 100644
@@ -414,34 +414,60 @@ gen_sessionid(struct nfsd4_session *ses)
 }
 
 /*
- * Give the client the number of slots it requests bound by
- * NFSD_MAX_SLOTS_PER_SESSION and by nfsd_drc_max_mem.
+ * 32 bytes of RPC header and 44 bytes of sequence operation response
+ * not included in NFSD_SLOT_CACHE_SIZE
+ * */
+#define NFSD_MIN_HDR_SEQ_SZ  (32 + 44)
+
+/*
+ * Give the client the number of ca_maxresponsesize_cached slots it requests
+ * bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by
+ * nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.
+ *
+ * The ca_maxresponssize_cached definition includes the size
+ * of the rpc header with the variable length security flavor credential
+ * plus verifier (32 bytes with AUTH_SYS and NULL verifier)
+ * as well as the encoded SEQUENCE operation response (44 bytes)
+ * which are not included in NFSD_SLOT_CACHE_SIZE.
+ * We err on the side of being a bit small when AUTH_SYS/NULL verifier
+ * is not used.
  *
  * If we run out of reserved DRC memory we should (up to a point) re-negotiate
  * active sessions and reduce their slot usage to make rooom for new
  * connections. For now we just fail the create session.
  */
-static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
+static int set_forechannel_drc_size(struct nfsd4_channel_attrs *fchan)
 {
-	int mem;
+	int mem, size = fchan->maxresp_cached - NFSD_MIN_HDR_SEQ_SZ;
 
-	if (fchan->maxreqs < 1)
+	if (fchan->maxreqs < 1 || size <= 0)
 		return nfserr_inval;
-	else if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
-		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
 
-	mem = fchan->maxreqs * NFSD_SLOT_CACHE_SIZE;
+	if (size > NFSD_SLOT_CACHE_SIZE)
+		size = NFSD_SLOT_CACHE_SIZE;
+
+	/* bound the maxreqs by NFSD_MAX_MEM_PER_SESSION */
+	mem = fchan->maxreqs * size;
+	if (mem > NFSD_MAX_MEM_PER_SESSION) {
+		fchan->maxreqs = NFSD_MAX_MEM_PER_SESSION / size;
+		if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
+			fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
+		mem = fchan->maxreqs * size;
+	}
 
 	spin_lock(&nfsd_drc_lock);
-	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem)
-		mem = ((nfsd_drc_max_mem - nfsd_drc_mem_used) /
-				NFSD_SLOT_CACHE_SIZE) * NFSD_SLOT_CACHE_SIZE;
+	/* bound the total session drc memory ussage */
+	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem) {
+		fchan->maxreqs = (nfsd_drc_max_mem - nfsd_drc_mem_used) / size;
+		mem = fchan->maxreqs * size;
+	}
 	nfsd_drc_mem_used += mem;
 	spin_unlock(&nfsd_drc_lock);
 
-	fchan->maxreqs = mem / NFSD_SLOT_CACHE_SIZE;
 	if (fchan->maxreqs == 0)
-		return nfserr_resource;
+		return nfserr_serverfault;
+
+	fchan->maxresp_cached = size + NFSD_MIN_HDR_SEQ_SZ;
 	return 0;
 }
 
@@ -466,9 +492,6 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 		fchan->maxresp_sz = maxcount;
 	session_fchan->maxresp_sz = fchan->maxresp_sz;
 
-	session_fchan->maxresp_cached = NFSD_SLOT_CACHE_SIZE;
-	fchan->maxresp_cached = session_fchan->maxresp_cached;
-
 	/* Use the client's maxops if possible */
 	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
 		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
@@ -478,9 +501,12 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 	 * recover pages from existing sessions. For now fail session
 	 * creation.
 	 */
-	status = set_forechannel_maxreqs(fchan);
+	status = set_forechannel_drc_size(fchan);
 
+	session_fchan->maxresp_cached = fchan->maxresp_cached;
 	session_fchan->maxreqs = fchan->maxreqs;
+
+	dprintk("%s status %d\n", __func__, status);
 	return status;
 }
 
include/linux/nfsd/state.h +6 -2
index fb0c404..ff0b771 100644
@@ -92,13 +92,17 @@ struct nfs4_cb_conn {
 	struct rpc_cred	*	cb_cred;
 };
 
-/* Maximum number of slots per session. 128 is useful for long haul TCP */
-#define NFSD_MAX_SLOTS_PER_SESSION	128
+/* Maximum number of slots per session. 160 is useful for long haul TCP */
+#define NFSD_MAX_SLOTS_PER_SESSION     160
 /* Maximum number of pages per slot cache entry */
 #define NFSD_PAGES_PER_SLOT	1
 #define NFSD_SLOT_CACHE_SIZE		PAGE_SIZE
 /* Maximum number of operations per session compound */
 #define NFSD_MAX_OPS_PER_COMPOUND	16
+/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
+#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION	32
+#define NFSD_MAX_MEM_PER_SESSION  \
+		(NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NFSD_SLOT_CACHE_SIZE)
 
 struct nfsd4_cache_entry {
 	__be32		ce_status;
Signature · 1 line
1.6.2.5
#3 lore raw

PATCH 3/5 nfsd41: use session maxreqs for sequence target and highest slotid

From: Andy Adamson <[email protected]>

This fixes a bug in the sequence operation reply.

The sequence operation returns the highest slotid it will accept in the future in sr_highest_slotid, and the highest slotid it prefers the client to use. Since we do not re-negotiate the session slot table yet, these should both always be set to the session ca_maxrequests.

---
 fs/nfsd/nfs4state.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
fs/nfsd/nfs4state.c +5 -1
index a691139..ada10ad 100644
@@ -1129,7 +1129,6 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
 	 * session inactivity timer fires and a solo sequence operation
 	 * is sent (lease renewal).
 	 */
-	seq->maxslots = resp->cstate.session->se_fchannel.maxreqs;
 
 	/* Either returns 0 or nfserr_retry_uncached */
 	status = nfsd4_enc_sequence_replay(resp->rqstp->rq_argp, resp);
@@ -1493,6 +1492,11 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	slot = &session->se_slots[seq->slotid];
 	dprintk("%s: slotid %d\n", __func__, seq->slotid);
 
+	/* We do not negotiate the number of slots yet, so set the
+	 * maxslots to the session maxreqs which is used to encode
+	 * sr_highest_slotid and the sr_target_slot id to maxslots */
+	seq->maxslots = session->se_fchannel.maxreqs;
+
 	status = check_slot_seqid(seq->seqid, slot->sl_seqid, slot->sl_inuse);
 	if (status == nfserr_replay_cache) {
 		cstate->slot = slot;
Signature · 1 line
1.6.2.5
#4 lore raw

PATCH 4/5 nfsd41: replace nfserr_resource in pure nfs41 responses

From: Andy Adamson <[email protected]>

nfserr_resource is not a legal error for NFSv4.1. Replace it with nfserr_serverfault for EXCHANGE_ID and CREATE_SESSION processing.

We will also need to map nfserr_resource to other errors in routines shared by NFSv4.0 and NFSv4.1

---
 fs/nfsd/nfs4state.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfs4state.c +2 -2
index ada10ad..4695cec 100644
@@ -515,7 +515,7 @@ alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 		   struct nfsd4_create_session *cses)
 {
 	struct nfsd4_session *new, tmp;
-	int idx, status = nfserr_resource, slotsize;
+	int idx, status = nfserr_serverfault, slotsize;
 
 	memset(&tmp, 0, sizeof(tmp));
 
@@ -1278,7 +1278,7 @@ out_new:
 	/* Normal case */
 	new = create_client(exid->clname, dname);
 	if (new == NULL) {
-		status = nfserr_resource;
+		status = nfserr_serverfault;
 		goto out;
 	}
 
Signature · 1 line
1.6.2.5
#5 lore raw

PATCH 5/5 nfsd41: replace page based DRC with buffer based DRC

From: Andy Adamson <[email protected]>

Use NFSD_SLOT_CACHE_SIZE size buffers for sessions DRC instead of holding nfsd pages in cache.

Connectathon testing has shown that 1024 bytes for encoded compound operation responses past the sequence operation is sufficient, 512 bytes is a little too small. Set NFSD_SLOT_CACHE_SIZE to 1024.

Allocate memory for the session DRC in the CREATE_SESSION operation to guarantee that the memory resource is available for caching responses. Allocate each slot individually in preparation for slot table size negotiation.

Remove struct nfsd4_cache_entry and helper functions for the old page-based DRC.

The iov_len calculation in nfs4svc_encode_compoundres is now always correct, clean up the nfs4svc_encode_compoundres session logic.

The nfsd4_compound_state statp pointer is also not used. Remove nfsd4_set_statp().

Move useful nfsd4_cache_entry fields into nfsd4_slot.

---
 fs/nfsd/nfs4state.c        |  207 ++++++++++++--------------------------------
 fs/nfsd/nfs4xdr.c          |   13 ++--
 fs/nfsd/nfssvc.c           |    4 -
 include/linux/nfsd/state.h |   27 ++----
 include/linux/nfsd/xdr4.h  |    5 +-
 5 files changed, 74 insertions(+), 182 deletions(-)
fs/nfsd/nfs4state.c +56 -151
index 4695cec..2d72d5c 100644
@@ -510,12 +510,22 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 	return status;
 }
 
+static void
+free_session_slots(struct nfsd4_session *ses)
+{
+	int i;
+
+	for (i = 0; i < ses->se_fchannel.maxreqs; i++)
+		kfree(ses->se_slots[i]);
+}
+
 static int
 alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 		   struct nfsd4_create_session *cses)
 {
 	struct nfsd4_session *new, tmp;
-	int idx, status = nfserr_serverfault, slotsize;
+	struct nfsd4_slot *sp;
+	int idx, status = nfserr_serverfault, slotsize, cachesize, i;
 
 	memset(&tmp, 0, sizeof(tmp));
 
@@ -526,14 +536,23 @@ alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 	if (status)
 		goto out;
 
-	/* allocate struct nfsd4_session and slot table in one piece */
-	slotsize = tmp.se_fchannel.maxreqs * sizeof(struct nfsd4_slot);
+	/* allocate struct nfsd4_session and slot table pointers in one piece */
+	slotsize = tmp.se_fchannel.maxreqs * sizeof(struct nfsd4_slot *);
 	new = kzalloc(sizeof(*new) + slotsize, GFP_KERNEL);
 	if (!new)
 		goto out;
 
 	memcpy(new, &tmp, sizeof(*new));
 
+	/* allocate each struct nfsd4_slot and data cache in one piece */
+	cachesize = new->se_fchannel.maxresp_cached - NFSD_MIN_HDR_SEQ_SZ;
+	for (i = 0; i < new->se_fchannel.maxreqs; i++) {
+		sp = kzalloc(sizeof(*sp) + cachesize, GFP_KERNEL);
+		if (!sp)
+			goto out_free;
+		new->se_slots[i] = sp;
+	}
+
 	new->se_client = clp;
 	gen_sessionid(new);
 	idx = hash_sessionid(&new->se_sessionid);
@@ -550,6 +569,10 @@ alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 	status = nfs_ok;
 out:
 	return status;
+out_free:
+	free_session_slots(new);
+	kfree(new);
+	goto out;
 }
 
 /* caller must hold sessionid_lock */
@@ -592,22 +615,16 @@ release_session(struct nfsd4_session *ses)
 	nfsd4_put_session(ses);
 }
 
-static void nfsd4_release_respages(struct page **respages, short resused);
-
 void
 free_session(struct kref *kref)
 {
 	struct nfsd4_session *ses;
-	int i;
 
 	ses = container_of(kref, struct nfsd4_session, se_ref);
-	for (i = 0; i < ses->se_fchannel.maxreqs; i++) {
-		struct nfsd4_cache_entry *e = &ses->se_slots[i].sl_cache_entry;
-		nfsd4_release_respages(e->ce_respages, e->ce_resused);
-	}
 	spin_lock(&nfsd_drc_lock);
 	nfsd_drc_mem_used -= ses->se_fchannel.maxreqs * NFSD_SLOT_CACHE_SIZE;
 	spin_unlock(&nfsd_drc_lock);
+	free_session_slots(ses);
 	kfree(ses);
 }
 
@@ -964,116 +981,32 @@ out_err:
 	return;
 }
 
-void
-nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
-{
-	struct nfsd4_compoundres *resp = rqstp->rq_resp;
-
-	resp->cstate.statp = statp;
-}
-
-/*
- * Dereference the result pages.
- */
-static void
-nfsd4_release_respages(struct page **respages, short resused)
-{
-	int i;
-
-	dprintk("--> %s\n", __func__);
-	for (i = 0; i < resused; i++) {
-		if (!respages[i])
-			continue;
-		put_page(respages[i]);
-		respages[i] = NULL;
-	}
-}
-
-static void
-nfsd4_copy_pages(struct page **topages, struct page **frompages, short count)
-{
-	int i;
-
-	for (i = 0; i < count; i++) {
-		topages[i] = frompages[i];
-		if (!topages[i])
-			continue;
-		get_page(topages[i]);
-	}
-}
-
 /*
- * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing the previous
- * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where the total
- * length of the XDR response is less than se_fmaxresp_cached
- * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used for a
- * of the reply (e.g. readdir).
- *
- * Store the base and length of the rq_req.head[0] page
- * of the NFSv4.1 data, just past the rpc header.
+ * Cache a reply. nfsd4_check_drc_limit() has bounded the cache size.
  */
 void
 nfsd4_store_cache_entry(struct nfsd4_compoundres *resp)
 {
-	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
-	struct svc_rqst *rqstp = resp->rqstp;
-	struct kvec *resv = &rqstp->rq_res.head[0];
-
-	dprintk("--> %s entry %p\n", __func__, entry);
+	struct nfsd4_slot *slot = resp->cstate.slot;
+	unsigned int base;
 
-	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
-	entry->ce_opcnt = resp->opcnt;
-	entry->ce_status = resp->cstate.status;
+	dprintk("--> %s slot %p\n", __func__, slot);
 
-	/*
-	 * Don't need a page to cache just the sequence operation - the slot
-	 * does this for us!
-	 */
+	slot->sl_opcnt = resp->opcnt;
+	slot->sl_status = resp->cstate.status;
 
 	if (nfsd4_not_cached(resp)) {
-		entry->ce_resused = 0;
-		entry->ce_rpchdrlen = 0;
-		dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
-			resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		slot->sl_datalen = 0;
 		return;
 	}
-	entry->ce_resused = rqstp->rq_resused;
-	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
-		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
-	nfsd4_copy_pages(entry->ce_respages, rqstp->rq_respages,
-			 entry->ce_resused);
-	entry->ce_datav.iov_base = resp->cstate.statp;
-	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
-				(char *)page_address(rqstp->rq_respages[0]));
-	/* Current request rpc header length*/
-	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
-				(char *)page_address(rqstp->rq_respages[0]);
-}
-
-/*
- * We keep the rpc header, but take the nfs reply from the replycache.
- */
-static int
-nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
-			struct nfsd4_cache_entry *entry)
-{
-	struct svc_rqst *rqstp = resp->rqstp;
-	struct kvec *resv = &resp->rqstp->rq_res.head[0];
-	int len;
-
-	/* Current request rpc header length*/
-	len = (char *)resp->cstate.statp -
-			(char *)page_address(rqstp->rq_respages[0]);
-	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
-		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
-			entry->ce_datav.iov_len);
-		return 0;
-	}
-	/* copy the cached reply nfsd data past the current rpc header */
-	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,
-		entry->ce_datav.iov_len);
-	resv->iov_len = len + entry->ce_datav.iov_len;
-	return 1;
+	slot->sl_datalen = (char *)resp->p - (char *)resp->cstate.datap;
+	base = (char *)resp->cstate.datap -
+					(char *)resp->xbuf->head[0].iov_base;
+	if (read_bytes_from_xdr_buf(resp->xbuf, base, slot->sl_data,
+				    slot->sl_datalen))
+		printk(KERN_WARNING
+			"nfsd: sessions DRC could not cache compound\n");
+	return;
 }
 
 /*
@@ -1091,14 +1024,14 @@ nfsd4_enc_sequence_replay(struct nfsd4_compoundargs *args,
 	struct nfsd4_slot *slot = resp->cstate.slot;
 
 	dprintk("--> %s resp->opcnt %d cachethis %u \n", __func__,
-		resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		resp->opcnt, resp->cstate.slot->sl_cachethis);
 
 	/* Encode the replayed sequence operation */
 	op = &args->ops[resp->opcnt - 1];
 	nfsd4_encode_operation(resp, op);
 
 	/* Return nfserr_retry_uncached_rep in next operation. */
-	if (args->opcnt > 1 && slot->sl_cache_entry.ce_cachethis == 0) {
+	if (args->opcnt > 1 && slot->sl_cachethis == 0) {
 		op = &args->ops[resp->opcnt++];
 		op->status = nfserr_retry_uncached_rep;
 		nfsd4_encode_operation(resp, op);
@@ -1107,57 +1040,29 @@ nfsd4_enc_sequence_replay(struct nfsd4_compoundargs *args,
 }
 
 /*
- * Keep the first page of the replay. Copy the NFSv4.1 data from the first
- * cached page.  Replace any futher replay pages from the cache.
+ * The sequence operation is not cached because we can use the slot and
+ * session values.
  */
 __be32
 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
 			 struct nfsd4_sequence *seq)
 {
-	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	struct nfsd4_slot *slot = resp->cstate.slot;
 	__be32 status;
 
-	dprintk("--> %s entry %p\n", __func__, entry);
-
-	/*
-	 * If this is just the sequence operation, we did not keep
-	 * a page in the cache entry because we can just use the
-	 * slot info stored in struct nfsd4_sequence that was checked
-	 * against the slot in nfsd4_sequence().
-	 *
-	 * This occurs when seq->cachethis is FALSE, or when the client
-	 * session inactivity timer fires and a solo sequence operation
-	 * is sent (lease renewal).
-	 */
+	dprintk("--> %s slot %p\n", __func__, slot);
 
 	/* Either returns 0 or nfserr_retry_uncached */
 	status = nfsd4_enc_sequence_replay(resp->rqstp->rq_argp, resp);
 	if (status == nfserr_retry_uncached_rep)
 		return status;
 
-	if (!nfsd41_copy_replay_data(resp, entry)) {
-		/*
-		 * Not enough room to use the replay rpc header, send the
-		 * cached header. Release all the allocated result pages.
-		 */
-		svc_free_res_pages(resp->rqstp);
-		nfsd4_copy_pages(resp->rqstp->rq_respages, entry->ce_respages,
-			entry->ce_resused);
-	} else {
-		/* Release all but the first allocated result page */
-
-		resp->rqstp->rq_resused--;
-		svc_free_res_pages(resp->rqstp);
-
-		nfsd4_copy_pages(&resp->rqstp->rq_respages[1],
-				 &entry->ce_respages[1],
-				 entry->ce_resused - 1);
-	}
+	/* The sequence operation has been encoded, cstate->datap set. */
+	memcpy(resp->cstate.datap, slot->sl_data, slot->sl_datalen);
 
-	resp->rqstp->rq_resused = entry->ce_resused;
-	resp->opcnt = entry->ce_opcnt;
-	resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
-	status = entry->ce_status;
+	resp->opcnt = slot->sl_opcnt;
+	resp->p = resp->cstate.datap + XDR_QUADLEN(slot->sl_datalen);
+	status = slot->sl_status;
 
 	return status;
 }
@@ -1489,7 +1394,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	if (seq->slotid >= session->se_fchannel.maxreqs)
 		goto out;
 
-	slot = &session->se_slots[seq->slotid];
+	slot = session->se_slots[seq->slotid];
 	dprintk("%s: slotid %d\n", __func__, seq->slotid);
 
 	/* We do not negotiate the number of slots yet, so set the
@@ -1502,7 +1407,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 		cstate->slot = slot;
 		cstate->session = session;
 		/* Return the cached reply status and set cstate->status
-		 * for nfsd4_svc_encode_compoundres processing */
+		 * for nfsd4_proc_compound processing */
 		status = nfsd4_replay_cache_entry(resp, seq);
 		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
@@ -1513,7 +1418,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	/* Success! bump slot seqid */
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
-	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
+	slot->sl_cachethis = seq->cachethis;
 
 	cstate->slot = slot;
 	cstate->session = session;
fs/nfsd/nfs4xdr.c +6 -7
index fdf632b..49824ea 100644
@@ -3064,6 +3064,7 @@ nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 	WRITE32(0);
 
 	ADJUST_ARGS();
+	resp->cstate.datap = p; /* DRC cache data pointer */
 	return 0;
 }
 
@@ -3166,7 +3167,7 @@ static int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
 		return status;
 
 	session = resp->cstate.session;
-	if (session == NULL || slot->sl_cache_entry.ce_cachethis == 0)
+	if (session == NULL || slot->sl_cachethis == 0)
 		return status;
 
 	if (resp->opcnt >= args->opcnt)
@@ -3291,6 +3292,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	/*
 	 * All that remains is to write the tag and operation count...
 	 */
+	struct nfsd4_compound_state *cs = &resp->cstate;
 	struct kvec *iov;
 	p = resp->tagp;
 	*p++ = htonl(resp->taglen);
@@ -3304,14 +3306,11 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 		iov = &rqstp->rq_res.head[0];
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
-	if (nfsd4_has_session(&resp->cstate)) {
-		if (resp->cstate.status == nfserr_replay_cache &&
-				!nfsd4_not_cached(resp)) {
-			iov->iov_len = resp->cstate.iovlen;
-		} else {
+	if (nfsd4_has_session(cs)) {
+		if (cs->status != nfserr_replay_cache) {
 			nfsd4_store_cache_entry(resp);
 			dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
-			resp->cstate.slot->sl_inuse = 0;
+			resp->cstate.slot->sl_inuse = false;
 		}
 		nfsd4_put_session(resp->cstate.session);
 	}
fs/nfsd/nfssvc.c +0 -4
index d68cd05..944ef01 100644
@@ -576,10 +576,6 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *statp)
 		+ rqstp->rq_res.head[0].iov_len;
 	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
 
-	/* NFSv4.1 DRC requires statp */
-	if (rqstp->rq_vers == 4)
-		nfsd4_set_statp(rqstp, statp);
-
 	/* Now call the procedure handler, and encode NFS status. */
 	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
 	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
include/linux/nfsd/state.h +10 -17
index ff0b771..e745100 100644
@@ -94,30 +94,23 @@ struct nfs4_cb_conn {
 
 /* Maximum number of slots per session. 160 is useful for long haul TCP */
 #define NFSD_MAX_SLOTS_PER_SESSION     160
-/* Maximum number of pages per slot cache entry */
-#define NFSD_PAGES_PER_SLOT	1
-#define NFSD_SLOT_CACHE_SIZE		PAGE_SIZE
 /* Maximum number of operations per session compound */
 #define NFSD_MAX_OPS_PER_COMPOUND	16
+/* Maximum  session per slot cache size */
+#define NFSD_SLOT_CACHE_SIZE		1024
 /* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
 #define NFSD_CACHE_SIZE_SLOTS_PER_SESSION	32
 #define NFSD_MAX_MEM_PER_SESSION  \
 		(NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NFSD_SLOT_CACHE_SIZE)
 
-struct nfsd4_cache_entry {
-	__be32		ce_status;
-	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
-	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
-	int		ce_cachethis;
-	short		ce_resused;
-	int		ce_opcnt;
-	int		ce_rpchdrlen;
-};
-
 struct nfsd4_slot {
-	bool				sl_inuse;
-	u32				sl_seqid;
-	struct nfsd4_cache_entry	sl_cache_entry;
+	bool	sl_inuse;
+	u32	sl_seqid;
+	int	sl_cachethis;
+	int	sl_opcnt;
+	__be32	sl_status;
+	u32	sl_datalen;
+	char	sl_data[];
 };
 
 struct nfsd4_channel_attrs {
@@ -159,7 +152,7 @@ struct nfsd4_session {
 	struct nfs4_sessionid	se_sessionid;
 	struct nfsd4_channel_attrs se_fchannel;
 	struct nfsd4_channel_attrs se_bchannel;
-	struct nfsd4_slot	se_slots[];	/* forward channel slots */
+	struct nfsd4_slot	*se_slots[];	/* forward channel slots */
 };
 
 static inline void
include/linux/nfsd/xdr4.h +2 -3
index 3f71660..73164c2 100644
@@ -51,7 +51,7 @@ struct nfsd4_compound_state {
 	/* For sessions DRC */
 	struct nfsd4_session	*session;
 	struct nfsd4_slot	*slot;
-	__be32			*statp;
+	__be32			*datap;
 	size_t			iovlen;
 	u32			minorversion;
 	u32			status;
@@ -472,8 +472,7 @@ static inline bool nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
 
 static inline bool nfsd4_not_cached(struct nfsd4_compoundres *resp)
 {
-	return !resp->cstate.slot->sl_cache_entry.ce_cachethis ||
-			nfsd4_is_solo_sequence(resp);
+	return !resp->cstate.slot->sl_cachethis || nfsd4_is_solo_sequence(resp);
 }
 
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
Signature · 1 line
1.6.2.5
#6 lore raw

Re: [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

Quoted 48 lines · On Thu, Aug 27, 2009 at 12:07:44PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

Use NFSD_SLOT_CACHE_SIZE size buffers for sessions DRC instead of holding nfsd pages in cache.

Connectathon testing has shown that 1024 bytes for encoded compound operation responses past the sequence operation is sufficient, 512 bytes is a little too small. Set NFSD_SLOT_CACHE_SIZE to 1024.

Allocate memory for the session DRC in the CREATE_SESSION operation to guarantee that the memory resource is available for caching responses. Allocate each slot individually in preparation for slot table size negotiation.

Remove struct nfsd4_cache_entry and helper functions for the old page-based DRC.

The iov_len calculation in nfs4svc_encode_compoundres is now always correct, clean up the nfs4svc_encode_compoundres session logic.

The nfsd4_compound_state statp pointer is also not used. Remove nfsd4_set_statp().

Move useful nfsd4_cache_entry fields into nfsd4_slot.

---
 fs/nfsd/nfs4state.c        |  207 ++++++++++++--------------------------------
 fs/nfsd/nfs4xdr.c          |   13 ++--
 fs/nfsd/nfssvc.c           |    4 -
 include/linux/nfsd/state.h |   27 ++----
 include/linux/nfsd/xdr4.h  |    5 +-
 5 files changed, 74 insertions(+), 182 deletions(-)
fs/nfsd/nfs4state.c +7 -0
index 4695cec..2d72d5c 100644
@@ -510,12 +510,22 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 	return status;
 }
 
+static void
+free_session_slots(struct nfsd4_session *ses)
+{
+	int i;
+
+	for (i = 0; i < ses->se_fchannel.maxreqs; i++)
+		kfree(ses->se_slots[i]);

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_rqst *rqstp, · line 528

Quoted 10 lines
fs/nfsd/nfs4state.c +4 -1
+}
+
 static int
 alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 		   struct nfsd4_create_session *cses)
 {
 	struct nfsd4_session *new, tmp;
-	int idx, status = nfserr_serverfault, slotsize;
+	struct nfsd4_slot *sp;
+	int idx, status = nfserr_serverfault, slotsize, cachesize, i;

Just as a style thing: that list's getting a little long. Could you keep at least "status" on a separate line?

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp, · line 541

Quoted 12 lines
fs/nfsd/nfs4state.c +2 -2
 	memset(&tmp, 0, sizeof(tmp));
 
@@ -526,14 +536,23 @@ alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 	if (status)
 		goto out;
 
-	/* allocate struct nfsd4_session and slot table in one piece */
-	slotsize = tmp.se_fchannel.maxreqs * sizeof(struct nfsd4_slot);
+	/* allocate struct nfsd4_session and slot table pointers in one piece */
+	slotsize = tmp.se_fchannel.maxreqs * sizeof(struct nfsd4_slot *);
 	new = kzalloc(sizeof(*new) + slotsize, GFP_KERNEL);

I think this is OK for now, but maybe stick something like:

BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(struct nfsd4_slot)
+ sizeof(struct nfsd4_session) > PAGE_SIZE);

in state.h just to warn anyone who wants to blindly bump up NFSD_MAX_SLOTS_PER_SESSION. (It's not really forbidden to kmalloc more than a page, but it's also not reliable, and if it becomes necessary then we'd rather find some way to code around it.)

Quoted 170 lines
fs/nfsd/nfs4state.c +21 -103
 	if (!new)
 		goto out;
 
 	memcpy(new, &tmp, sizeof(*new));
 
+	/* allocate each struct nfsd4_slot and data cache in one piece */
+	cachesize = new->se_fchannel.maxresp_cached - NFSD_MIN_HDR_SEQ_SZ;
+	for (i = 0; i < new->se_fchannel.maxreqs; i++) {
+		sp = kzalloc(sizeof(*sp) + cachesize, GFP_KERNEL);
+		if (!sp)
+			goto out_free;
+		new->se_slots[i] = sp;
+	}
+
 	new->se_client = clp;
 	gen_sessionid(new);
 	idx = hash_sessionid(&new->se_sessionid);
@@ -550,6 +569,10 @@ alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *clp,
 	status = nfs_ok;
 out:
 	return status;
+out_free:
+	free_session_slots(new);
+	kfree(new);
+	goto out;
 }
 
 /* caller must hold sessionid_lock */
@@ -592,22 +615,16 @@ release_session(struct nfsd4_session *ses)
 	nfsd4_put_session(ses);
 }
 
-static void nfsd4_release_respages(struct page **respages, short resused);
-
 void
 free_session(struct kref *kref)
 {
 	struct nfsd4_session *ses;
-	int i;
 
 	ses = container_of(kref, struct nfsd4_session, se_ref);
-	for (i = 0; i < ses->se_fchannel.maxreqs; i++) {
-		struct nfsd4_cache_entry *e = &ses->se_slots[i].sl_cache_entry;
-		nfsd4_release_respages(e->ce_respages, e->ce_resused);
-	}
 	spin_lock(&nfsd_drc_lock);
 	nfsd_drc_mem_used -= ses->se_fchannel.maxreqs * NFSD_SLOT_CACHE_SIZE;
 	spin_unlock(&nfsd_drc_lock);
+	free_session_slots(ses);
 	kfree(ses);
 }
 
@@ -964,116 +981,32 @@ out_err:
 	return;
 }
 
-void
-nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
-{
-	struct nfsd4_compoundres *resp = rqstp->rq_resp;
-
-	resp->cstate.statp = statp;
-}
-
-/*
- * Dereference the result pages.
- */
-static void
-nfsd4_release_respages(struct page **respages, short resused)
-{
-	int i;
-
-	dprintk("--> %s\n", __func__);
-	for (i = 0; i < resused; i++) {
-		if (!respages[i])
-			continue;
-		put_page(respages[i]);
-		respages[i] = NULL;
-	}
-}
-
-static void
-nfsd4_copy_pages(struct page **topages, struct page **frompages, short count)
-{
-	int i;
-
-	for (i = 0; i < count; i++) {
-		topages[i] = frompages[i];
-		if (!topages[i])
-			continue;
-		get_page(topages[i]);
-	}
-}
-
 /*
- * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing the previous
- * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where the total
- * length of the XDR response is less than se_fmaxresp_cached
- * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used for a
- * of the reply (e.g. readdir).
- *
- * Store the base and length of the rq_req.head[0] page
- * of the NFSv4.1 data, just past the rpc header.
+ * Cache a reply. nfsd4_check_drc_limit() has bounded the cache size.
  */
 void
 nfsd4_store_cache_entry(struct nfsd4_compoundres *resp)
 {
-	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
-	struct svc_rqst *rqstp = resp->rqstp;
-	struct kvec *resv = &rqstp->rq_res.head[0];
-
-	dprintk("--> %s entry %p\n", __func__, entry);
+	struct nfsd4_slot *slot = resp->cstate.slot;
+	unsigned int base;
 
-	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
-	entry->ce_opcnt = resp->opcnt;
-	entry->ce_status = resp->cstate.status;
+	dprintk("--> %s slot %p\n", __func__, slot);
 
-	/*
-	 * Don't need a page to cache just the sequence operation - the slot
-	 * does this for us!
-	 */
+	slot->sl_opcnt = resp->opcnt;
+	slot->sl_status = resp->cstate.status;
 
 	if (nfsd4_not_cached(resp)) {
-		entry->ce_resused = 0;
-		entry->ce_rpchdrlen = 0;
-		dprintk("%s Just cache SEQUENCE. ce_cachethis %d\n", __func__,
-			resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		slot->sl_datalen = 0;
 		return;
 	}
-	entry->ce_resused = rqstp->rq_resused;
-	if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
-		entry->ce_resused = NFSD_PAGES_PER_SLOT + 1;
-	nfsd4_copy_pages(entry->ce_respages, rqstp->rq_respages,
-			 entry->ce_resused);
-	entry->ce_datav.iov_base = resp->cstate.statp;
-	entry->ce_datav.iov_len = resv->iov_len - ((char *)resp->cstate.statp -
-				(char *)page_address(rqstp->rq_respages[0]));
-	/* Current request rpc header length*/
-	entry->ce_rpchdrlen = (char *)resp->cstate.statp -
-				(char *)page_address(rqstp->rq_respages[0]);
-}
-
-/*
- * We keep the rpc header, but take the nfs reply from the replycache.
- */
-static int
-nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
-			struct nfsd4_cache_entry *entry)
-{
-	struct svc_rqst *rqstp = resp->rqstp;
-	struct kvec *resv = &resp->rqstp->rq_res.head[0];
-	int len;
-
-	/* Current request rpc header length*/
-	len = (char *)resp->cstate.statp -
-			(char *)page_address(rqstp->rq_respages[0]);
-	if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
-		dprintk("%s v41 cached reply too large (%Zd).\n", __func__,
-			entry->ce_datav.iov_len);
-		return 0;
-	}
-	/* copy the cached reply nfsd data past the current rpc header */
-	memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_base,

fs/nfsd/nfs4state.c out_err: · line 1008

Quoted 10 lines
fs/nfsd/nfs4state.c +7 -3
-		entry->ce_datav.iov_len);
-	resv->iov_len = len + entry->ce_datav.iov_len;
-	return 1;
+	slot->sl_datalen = (char *)resp->p - (char *)resp->cstate.datap;
+	base = (char *)resp->cstate.datap -
+					(char *)resp->xbuf->head[0].iov_base;
+	if (read_bytes_from_xdr_buf(resp->xbuf, base, slot->sl_data,
+				    slot->sl_datalen))
+		printk(KERN_WARNING
+			"nfsd: sessions DRC could not cache compound\n");

I'd make this WARN("nfsd:...") just to make it completely clear it's a kernel bug. (This case should be caught by nfsd4_check_drc_limit unless we've messed something up, right?)

Quoted 208 lines
fs/nfsd/nfs4state.c +15 -42
+	return;
 }
 
 /*
@@ -1091,14 +1024,14 @@ nfsd4_enc_sequence_replay(struct nfsd4_compoundargs *args,
 	struct nfsd4_slot *slot = resp->cstate.slot;
 
 	dprintk("--> %s resp->opcnt %d cachethis %u \n", __func__,
-		resp->opcnt, resp->cstate.slot->sl_cache_entry.ce_cachethis);
+		resp->opcnt, resp->cstate.slot->sl_cachethis);
 
 	/* Encode the replayed sequence operation */
 	op = &args->ops[resp->opcnt - 1];
 	nfsd4_encode_operation(resp, op);
 
 	/* Return nfserr_retry_uncached_rep in next operation. */
-	if (args->opcnt > 1 && slot->sl_cache_entry.ce_cachethis == 0) {
+	if (args->opcnt > 1 && slot->sl_cachethis == 0) {
 		op = &args->ops[resp->opcnt++];
 		op->status = nfserr_retry_uncached_rep;
 		nfsd4_encode_operation(resp, op);
@@ -1107,57 +1040,29 @@ nfsd4_enc_sequence_replay(struct nfsd4_compoundargs *args,
 }
 
 /*
- * Keep the first page of the replay. Copy the NFSv4.1 data from the first
- * cached page.  Replace any futher replay pages from the cache.
+ * The sequence operation is not cached because we can use the slot and
+ * session values.
  */
 __be32
 nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
 			 struct nfsd4_sequence *seq)
 {
-	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
+	struct nfsd4_slot *slot = resp->cstate.slot;
 	__be32 status;
 
-	dprintk("--> %s entry %p\n", __func__, entry);
-
-	/*
-	 * If this is just the sequence operation, we did not keep
-	 * a page in the cache entry because we can just use the
-	 * slot info stored in struct nfsd4_sequence that was checked
-	 * against the slot in nfsd4_sequence().
-	 *
-	 * This occurs when seq->cachethis is FALSE, or when the client
-	 * session inactivity timer fires and a solo sequence operation
-	 * is sent (lease renewal).
-	 */
+	dprintk("--> %s slot %p\n", __func__, slot);
 
 	/* Either returns 0 or nfserr_retry_uncached */
 	status = nfsd4_enc_sequence_replay(resp->rqstp->rq_argp, resp);
 	if (status == nfserr_retry_uncached_rep)
 		return status;
 
-	if (!nfsd41_copy_replay_data(resp, entry)) {
-		/*
-		 * Not enough room to use the replay rpc header, send the
-		 * cached header. Release all the allocated result pages.
-		 */
-		svc_free_res_pages(resp->rqstp);
-		nfsd4_copy_pages(resp->rqstp->rq_respages, entry->ce_respages,
-			entry->ce_resused);
-	} else {
-		/* Release all but the first allocated result page */
-
-		resp->rqstp->rq_resused--;
-		svc_free_res_pages(resp->rqstp);
-
-		nfsd4_copy_pages(&resp->rqstp->rq_respages[1],
-				 &entry->ce_respages[1],
-				 entry->ce_resused - 1);
-	}
+	/* The sequence operation has been encoded, cstate->datap set. */
+	memcpy(resp->cstate.datap, slot->sl_data, slot->sl_datalen);
 
-	resp->rqstp->rq_resused = entry->ce_resused;
-	resp->opcnt = entry->ce_opcnt;
-	resp->cstate.iovlen = entry->ce_datav.iov_len + entry->ce_rpchdrlen;
-	status = entry->ce_status;
+	resp->opcnt = slot->sl_opcnt;
+	resp->p = resp->cstate.datap + XDR_QUADLEN(slot->sl_datalen);
+	status = slot->sl_status;
 
 	return status;
 }
@@ -1489,7 +1394,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	if (seq->slotid >= session->se_fchannel.maxreqs)
 		goto out;
 
-	slot = &session->se_slots[seq->slotid];
+	slot = session->se_slots[seq->slotid];
 	dprintk("%s: slotid %d\n", __func__, seq->slotid);
 
 	/* We do not negotiate the number of slots yet, so set the
@@ -1502,7 +1407,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 		cstate->slot = slot;
 		cstate->session = session;
 		/* Return the cached reply status and set cstate->status
-		 * for nfsd4_svc_encode_compoundres processing */
+		 * for nfsd4_proc_compound processing */
 		status = nfsd4_replay_cache_entry(resp, seq);
 		cstate->status = nfserr_replay_cache;
 		goto replay_cache;
@@ -1513,7 +1418,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	/* Success! bump slot seqid */
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
-	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
+	slot->sl_cachethis = seq->cachethis;
 
 	cstate->slot = slot;
 	cstate->session = session;
fs/nfsd/nfs4xdr.c +6 -7
index fdf632b..49824ea 100644
@@ -3064,6 +3064,7 @@ nfsd4_encode_sequence(struct nfsd4_compoundres *resp, int nfserr,
 	WRITE32(0);
 
 	ADJUST_ARGS();
+	resp->cstate.datap = p; /* DRC cache data pointer */
 	return 0;
 }
 
@@ -3166,7 +3167,7 @@ static int nfsd4_check_drc_limit(struct nfsd4_compoundres *resp)
 		return status;
 
 	session = resp->cstate.session;
-	if (session == NULL || slot->sl_cache_entry.ce_cachethis == 0)
+	if (session == NULL || slot->sl_cachethis == 0)
 		return status;
 
 	if (resp->opcnt >= args->opcnt)
@@ -3291,6 +3292,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 	/*
 	 * All that remains is to write the tag and operation count...
 	 */
+	struct nfsd4_compound_state *cs = &resp->cstate;
 	struct kvec *iov;
 	p = resp->tagp;
 	*p++ = htonl(resp->taglen);
@@ -3304,14 +3306,11 @@ nfs4svc_encode_compoundres(struct svc_rqst *rqstp, __be32 *p, struct nfsd4_compo
 		iov = &rqstp->rq_res.head[0];
 	iov->iov_len = ((char*)resp->p) - (char*)iov->iov_base;
 	BUG_ON(iov->iov_len > PAGE_SIZE);
-	if (nfsd4_has_session(&resp->cstate)) {
-		if (resp->cstate.status == nfserr_replay_cache &&
-				!nfsd4_not_cached(resp)) {
-			iov->iov_len = resp->cstate.iovlen;
-		} else {
+	if (nfsd4_has_session(cs)) {
+		if (cs->status != nfserr_replay_cache) {
 			nfsd4_store_cache_entry(resp);
 			dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);
-			resp->cstate.slot->sl_inuse = 0;
+			resp->cstate.slot->sl_inuse = false;
 		}
 		nfsd4_put_session(resp->cstate.session);
 	}
fs/nfsd/nfssvc.c +0 -4
index d68cd05..944ef01 100644
@@ -576,10 +576,6 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *statp)
 		+ rqstp->rq_res.head[0].iov_len;
 	rqstp->rq_res.head[0].iov_len += sizeof(__be32);
 
-	/* NFSv4.1 DRC requires statp */
-	if (rqstp->rq_vers == 4)
-		nfsd4_set_statp(rqstp, statp);
-
 	/* Now call the procedure handler, and encode NFS status. */
 	nfserr = proc->pc_func(rqstp, rqstp->rq_argp, rqstp->rq_resp);
 	nfserr = map_new_errors(rqstp->rq_vers, nfserr);
include/linux/nfsd/state.h +2 -13
index ff0b771..e745100 100644
@@ -94,30 +94,23 @@ struct nfs4_cb_conn {
 
 /* Maximum number of slots per session. 160 is useful for long haul TCP */
 #define NFSD_MAX_SLOTS_PER_SESSION     160
-/* Maximum number of pages per slot cache entry */
-#define NFSD_PAGES_PER_SLOT	1
-#define NFSD_SLOT_CACHE_SIZE		PAGE_SIZE
 /* Maximum number of operations per session compound */
 #define NFSD_MAX_OPS_PER_COMPOUND	16
+/* Maximum  session per slot cache size */
+#define NFSD_SLOT_CACHE_SIZE		1024
 /* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
 #define NFSD_CACHE_SIZE_SLOTS_PER_SESSION	32
 #define NFSD_MAX_MEM_PER_SESSION  \
 		(NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NFSD_SLOT_CACHE_SIZE)
 
-struct nfsd4_cache_entry {
-	__be32		ce_status;
-	struct kvec	ce_datav; /* encoded NFSv4.1 data in rq_res.head[0] */
-	struct page	*ce_respages[NFSD_PAGES_PER_SLOT + 1];
-	int		ce_cachethis;
-	short		ce_resused;
-	int		ce_opcnt;
-	int		ce_rpchdrlen;
-};
-
 struct nfsd4_slot {

include/linux/nfsd/state.h struct nfs4_cb_conn { · line 113

Quoted 10 lines
include/linux/nfsd/state.h +7 -3
-	bool				sl_inuse;
-	u32				sl_seqid;
-	struct nfsd4_cache_entry	sl_cache_entry;
+	bool	sl_inuse;
+	u32	sl_seqid;
+	int	sl_cachethis;
+	int	sl_opcnt;
+	__be32	sl_status;
+	u32	sl_datalen;
+	char	sl_data[];

Could you just move sl_inuse to the end? It'll save a few bytes in the structure (because the compiler will probably stick 3 bytes after it to align sl_seqid.)

--b.

Quoted 38 lines
include/linux/nfsd/state.h +1 -1
 };
 
 struct nfsd4_channel_attrs {
@@ -159,7 +152,7 @@ struct nfsd4_session {
 	struct nfs4_sessionid	se_sessionid;
 	struct nfsd4_channel_attrs se_fchannel;
 	struct nfsd4_channel_attrs se_bchannel;
-	struct nfsd4_slot	se_slots[];	/* forward channel slots */
+	struct nfsd4_slot	*se_slots[];	/* forward channel slots */
 };
 
 static inline void
include/linux/nfsd/xdr4.h +2 -3
index 3f71660..73164c2 100644
@@ -51,7 +51,7 @@ struct nfsd4_compound_state {
 	/* For sessions DRC */
 	struct nfsd4_session	*session;
 	struct nfsd4_slot	*slot;
-	__be32			*statp;
+	__be32			*datap;
 	size_t			iovlen;
 	u32			minorversion;
 	u32			status;
@@ -472,8 +472,7 @@ static inline bool nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
 
 static inline bool nfsd4_not_cached(struct nfsd4_compoundres *resp)
 {
-	return !resp->cstate.slot->sl_cache_entry.ce_cachethis ||
-			nfsd4_is_solo_sequence(resp);
+	return !resp->cstate.slot->sl_cachethis || nfsd4_is_solo_sequence(resp);
 }
 
 #define NFS4_SVC_XDRSIZE		sizeof(struct nfsd4_compoundargs)
Signature · 1 line
1.6.2.5
#7 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

Quoted 6 lines · J. Bruce Fields

I think this is OK for now, but maybe stick something like:

BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(struct nfsd4_slot)
+ sizeof(struct nfsd4_session) > PAGE_SIZE);

in state.h just to warn anyone who wants to blindly bump up

I think that the BUILD_BUG_ON should be placed in a function so it gets to compile, so it'd better be defined here and not in state.h (where it also is in the right context)

Benny

Quoted 5 lines

NFSD_MAX_SLOTS_PER_SESSION. (It's not really forbidden to kmalloc more than a page, but it's also not reliable, and if it becomes necessary then we'd rather find some way to code around it.)

#8 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

Quoted 10 lines · Benny Halevy
Quoted 6 lines · J. Bruce Fields

I think this is OK for now, but maybe stick something like:

BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(struct nfsd4_slot)
+ sizeof(struct nfsd4_session) > PAGE_SIZE);

in state.h just to warn anyone who wants to blindly bump up

I think that the BUILD_BUG_ON should be placed in a function so it gets to compile,

Whoops, thanks--I hadn't noticed that.

Quoted 2 lines

so it'd better be defined here and not in state.h
(where it also is in the right context)

Sounds fine.--b.

#9 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

OK - Thanks for the review. New patches to follow..

-->Andy

Quoted 2 lines · On Mon, Aug 31, 2009 at 9:08 AM, J. Bruce Fields<[email protected]> = wrote:
Quoted 1 line · Benny Halevy

On Aug. 29, 2009, 0:33 +0300, "J. Bruce Fields" <bfields-uC3wQj2KruMpug/[email protected]=

Quoted 3 lines · g> wrote:
Quoted 3 lines
Quoted 3 lines

I think this is OK for now, but maybe stick something like:

=A0 =A0 BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(struct nf=

sd4_slot)

Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 + sizeof(struct nfsd4_sess=

ion) > PAGE_SIZE);

Quoted 17 lines
Quoted 5 lines
Quoted 2 lines

in state.h just to warn anyone who wants to blindly bump up

I think that the BUILD_BUG_ON should be placed in a function so it gets to compile,

Whoops, thanks--I hadn't noticed that.

Quoted 2 lines

so it'd better be defined here and not in state.h
(where it also is in the right context)

Sounds fine.--b.
_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#10 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

In reply to J. Bruce Fields
Quoted 4 lines · On Fri, Aug 28, 2009 at 5:33 PM, J. Bruce Fields<[email protected]> = wrote:
Quoted 3 lines · On Thu, Aug 27, 2009 at 12:07:44PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

Use NFSD_SLOT_CACHE_SIZE size buffers for sessions DRC instead of ho=

lding nfsd

Quoted 3 lines
Quoted 3 lines

pages in cache.

Connectathon testing has shown that 1024 bytes for encoded compound =

operation

Quoted 1 line
Quoted 1 line

responses past the sequence operation is sufficient, 512 bytes is a =

little too

Quoted 4 lines
Quoted 4 lines

small. Set NFSD_SLOT_CACHE_SIZE to 1024.

Allocate memory for the session DRC in the CREATE_SESSION operation to guarantee that the memory resource is available for caching respo=

nses.

Quoted 1 line
Quoted 1 line

Allocate each slot individually in preparation for slot table size n=

egotiation.

Quoted 2 lines
Quoted 2 lines

Remove struct nfsd4_cache_entry and helper functions for the old pag=

e-based

Quoted 5 lines
Quoted 5 lines

DRC.

The iov_len calculation in nfs4svc_encode_compoundres is now always correct, clean up the nfs4svc_encode_compoundres session logic.

Quoted 8 lines
Quoted 8 lines

The nfsd4_compound_state statp pointer is also not used. Remove nfsd4_set_statp().

Move useful nfsd4_cache_entry fields into nfsd4_slot.

---
=A0fs/nfsd/nfs4state.c =A0 =A0 =A0 =A0| =A0207 ++++++++++++---------=

-----------------------

Quoted 11 lines

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_r=

Quoted 11 lines

=A0fs/nfsd/nfs4xdr.c =A0 =A0 =A0 =A0 =A0| =A0 13 ++-- =A0fs/nfsd/nfssvc.c =A0 =A0 =A0 =A0 =A0 | =A0 =A04 - =A0include/linux/nfsd/state.h | =A0 27 ++---- =A0include/linux/nfsd/xdr4.h =A0| =A0 =A05 +- =A05 files changed, 74 insertions(+), 182 deletions(-)

fs/nfsd/nfs4state.c +0 -0
index 4695cec..2d72d5c 100644
@@ -510,12 +510,22 @@ static int init_forechannel_attrs(struct svc_r=

qst *rqstp,

Quoted 14 lines
Quoted 12 lines
fs/nfsd/nfs4state.c +9 -0
 =A0 =A0 =A0 return status;
 =A0}
 
+static void
+free_session_slots(struct nfsd4_session *ses)
+{
+ =A0 =A0 int i;
+
+ =A0 =A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++)
+ =A0 =A0 =A0 =A0 =A0 =A0 kfree(ses->se_slots[i]);
+}
+
Quoted 2 lines

=A0static int
=A0alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client *cl=

p,

Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct nfsd4_create_session *cses=

)

Quoted 5 lines
Quoted 5 lines

=A0{
=A0 =A0 =A0 struct nfsd4_session *new, tmp;
- =A0 =A0 int idx, status =3D nfserr_serverfault, slotsize;
+ =A0 =A0 struct nfsd4_slot *sp;
+ =A0 =A0 int idx, status =3D nfserr_serverfault, slotsize, cachesiz=

e, i;

Quoted 2 lines

Just as a style thing: that list's getting a little long. =A0Could yo=

u

Quoted 6 lines

keep at least "status" on a separate line?

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str=

Quoted 4 lines

=A0 =A0 =A0 memset(&tmp, 0, sizeof(tmp));

fs/nfsd/nfs4state.c +0 -0
@@ -526,14 +536,23 @@ alloc_init_session(struct svc_rqst *rqstp, str=

uct nfs4_client *clp,

Quoted 4 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 529

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -1
 =A0 =A0 =A0 if (status)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
- =A0 =A0 /* allocate struct nfsd4_session and slot table in one pie=

ce */

Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 530

Quoted 1 line
fs/nfsd/nfs4state.c +0 -1
- =A0 =A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(struct nfsd4=

_slot);

Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 539

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 /* allocate struct nfsd4_session and slot table pointers i=

n one piece */

Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 540

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(struct nfsd4=

_slot *);

Quoted 5 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 541

Quoted 1 line
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 new =3D kzalloc(sizeof(*new) + slotsize, GFP_KERNEL);

I think this is OK for now, but maybe stick something like:

=A0 =A0 =A0 =A0BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(struc=

t nfsd4_slot)

Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0+ sizeof(struct nfsd4_=

session) > PAGE_SIZE);

Quoted 3 lines

in state.h just to warn anyone who wants to blindly bump up NFSD_MAX_SLOTS_PER_SESSION. =A0(It's not really forbidden to kmalloc =

more

Quoted 9 lines

than a page, but it's also not reliable, and if it becomes necessary then we'd rather find some way to code around it.)

Quoted 6 lines
fs/nfsd/nfs4state.c +1 -0
 =A0 =A0 =A0 if (!new)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
 =A0 =A0 =A0 memcpy(new, &tmp, sizeof(*new));
 
+ =A0 =A0 /* allocate each struct nfsd4_slot and data cache in one p=

iece */

Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 548

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 cachesize =3D new->se_fchannel.maxresp_cached - NFSD_MIN_H=

DR_SEQ_SZ;

Quoted 2 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, str= · line 550

Quoted 2 lines
fs/nfsd/nfs4state.c +2 -0
+ =A0 =A0 for (i =3D 0; i < new->se_fchannel.maxreqs; i++) {
+ =A0 =A0 =A0 =A0 =A0 =A0 sp =3D kzalloc(sizeof(*sp) + cachesize, GF=

P_KERNEL);

Quoted 9 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, stru=

Quoted 9 lines
fs/nfsd/nfs4state.c +5 -0
+ =A0 =A0 =A0 =A0 =A0 =A0 if (!sp)
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out_free;
+ =A0 =A0 =A0 =A0 =A0 =A0 new->se_slots[i] =3D sp;
+ =A0 =A0 }
+
 =A0 =A0 =A0 new->se_client =3D clp;
 =A0 =A0 =A0 gen_sessionid(new);
 =A0 =A0 =A0 idx =3D hash_sessionid(&new->se_sessionid);
@@ -550,6 +569,10 @@ alloc_init_session(struct svc_rqst *rqstp, stru=

ct nfs4_client *clp,

Quoted 9 lines
Quoted 9 lines
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 status =3D nfs_ok;

=A0out:
=A0 =A0 =A0 return status;
+out_free:
+ =A0 =A0 free_session_slots(new);
+ =A0 =A0 kfree(new);
+ =A0 =A0 goto out;
=A0}

Quoted 6 lines

fs/nfsd/nfs4state.c release_session(struct nfsd4_session *ses) · line 595

Quoted 6 lines

=A0/* caller must hold sessionid_lock */

fs/nfsd/nfs4state.c +0 -1
@@ -592,22 +615,16 @@ release_session(struct nfsd4_session *ses)
 =A0 =A0 =A0 nfsd4_put_session(ses);
 =A0}
 
-static void nfsd4_release_respages(struct page **respages, short re=

sused);

Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfs4state.c +0 -1
-

=A0void
=A0free_session(struct kref *kref)
=A0{
=A0 =A0 =A0 struct nfsd4_session *ses;
- =A0 =A0 int i;

=A0 =A0 =A0 ses =3D container_of(kref, struct nfsd4_session, se_ref)=

;

Quoted 2 lines
Quoted 2 lines

- =A0 =A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 struct nfsd4_cache_entry *e =3D &ses->se_s=

lots[i].sl_cache_entry;

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_release_respages(e->ce_respages, e->=

ce_resused);

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 }
=A0 =A0 =A0 spin_lock(&nfsd_drc_lock);
=A0 =A0 =A0 nfsd_drc_mem_used -=3D ses->se_fchannel.maxreqs * NFSD_S=

LOT_CACHE_SIZE;

Quoted 28 lines
Quoted 28 lines

=A0 =A0 =A0 spin_unlock(&nfsd_drc_lock);
+ =A0 =A0 free_session_slots(ses);
=A0 =A0 =A0 kfree(ses);
=A0}

fs/nfsd/nfs4state.c +0 -19
@@ -964,116 +981,32 @@ out_err:
 =A0 =A0 =A0 return;
 =A0}
 
-void
-nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
-{
- =A0 =A0 struct nfsd4_compoundres *resp =3D rqstp->rq_resp;
-
- =A0 =A0 resp->cstate.statp =3D statp;
-}
-
-/*
- * Dereference the result pages.
- */
-static void
-nfsd4_release_respages(struct page **respages, short resused)
-{
- =A0 =A0 int i;
-
- =A0 =A0 dprintk("--> %s\n", __func__);
- =A0 =A0 for (i =3D 0; i < resused; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 if (!respages[i])
Quoted 8 lines

fs/nfsd/nfs4state.c out_err: · line 993

Quoted 8 lines
fs/nfsd/nfs4state.c +0 -8
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
- =A0 =A0 =A0 =A0 =A0 =A0 put_page(respages[i]);
- =A0 =A0 =A0 =A0 =A0 =A0 respages[i] =3D NULL;
- =A0 =A0 }
-}
-
-static void
-nfsd4_copy_pages(struct page **topages, struct page **frompages, sh=

ort count)

Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +0 -5
-{
- =A0 =A0 int i;
-
- =A0 =A0 for (i =3D 0; i < count; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 topages[i] =3D frompages[i];
Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfs4state.c +0 -6
- =A0 =A0 =A0 =A0 =A0 =A0 if (!topages[i])
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
- =A0 =A0 =A0 =A0 =A0 =A0 get_page(topages[i]);
- =A0 =A0 }
-}
-

=A0/*
- * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing th=

e previous

Quoted 1 line
Quoted 1 line

- * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where t=

he total

Quoted 2 lines
Quoted 2 lines

- * length of the XDR response is less than se_fmaxresp_cached
- * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is used =

for a

Quoted 5 lines
Quoted 5 lines

- * of the reply (e.g. readdir).
- *
- * Store the base and length of the rq_req.head[0] page
- * of the NFSv4.1 data, just past the rpc header.
+ * Cache a reply. nfsd4_check_drc_limit() has bounded the cache siz=

e.

Quoted 5 lines
Quoted 5 lines

=A0 */
=A0void
=A0nfsd4_store_cache_entry(struct nfsd4_compoundres *resp) =A0{
- =A0 =A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.slot->sl=

_cache_entry;

Quoted 8 lines
Quoted 8 lines

- =A0 =A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =A0 =A0 struct kvec *resv =3D &rqstp->rq_res.head[0]; -
- =A0 =A0 dprintk("--> %s entry %p\n", __func__, entry);
+ =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot;
+ =A0 =A0 unsigned int base;

- =A0 =A0 nfsd4_release_respages(entry->ce_respages, entry->ce_resus=

ed);

Quoted 6 lines
Quoted 6 lines

- =A0 =A0 entry->ce_opcnt =3D resp->opcnt;
- =A0 =A0 entry->ce_status =3D resp->cstate.status;
+ =A0 =A0 dprintk("--> %s slot %p\n", __func__, slot);

- =A0 =A0 /*
- =A0 =A0 =A0* Don't need a page to cache just the sequence operatio=

n - the slot

Quoted 9 lines
Quoted 9 lines

- =A0 =A0 =A0* does this for us!
- =A0 =A0 =A0*/
+ =A0 =A0 slot->sl_opcnt =3D resp->opcnt;
+ =A0 =A0 slot->sl_status =3D resp->cstate.status;

=A0 =A0 =A0 if (nfsd4_not_cached(resp)) {
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused =3D 0;
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_rpchdrlen =3D 0;
- =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s Just cache SEQUENCE. ce_cachet=

his %d\n", __func__,

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_cach=

e_entry.ce_cachethis);

Quoted 6 lines
Quoted 6 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 slot->sl_datalen =3D 0;
=A0 =A0 =A0 =A0 =A0 =A0 =A0 return;
=A0 =A0 =A0 }
- =A0 =A0 entry->ce_resused =3D rqstp->rq_resused;
- =A0 =A0 if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused =3D NFSD_PAGES_PER_SLOT =

+ 1;

Quoted 6 lines
Quoted 4 lines

- =A0 =A0 nfsd4_copy_pages(entry->ce_respages, rqstp->rq_respages,
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0entry->ce_resused);
- =A0 =A0 entry->ce_datav.iov_base =3D resp->cstate.statp;
- =A0 =A0 entry->ce_datav.iov_len =3D resv->iov_len - ((char *)resp-=

cstate.statp -

Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *)pa=

ge_address(rqstp->rq_respages[0]));

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 /* Current request rpc header length*/
- =A0 =A0 entry->ce_rpchdrlen =3D (char *)resp->cstate.statp -
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *)pa=

ge_address(rqstp->rq_respages[0]);

Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -4
-}
-
-/*
- * We keep the rpc header, but take the nfs reply from the replycac=

he.

Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines

- */

fs/nfsd/nfs4state.c +0 -3
-static int
-nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 struct nfsd4_cache_entry *=

entry)

Quoted 8 lines

fs/nfsd/nfs4state.c

Quoted 8 lines
fs/nfsd/nfs4state.c +0 -8
-{
- =A0 =A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =A0 =A0 struct kvec *resv =3D &resp->rqstp->rq_res.head[0];
- =A0 =A0 int len;
-
- =A0 =A0 /* Current request rpc header length*/
- =A0 =A0 len =3D (char *)resp->cstate.statp -
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *)page_address(rqstp=

->rq_respages[0]);

Quoted 2 lines
Quoted 2 lines

- =A0 =A0 if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
- =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s v41 cached reply too large (%Z=

d).\n", __func__,

Quoted 4 lines
Quoted 4 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_datav.iov_len);
- =A0 =A0 =A0 =A0 =A0 =A0 return 0;
- =A0 =A0 }
- =A0 =A0 /* copy the cached reply nfsd data past the current rpc he=

ader */

Quoted 1 line
Quoted 1 line

- =A0 =A0 memcpy((char *)resv->iov_base + len, entry->ce_datav.iov_b=

ase,

Quoted 4 lines
Quoted 4 lines

- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_datav.iov_len);
- =A0 =A0 resv->iov_len =3D len + entry->ce_datav.iov_len;
- =A0 =A0 return 1;
+ =A0 =A0 slot->sl_datalen =3D (char *)resp->p - (char *)resp->cstat=

e.datap;

Quoted 2 lines
Quoted 2 lines

+ =A0 =A0 base =3D (char *)resp->cstate.datap -
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0 (char *)resp->xbuf->head[0].iov_base;

Quoted 1 line
Quoted 1 line

+ =A0 =A0 if (read_bytes_from_xdr_buf(resp->xbuf, base, slot->sl_dat=

a,

Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sl=

ot->sl_datalen))

Quoted 2 lines
Quoted 2 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 printk(KERN_WARNING
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 "nfsd: sessions DRC could =

not cache compound\n");

Quoted 2 lines

I'd make this WARN("nfsd:...") just to make it completely clear it's =

a

Quoted 1 line

kernel bug. =A0(This case should be caught by nfsd4_check_drc_limit u=

nless

Quoted 7 lines

we've messed something up, right?)

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_compo=

Quoted 5 lines

+ =A0 =A0 return;
=A0}

=A0/*

fs/nfsd/nfs4state.c +0 -0
@@ -1091,14 +1024,14 @@ nfsd4_enc_sequence_replay(struct nfsd4_compo=

undargs *args,

Quoted 3 lines
Quoted 3 lines
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot;
 

=A0 =A0 =A0 dprintk("--> %s resp->opcnt %d cachethis %u \n", __func_=

_,

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 resp->opcnt, resp->cstate.slot->sl_cache_e=

ntry.ce_cachethis);

Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 resp->opcnt, resp->cstate.slot->sl_cacheth=

is);

Quoted 6 lines
Quoted 6 lines

=A0 =A0 =A0 /* Encode the replayed sequence operation */ =A0 =A0 =A0 op =3D &args->ops[resp->opcnt - 1]; =A0 =A0 =A0 nfsd4_encode_operation(resp, op);

=A0 =A0 =A0 /* Return nfserr_retry_uncached_rep in next operation. *=

/

Quoted 1 line
Quoted 1 line

- =A0 =A0 if (args->opcnt > 1 && slot->sl_cache_entry.ce_cachethis =3D=

=3D 0) {

Quoted 3 lines
Quoted 3 lines

+ =A0 =A0 if (args->opcnt > 1 && slot->sl_cachethis =3D=3D 0) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 op =3D &args->ops[resp->opcnt++]; =A0 =A0 =A0 =A0 =A0 =A0 =A0 op->status =3D nfserr_retry_uncached_rep=

;

Quoted 2 lines

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_compo=

Quoted 2 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_encode_operation(resp, op);

fs/nfsd/nfs4state.c +0 -0
@@ -1107,57 +1040,29 @@ nfsd4_enc_sequence_replay(struct nfsd4_compo=

undargs *args,

Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +0 -0
 =A0}
 

=A0/*
- * Keep the first page of the replay. Copy the NFSv4.1 data from th=

e first

Quoted 2 lines
Quoted 2 lines

- * cached page. =A0Replace any futher replay pages from the cache.
+ * The sequence operation is not cached because we can use the slot=

and

Quoted 5 lines
Quoted 5 lines

+ * session values.
=A0 */
=A0__be32
=A0nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct nfsd4_sequence=

*seq)

Quoted 2 lines
Quoted 2 lines

=A0{
- =A0 =A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.slot->sl=

_cache_entry;

Quoted 7 lines
Quoted 7 lines

+ =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot; =A0 =A0 =A0 __be32 status;

- =A0 =A0 dprintk("--> %s entry %p\n", __func__, entry);
-
- =A0 =A0 /*
- =A0 =A0 =A0* If this is just the sequence operation, we did not ke=

ep

Quoted 2 lines
Quoted 2 lines

- =A0 =A0 =A0* a page in the cache entry because we can just use the
- =A0 =A0 =A0* slot info stored in struct nfsd4_sequence that was ch=

ecked

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0* against the slot in nfsd4_sequence().
- =A0 =A0 =A0*
- =A0 =A0 =A0* This occurs when seq->cachethis is FALSE, or when the=

client

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0* session inactivity timer fires and a solo sequence op=

eration

Quoted 6 lines
Quoted 6 lines

- =A0 =A0 =A0* is sent (lease renewal).
- =A0 =A0 =A0*/
+ =A0 =A0 dprintk("--> %s slot %p\n", __func__, slot);

=A0 =A0 =A0 /* Either returns 0 or nfserr_retry_uncached */ =A0 =A0 =A0 status =3D nfsd4_enc_sequence_replay(resp->rqstp->rq_arg=

p, resp);

Quoted 6 lines
Quoted 6 lines

=A0 =A0 =A0 if (status =3D=3D nfserr_retry_uncached_rep) =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;

- =A0 =A0 if (!nfsd41_copy_replay_data(resp, entry)) {
- =A0 =A0 =A0 =A0 =A0 =A0 /*
- =A0 =A0 =A0 =A0 =A0 =A0 =A0* Not enough room to use the replay rpc=

header, send the

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0* cached header. Release all the alloca=

ted result pages.

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0*/
- =A0 =A0 =A0 =A0 =A0 =A0 svc_free_res_pages(resp->rqstp);
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_copy_pages(resp->rqstp->rq_respages,=

entry->ce_respages,

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused);
- =A0 =A0 } else {
- =A0 =A0 =A0 =A0 =A0 =A0 /* Release all but the first allocated res=

ult page */

Quoted 5 lines

fs/nfsd/nfs4state.c

Quoted 5 lines
fs/nfsd/nfs4state.c +0 -5
-
- =A0 =A0 =A0 =A0 =A0 =A0 resp->rqstp->rq_resused--;
- =A0 =A0 =A0 =A0 =A0 =A0 svc_free_res_pages(resp->rqstp);
-
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_copy_pages(&resp->rqstp->rq_respages=

[1],

Quoted 3 lines
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&entry-=

ce_respages[1],

Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0entry->=

ce_resused - 1);

Quoted 2 lines
Quoted 2 lines

- =A0 =A0 }
+ =A0 =A0 /* The sequence operation has been encoded, cstate->datap =

set. */

Quoted 1 line
Quoted 1 line

+ =A0 =A0 memcpy(resp->cstate.datap, slot->sl_data, slot->sl_datalen=

);

Quoted 4 lines
Quoted 4 lines

- =A0 =A0 resp->rqstp->rq_resused =3D entry->ce_resused;
- =A0 =A0 resp->opcnt =3D entry->ce_opcnt;
- =A0 =A0 resp->cstate.iovlen =3D entry->ce_datav.iov_len + entry->c=

e_rpchdrlen;

Quoted 3 lines
Quoted 3 lines

- =A0 =A0 status =3D entry->ce_status;
+ =A0 =A0 resp->opcnt =3D slot->sl_opcnt;
+ =A0 =A0 resp->p =3D resp->cstate.datap + XDR_QUADLEN(slot->sl_data=

len);

Quoted 8 lines
Quoted 8 lines

+ =A0 =A0 status =3D slot->sl_status;

=A0 =A0 =A0 return status;
=A0}

fs/nfsd/nfs4state.c +0 -0
@@ -1489,7 +1394,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 =A0 =A0 =A0 if (seq->slotid >=3D session->se_fchannel.maxreqs)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +1 -1
- =A0 =A0 slot =3D &session->se_slots[seq->slotid];
+ =A0 =A0 slot =3D session->se_slots[seq->slotid];
 =A0 =A0 =A0 dprintk("%s: slotid %d\n", __func__, seq->slotid);
 

=A0 =A0 =A0 /* We do not negotiate the number of slots yet, so set t=

he

Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +0 -0
@@ -1502,7 +1407,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->slot =3D slot;
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->session =3D session;

=A0 =A0 =A0 =A0 =A0 =A0 =A0 /* Return the cached reply status and se=

t cstate->status

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0* for nfsd4_svc_encode_compoundres proc=

essing */

Quoted 2 lines
Quoted 2 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0* for nfsd4_proc_compound processing */ =A0 =A0 =A0 =A0 =A0 =A0 =A0 status =3D nfsd4_replay_cache_entry(resp=

, seq);

Quoted 9 lines
Quoted 9 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->status =3D nfserr_replay_cache; =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto replay_cache;

fs/nfsd/nfs4state.c +0 -0
@@ -1513,7 +1418,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,

=A0 =A0 =A0 /* Success! bump slot seqid */
=A0 =A0 =A0 slot->sl_inuse =3D true;
=A0 =A0 =A0 slot->sl_seqid =3D seq->seqid;
- =A0 =A0 slot->sl_cache_entry.ce_cachethis =3D seq->cachethis;
+ =A0 =A0 slot->sl_cachethis =3D seq->cachethis;

Quoted 7 lines

fs/nfsd/nfs4xdr.c nfsd4_encode_sequence(struct nfsd4_compoundres=

Quoted 7 lines

=A0 =A0 =A0 cstate->slot =3D slot;
=A0 =A0 =A0 cstate->session =3D session;

fs/nfsd/nfs4xdr.c +0 -0
index fdf632b..49824ea 100644
@@ -3064,6 +3064,7 @@ nfsd4_encode_sequence(struct nfsd4_compoundres=

*resp, int nfserr,

Quoted 8 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfsd4_=

Quoted 8 lines
fs/nfsd/nfs4xdr.c +1 -0
 =A0 =A0 =A0 WRITE32(0);
 
 =A0 =A0 =A0 ADJUST_ARGS();
+ =A0 =A0 resp->cstate.datap =3D p; /* DRC cache data pointer */
 =A0 =A0 =A0 return 0;
 =A0}
 
@@ -3166,7 +3167,7 @@ static int nfsd4_check_drc_limit(struct nfsd4_=

compoundres *resp)

Quoted 4 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfsd4_= · line 3169

Quoted 4 lines
fs/nfsd/nfs4xdr.c +0 -1
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;
 
 =A0 =A0 =A0 session =3D resp->cstate.session;
- =A0 =A0 if (session =3D=3D NULL || slot->sl_cache_entry.ce_cacheth=

is =3D=3D 0)

Quoted 5 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqst *rq=

Quoted 5 lines
fs/nfsd/nfs4xdr.c +1 -0
+ =A0 =A0 if (session =3D=3D NULL || slot->sl_cachethis =3D=3D 0)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;
 
 =A0 =A0 =A0 if (resp->opcnt >=3D args->opcnt)
@@ -3291,6 +3292,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *rq=

stp, __be32 *p, struct nfsd4_compo

Quoted 2 lines
Quoted 2 lines

=A0 =A0 =A0 /*
=A0 =A0 =A0 =A0* All that remains is to write the tag and operation =

count...

Quoted 6 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqst *=

Quoted 6 lines

=A0 =A0 =A0 =A0*/
+ =A0 =A0 struct nfsd4_compound_state *cs =3D &resp->cstate; =A0 =A0 =A0 struct kvec *iov;
=A0 =A0 =A0 p =3D resp->tagp;
=A0 =A0 =A0 *p++ =3D htonl(resp->taglen);

fs/nfsd/nfs4xdr.c +0 -0
@@ -3304,14 +3306,11 @@ nfs4svc_encode_compoundres(struct svc_rqst *=

rqstp, __be32 *p, struct nfsd4_compo

Quoted 2 lines
Quoted 2 lines
fs/nfsd/nfs4xdr.c +0 -0
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iov =3D &rqstp->rq_res.head[0];

=A0 =A0 =A0 iov->iov_len =3D ((char*)resp->p) - (char*)iov->iov_base=

;

Quoted 3 lines
Quoted 3 lines

=A0 =A0 =A0 BUG_ON(iov->iov_len > PAGE_SIZE);
- =A0 =A0 if (nfsd4_has_session(&resp->cstate)) {
- =A0 =A0 =A0 =A0 =A0 =A0 if (resp->cstate.status =3D=3D nfserr_repl=

ay_cache &&

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 !nfsd4_not=

_cached(resp)) {

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iov->iov_len =3D resp->cst=

ate.iovlen;

Quoted 4 lines
Quoted 4 lines

- =A0 =A0 =A0 =A0 =A0 =A0 } else {
+ =A0 =A0 if (nfsd4_has_session(cs)) {
+ =A0 =A0 =A0 =A0 =A0 =A0 if (cs->status !=3D nfserr_replay_cache) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_store_cache_entry(=

resp);

Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s: SET SLOT ST=

ATE TO AVAILABLE\n", __func__);

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_inus=

e =3D 0;

Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_inus=

e =3D false;

Quoted 8 lines

fs/nfsd/nfssvc.c nfsd_dispatch(struct svc_rqst *rqstp, __be32 *s=

Quoted 8 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 }
=A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_put_session(resp->cstate.session); =A0 =A0 =A0 }

fs/nfsd/nfssvc.c +0 -0
index d68cd05..944ef01 100644
@@ -576,10 +576,6 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 *s=

tatp)

Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfssvc.c +0 -4
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 + rqstp->rq_res.head[0].iov_len;
 =A0 =A0 =A0 rqstp->rq_res.head[0].iov_len +=3D sizeof(__be32);
 
- =A0 =A0 /* NFSv4.1 DRC requires statp */
- =A0 =A0 if (rqstp->rq_vers =3D=3D 4)
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_set_statp(rqstp, statp);
-

=A0 =A0 =A0 /* Now call the procedure handler, and encode NFS status=

=2E */

Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 nfserr =3D proc->pc_func(rqstp, rqstp->rq_argp, rqstp->r=

q_resp);

Quoted 8 lines
Quoted 8 lines

=A0 =A0 =A0 nfserr =3D map_new_errors(rqstp->rq_vers, nfserr);

include/linux/nfsd/state.h +0 -0
index ff0b771..e745100 100644
@@ -94,30 +94,23 @@ struct nfs4_cb_conn {
 

=A0/* Maximum number of slots per session. 160 is useful for long ha=

ul TCP */

Quoted 4 lines
Quoted 4 lines

=A0#define NFSD_MAX_SLOTS_PER_SESSION =A0 =A0 160

include/linux/nfsd/state.h +0 -3
-/* Maximum number of pages per slot cache entry */
-#define NFSD_PAGES_PER_SLOT =A01
-#define NFSD_SLOT_CACHE_SIZE =A0 =A0 =A0 =A0 PAGE_SIZE
Quoted 8 lines
Quoted 8 lines

=A0/* Maximum number of operations per session compound */ =A0#define NFSD_MAX_OPS_PER_COMPOUND =A0 =A016

include/linux/nfsd/state.h +2 -0
+/* Maximum =A0session per slot cache size */
+#define NFSD_SLOT_CACHE_SIZE =A0 =A0 =A0 =A0 1024

=A0/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */ =A0#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION =A0 =A032 =A0#define NFSD_MAX_MEM_PER_SESSION =A0\
=A0 =A0 =A0 =A0 =A0 =A0 =A0 (NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NFS=

D_SLOT_CACHE_SIZE)

Quoted 4 lines
Quoted 4 lines

-struct nfsd4_cache_entry {
- =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0ce_status;
- =A0 =A0 struct kvec =A0 =A0 ce_datav; /* encoded NFSv4.1 data in r=

q_res.head[0] */

Quoted 9 lines
Quoted 9 lines

- =A0 =A0 struct page =A0 =A0 *ce_respages[NFSD_PAGES_PER_SLOT + 1];
- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_cachethis;
- =A0 =A0 short =A0 =A0 =A0 =A0 =A0 ce_resused;
- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_opcnt;
- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_rpchdrlen;

include/linux/nfsd/state.h +0 -2
-};
-

=A0struct nfsd4_slot {
- =A0 =A0 bool =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

sl_inuse;

Quoted 1 line
Quoted 1 line

- =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

sl_seqid;

Quoted 10 lines
Quoted 8 lines

- =A0 =A0 struct nfsd4_cache_entry =A0 =A0 =A0 =A0sl_cache_entry;
+ =A0 =A0 bool =A0 =A0sl_inuse;
+ =A0 =A0 u32 =A0 =A0 sl_seqid;
+ =A0 =A0 int =A0 =A0 sl_cachethis;
+ =A0 =A0 int =A0 =A0 sl_opcnt;
+ =A0 =A0 __be32 =A0sl_status;
+ =A0 =A0 u32 =A0 =A0 sl_datalen;
+ =A0 =A0 char =A0 =A0sl_data[];

Could you just move sl_inuse to the end? =A0It'll save a few bytes in=

the

Quoted 1 line

structure (because the compiler will probably stick 3 bytes after it =

to

Quoted 1 line

align sl_seqid.)

How about this?

struct nfsd4_slot {

-       bool                            sl_inuse;
-       u32                             sl_seqid;
-       struct nfsd4_cache_entry        sl_cache_entry;
+       bool    sl_inuse;
+       bool    sl_cachethis;
+       u16     sl_opcnt;
+       u32     sl_seqid;
+       __be32  sl_status;
+       u32     sl_datalen;
+       char    sl_data[];

};

-->Andy

Quoted 10 lines

--b.

include/linux/nfsd/state.h struct nfsd4_session { · line 162

Quoted 8 lines

=A0};

=A0struct nfsd4_channel_attrs {

include/linux/nfsd/state.h +0 -1
@@ -159,7 +152,7 @@ struct nfsd4_session {
 =A0 =A0 =A0 struct nfs4_sessionid =A0 se_sessionid;
 =A0 =A0 =A0 struct nfsd4_channel_attrs se_fchannel;
 =A0 =A0 =A0 struct nfsd4_channel_attrs se_bchannel;
- =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 se_slots[]; =A0 =A0 /* forwa=

rd channel slots */

Quoted 1 line

include/linux/nfsd/state.h struct nfsd4_session { · line 155

Quoted 1 line
include/linux/nfsd/state.h +1 -0
+ =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 *se_slots[]; =A0 =A0/* forwa=

rd channel slots */

Quoted 15 lines
Quoted 8 lines
include/linux/nfsd/state.h +0 -0
 =A0};
 

=A0static inline void

include/linux/nfsd/xdr4.h +0 -0
index 3f71660..73164c2 100644
@@ -51,7 +51,7 @@ struct nfsd4_compound_state {
Quoted 7 lines

=A0 =A0 =A0 /* For sessions DRC */
=A0 =A0 =A0 struct nfsd4_session =A0 =A0*session;
=A0 =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 *slot;
- =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*statp;
+ =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*datap; =A0 =A0 =A0 size_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0iovlen; =A0 =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 minorversion=

;

Quoted 2 lines

include/linux/nfsd/xdr4.h static inline bool nfsd4_is_solo_sequence(struct=

Quoted 2 lines

=A0 =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 status;

include/linux/nfsd/xdr4.h +0 -0
@@ -472,8 +472,7 @@ static inline bool nfsd4_is_solo_sequence(struct=

nfsd4_compoundres *resp)

Quoted 2 lines
Quoted 2 lines

=A0static inline bool nfsd4_not_cached(struct nfsd4_compoundres *res=

p)

Quoted 3 lines
Quoted 3 lines

=A0{
- =A0 =A0 return !resp->cstate.slot->sl_cache_entry.ce_cachethis ||
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_is_solo_sequence(res=

p);

Quoted 1 line
Quoted 1 line

+ =A0 =A0 return !resp->cstate.slot->sl_cachethis || nfsd4_is_solo_s=

equence(resp);

Quoted 3 lines
Quoted 3 lines

=A0}

=A0#define NFS4_SVC_XDRSIZE =A0 =A0 =A0 =A0 =A0 =A0 sizeof(struct nf=

sd4_compoundargs)

Quoted 8 lines
Quoted 3 lines

--
1.6.2.5

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

A message that is not in the archive (sent off-list or not yet received): 89c397150909010648v4a4f5db8t87e09717e5a2c950-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#12 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

On Tue, Sep 01, 2009 at 09:48:41AM -0400, William A. (Andy) Adamson wro= te:

Quoted 6 lines
Quoted 4 lines · On Fri, Aug 28, 2009 at 5:33 PM, J. Bruce Fields<[email protected]= wrote:
Quoted 3 lines · On Thu, Aug 27, 2009 at 12:07:44PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

Use NFSD_SLOT_CACHE_SIZE size buffers for sessions DRC instead of =

holding nfsd

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

pages in cache.

Connectathon testing has shown that 1024 bytes for encoded compoun=

d operation

Quoted 1 line
Quoted 1 line
Quoted 1 line

responses past the sequence operation is sufficient, 512 bytes is =

a little too

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

small. Set NFSD_SLOT_CACHE_SIZE to 1024.

Allocate memory for the session DRC in the CREATE_SESSION operatio=

n

Quoted 1 line
Quoted 1 line
Quoted 1 line

to guarantee that the memory resource is available for caching res=

ponses.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Allocate each slot individually in preparation for slot table size=

negotiation.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Remove struct nfsd4_cache_entry and helper functions for the old p=

age-based

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

DRC.

The iov_len calculation in nfs4svc_encode_compoundres is now alway=

s

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

correct, clean up the nfs4svc_encode_compoundres session logic.

The nfsd4_compound_state statp pointer is also not used. Remove nfsd4_set_statp().

Move useful nfsd4_cache_entry fields into nfsd4_slot.

---
=C2=A0fs/nfsd/nfs4state.c =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0207 +=

+++++++++++--------------------------------

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0fs/nfsd/nfs4xdr.c =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0=

13 ++--

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0fs/nfsd/nfssvc.c =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0=

=C2=A04 -

Quoted 9 lines
Quoted 9 lines

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc=

Quoted 9 lines

=C2=A0include/linux/nfsd/state.h | =C2=A0 27 ++---- =C2=A0include/linux/nfsd/xdr4.h =C2=A0| =C2=A0 =C2=A05 +- =C2=A05 files changed, 74 insertions(+), 182 deletions(-)

fs/nfsd/nfs4state.c +0 -0
index 4695cec..2d72d5c 100644
@@ -510,12 +510,22 @@ static int init_forechannel_attrs(struct svc=

_rqst *rqstp,

Quoted 10 lines
Quoted 10 lines

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_rqst *rqstp,

Quoted 10 lines
fs/nfsd/nfs4state.c +7 -0
 =C2=A0 =C2=A0 =C2=A0 return status;
 =C2=A0}
 
+static void
+free_session_slots(struct nfsd4_session *ses)
+{
+ =C2=A0 =C2=A0 int i;
+
+ =C2=A0 =C2=A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++)
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 kfree(ses->se_slots[i]=

);

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +2 -0
+}
+

=C2=A0static int
=C2=A0alloc_init_session(struct svc_rqst *rqstp, struct nfs4_clien=

t *clp,

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0stru=

ct nfsd4_create_session *cses)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=C2=A0{
=C2=A0 =C2=A0 =C2=A0 struct nfsd4_session *new, tmp;
- =C2=A0 =C2=A0 int idx, status =3D nfserr_serverfault, slotsize;
+ =C2=A0 =C2=A0 struct nfsd4_slot *sp;
+ =C2=A0 =C2=A0 int idx, status =3D nfserr_serverfault, slotsize, =

cachesize, i;

Quoted 2 lines
Quoted 2 lines

Just as a style thing: that list's getting a little long. =C2=A0Cou=

ld you

Quoted 6 lines
Quoted 6 lines

keep at least "status" on a separate line?

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s=

Quoted 4 lines

=C2=A0 =C2=A0 =C2=A0 memset(&tmp, 0, sizeof(tmp));

fs/nfsd/nfs4state.c +0 -0
@@ -526,14 +536,23 @@ alloc_init_session(struct svc_rqst *rqstp, s=

truct nfs4_client *clp,

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s= · line 529

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -1
 =C2=A0 =C2=A0 =C2=A0 if (status)
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto out;
 
- =C2=A0 =C2=A0 /* allocate struct nfsd4_session and slot table in=

one piece */

Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s= · line 530

Quoted 1 line
fs/nfsd/nfs4state.c +0 -1
- =C2=A0 =C2=A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(stru=

ct nfsd4_slot);

Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s= · line 539

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =C2=A0 =C2=A0 /* allocate struct nfsd4_session and slot table po=

inters in one piece */

Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s= · line 540

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =C2=A0 =C2=A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(stru=

ct nfsd4_slot *);

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 new =3D kzalloc(sizeof(*new) + slotsize, GFP_=

KERNEL);

Quoted 4 lines
Quoted 4 lines

I think this is OK for now, but maybe stick something like:

=C2=A0 =C2=A0 =C2=A0 =C2=A0BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION =

* sizeof(struct nfsd4_slot)

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=

=C2=A0 =C2=A0+ sizeof(struct nfsd4_session) > PAGE_SIZE);

Quoted 3 lines
Quoted 3 lines

in state.h just to warn anyone who wants to blindly bump up NFSD_MAX_SLOTS_PER_SESSION. =C2=A0(It's not really forbidden to kma=

lloc more

Quoted 1 line
Quoted 1 line

than a page, but it's also not reliable, and if it becomes necessar=

y

Quoted 8 lines
Quoted 8 lines

then we'd rather find some way to code around it.)

Quoted 6 lines

=C2=A0 =C2=A0 =C2=A0 if (!new)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto out;

=C2=A0 =C2=A0 =C2=A0 memcpy(new, &tmp, sizeof(*new));

+ =C2=A0 =C2=A0 /* allocate each struct nfsd4_slot and data cache =

in one piece */

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 cachesize =3D new->se_fchannel.maxresp_cached - NF=

SD_MIN_HDR_SEQ_SZ;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =C2=A0 =C2=A0 for (i =3D 0; i < new->se_fchannel.maxreqs; i++) {
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D kzalloc(sizeof(=

*sp) + cachesize, GFP_KERNEL);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!sp)
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 goto out_free;

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 new->se_slots[i] =3D s=

p;

Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, st=

Quoted 6 lines

+ =C2=A0 =C2=A0 }
+
=C2=A0 =C2=A0 =C2=A0 new->se_client =3D clp;
=C2=A0 =C2=A0 =C2=A0 gen_sessionid(new);
=C2=A0 =C2=A0 =C2=A0 idx =3D hash_sessionid(&new->se_sessionid);

fs/nfsd/nfs4state.c +0 -0
@@ -550,6 +569,10 @@ alloc_init_session(struct svc_rqst *rqstp, st=

ruct nfs4_client *clp,

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
fs/nfsd/nfs4state.c +0 -0
 =C2=A0 =C2=A0 =C2=A0 status =3D nfs_ok;

=C2=A0out:
=C2=A0 =C2=A0 =C2=A0 return status;
+out_free:
+ =C2=A0 =C2=A0 free_session_slots(new);
+ =C2=A0 =C2=A0 kfree(new);
+ =C2=A0 =C2=A0 goto out;
=C2=A0}

Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfs4state.c release_session(struct nfsd4_session *ses) · line 595

Quoted 6 lines

=C2=A0/* caller must hold sessionid_lock */

fs/nfsd/nfs4state.c +0 -1
@@ -592,22 +615,16 @@ release_session(struct nfsd4_session *ses)
 =C2=A0 =C2=A0 =C2=A0 nfsd4_put_session(ses);
 =C2=A0}
 
-static void nfsd4_release_respages(struct page **respages, short =

resused);

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfs4state.c +0 -1
-

=C2=A0void
=C2=A0free_session(struct kref *kref)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 struct nfsd4_session *ses;
- =C2=A0 =C2=A0 int i;

=C2=A0 =C2=A0 =C2=A0 ses =3D container_of(kref, struct nfsd4_sessi=

on, se_ref);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct nfsd4_cache_ent=

ry *e =3D &ses->se_slots[i].sl_cache_entry;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_release_respages=

(e->ce_respages, e->ce_resused);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 spin_lock(&nfsd_drc_lock);
=C2=A0 =C2=A0 =C2=A0 nfsd_drc_mem_used -=3D ses->se_fchannel.maxre=

qs * NFSD_SLOT_CACHE_SIZE;

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines

=C2=A0 =C2=A0 =C2=A0 spin_unlock(&nfsd_drc_lock);
+ =C2=A0 =C2=A0 free_session_slots(ses);
=C2=A0 =C2=A0 =C2=A0 kfree(ses);
=C2=A0}

fs/nfsd/nfs4state.c +0 -12
@@ -964,116 +981,32 @@ out_err:
 =C2=A0 =C2=A0 =C2=A0 return;
 =C2=A0}
 
-void
-nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
-{
- =C2=A0 =C2=A0 struct nfsd4_compoundres *resp =3D rqstp->rq_resp;
-
- =C2=A0 =C2=A0 resp->cstate.statp =3D statp;
-}
-
-/*
- * Dereference the result pages.
- */
-static void
Quoted 8 lines
Quoted 8 lines

fs/nfsd/nfs4state.c out_err: · line 986

Quoted 8 lines
fs/nfsd/nfs4state.c +0 -8
-nfsd4_release_respages(struct page **respages, short resused)
-{
- =C2=A0 =C2=A0 int i;
-
- =C2=A0 =C2=A0 dprintk("--> %s\n", __func__);
- =C2=A0 =C2=A0 for (i =3D 0; i < resused; i++) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!respages[i])
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 continue;

Quoted 7 lines
Quoted 7 lines

fs/nfsd/nfs4state.c out_err: · line 993

Quoted 7 lines
fs/nfsd/nfs4state.c +0 -7
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 put_page(respages[i]);
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 respages[i] =3D NULL;
- =C2=A0 =C2=A0 }
-}
-
-static void
-nfsd4_copy_pages(struct page **topages, struct page **frompages, =

short count)

Quoted 5 lines
Quoted 5 lines

fs/nfsd/nfs4state.c out_err: · line 998

Quoted 5 lines
fs/nfsd/nfs4state.c +0 -5
-{
- =C2=A0 =C2=A0 int i;
-
- =C2=A0 =C2=A0 for (i =3D 0; i < count; i++) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 topages[i] =3D frompag=

es[i];

Quoted 2 lines
Quoted 2 lines

fs/nfsd/nfs4state.c out_err: · line 1000

Quoted 2 lines
fs/nfsd/nfs4state.c +0 -2
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!topages[i])
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 continue;

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
fs/nfsd/nfs4state.c +0 -4
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 get_page(topages[i]);
- =C2=A0 =C2=A0 }
-}
-

=C2=A0/*
- * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing =

the previous

Quoted 1 line
Quoted 1 line
Quoted 1 line

- * pages. We add a page to NFSD_PAGES_PER_SLOT for the case where=

the total

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- * length of the XDR response is less than se_fmaxresp_cached
- * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is use=

d for a

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

- * of the reply (e.g. readdir).
- *
- * Store the base and length of the rq_req.head[0] page
- * of the NFSv4.1 data, just past the rpc header.
+ * Cache a reply. nfsd4_check_drc_limit() has bounded the cache s=

ize.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=C2=A0 */
=C2=A0void
=C2=A0nfsd4_store_cache_entry(struct nfsd4_compoundres *resp) =C2=A0{
- =C2=A0 =C2=A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.=

slot->sl_cache_entry;

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

- =C2=A0 =C2=A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =C2=A0 =C2=A0 struct kvec *resv =3D &rqstp->rq_res.head[0]; -
- =C2=A0 =C2=A0 dprintk("--> %s entry %p\n", __func__, entry);
+ =C2=A0 =C2=A0 struct nfsd4_slot *slot =3D resp->cstate.slot;
+ =C2=A0 =C2=A0 unsigned int base;

- =C2=A0 =C2=A0 nfsd4_release_respages(entry->ce_respages, entry->=

ce_resused);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- =C2=A0 =C2=A0 entry->ce_opcnt =3D resp->opcnt;
- =C2=A0 =C2=A0 entry->ce_status =3D resp->cstate.status;
+ =C2=A0 =C2=A0 dprintk("--> %s slot %p\n", __func__, slot);

- =C2=A0 =C2=A0 /*
- =C2=A0 =C2=A0 =C2=A0* Don't need a page to cache just the sequen=

ce operation - the slot

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

- =C2=A0 =C2=A0 =C2=A0* does this for us!
- =C2=A0 =C2=A0 =C2=A0*/
+ =C2=A0 =C2=A0 slot->sl_opcnt =3D resp->opcnt;
+ =C2=A0 =C2=A0 slot->sl_status =3D resp->cstate.status;

=C2=A0 =C2=A0 =C2=A0 if (nfsd4_not_cached(resp)) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry->ce_resused =3D =

0;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry->ce_rpchdrlen =3D=

0;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dprintk("%s Just cache=

SEQUENCE. ce_cachethis %d\n", __func__,

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 resp->cstate.slot->sl_cache_entry.ce_cachethis);

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->sl_datalen =3D 0=

;

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return; =C2=A0 =C2=A0 =C2=A0 }
- =C2=A0 =C2=A0 entry->ce_resused =3D rqstp->rq_resused;
- =C2=A0 =C2=A0 if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry->ce_resused =3D =

NFSD_PAGES_PER_SLOT + 1;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 nfsd4_copy_pages(entry->ce_respages, rqstp->rq_res=

pages,

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0entry->ce_resused);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 entry->ce_datav.iov_base =3D resp->cstate.statp;
- =C2=A0 =C2=A0 entry->ce_datav.iov_len =3D resv->iov_len - ((char=

*)resp->cstate.statp -

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (char *)page_address(rqstp->rq_respages= [0]));

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 /* Current request rpc header length*/
- =C2=A0 =C2=A0 entry->ce_rpchdrlen =3D (char *)resp->cstate.statp=

-

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (char *)page_address(rqstp->rq_respages= [0]);

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -4
-}
-
-/*
- * We keep the rpc header, but take the nfs reply from the replyc=

ache.

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines

- */

fs/nfsd/nfs4state.c +0 -3
-static int
-nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 struct nfsd4_cache_entry *entry)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

-{
- =C2=A0 =C2=A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =C2=A0 =C2=A0 struct kvec *resv =3D &resp->rqstp->rq_res.head[0]=

;

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

- =C2=A0 =C2=A0 int len;
-
- =C2=A0 =C2=A0 /* Current request rpc header length*/
- =C2=A0 =C2=A0 len =3D (char *)resp->cstate.statp -
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 (char *)page_address(rqstp->rq_respages[0]);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dprintk("%s v41 cached=

reply too large (%Zd).\n", __func__,

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 entry->ce_datav.iov_len);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0;
- =C2=A0 =C2=A0 }
- =C2=A0 =C2=A0 /* copy the cached reply nfsd data past the curren=

t rpc header */

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 memcpy((char *)resv->iov_base + len, entry->ce_dat=

av.iov_base,

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry->ce_datav.iov_le=

n);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 resv->iov_len =3D len + entry->ce_datav.iov_len;
- =C2=A0 =C2=A0 return 1;
+ =C2=A0 =C2=A0 slot->sl_datalen =3D (char *)resp->p - (char *)res=

p->cstate.datap;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =C2=A0 =C2=A0 base =3D (char *)resp->cstate.datap -
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (char *)res= p->xbuf->head[0].iov_base;

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 if (read_bytes_from_xdr_buf(resp->xbuf, base, slot=

->sl_data,

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 slot->sl_datalen))

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printk(KERN_WARNING
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 "nfsd: sessions DRC could not cache compound\n");

Quoted 2 lines
Quoted 2 lines

I'd make this WARN("nfsd:...") just to make it completely clear it'=

s a

Quoted 1 line
Quoted 1 line

kernel bug. =C2=A0(This case should be caught by nfsd4_check_drc_li=

mit unless

Quoted 7 lines
Quoted 7 lines

we've messed something up, right?)

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_com=

Quoted 5 lines

+ =C2=A0 =C2=A0 return;
=C2=A0}

=C2=A0/*

fs/nfsd/nfs4state.c +0 -0
@@ -1091,14 +1024,14 @@ nfsd4_enc_sequence_replay(struct nfsd4_com=

poundargs *args,

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 struct nfsd4_slot *slot =3D resp->cstate.slot=

;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0 =C2=A0 =C2=A0 dprintk("--> %s resp->opcnt %d cachethis %u \=

n", __func__,

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 resp->opcnt, resp->cst=

ate.slot->sl_cache_entry.ce_cachethis);

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 resp->opcnt, resp->cst=

ate.slot->sl_cachethis);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=C2=A0 =C2=A0 =C2=A0 /* Encode the replayed sequence operation */ =C2=A0 =C2=A0 =C2=A0 op =3D &args->ops[resp->opcnt - 1]; =C2=A0 =C2=A0 =C2=A0 nfsd4_encode_operation(resp, op);

=C2=A0 =C2=A0 =C2=A0 /* Return nfserr_retry_uncached_rep in next o=

peration. */

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 if (args->opcnt > 1 && slot->sl_cache_entry.ce_cac=

hethis =3D=3D 0) {

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 if (args->opcnt > 1 && slot->sl_cachethis =3D=3D 0=

) {

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 op =3D &args->ops=

[resp->opcnt++];

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 op->status =3D nf=

serr_retry_uncached_rep;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_encode_oper=

ation(resp, op);

Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_com=

Quoted 1 line
fs/nfsd/nfs4state.c +0 -0
@@ -1107,57 +1040,29 @@ nfsd4_enc_sequence_replay(struct nfsd4_com=

poundargs *args,

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +0 -0
 =C2=A0}
 

=C2=A0/*
- * Keep the first page of the replay. Copy the NFSv4.1 data from =

the first

Quoted 1 line
Quoted 1 line
Quoted 1 line

- * cached page. =C2=A0Replace any futher replay pages from the ca=

che.

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * The sequence operation is not cached because we can use the sl=

ot and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

+ * session values.
=C2=A0 */
=C2=A0__be32
=C2=A0nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp, =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0struct nfsd4_sequence *seq)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0{
- =C2=A0 =C2=A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.=

slot->sl_cache_entry;

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

+ =C2=A0 =C2=A0 struct nfsd4_slot *slot =3D resp->cstate.slot; =C2=A0 =C2=A0 =C2=A0 __be32 status;

- =C2=A0 =C2=A0 dprintk("--> %s entry %p\n", __func__, entry); -
- =C2=A0 =C2=A0 /*
- =C2=A0 =C2=A0 =C2=A0* If this is just the sequence operation, we=

did not keep

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0* a page in the cache entry because we can j=

ust use the

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0* slot info stored in struct nfsd4_sequence =

that was checked

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 =C2=A0* against the slot in nfsd4_sequence().
- =C2=A0 =C2=A0 =C2=A0*
- =C2=A0 =C2=A0 =C2=A0* This occurs when seq->cachethis is FALSE, =

or when the client

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0* session inactivity timer fires and a solo =

sequence operation

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

- =C2=A0 =C2=A0 =C2=A0* is sent (lease renewal).
- =C2=A0 =C2=A0 =C2=A0*/
+ =C2=A0 =C2=A0 dprintk("--> %s slot %p\n", __func__, slot);

=C2=A0 =C2=A0 =C2=A0 /* Either returns 0 or nfserr_retry_uncached =

*/

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 status =3D nfsd4_enc_sequence_replay(resp->rq=

stp->rq_argp, resp);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=C2=A0 =C2=A0 =C2=A0 if (status =3D=3D nfserr_retry_uncached_rep) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return status;

- =C2=A0 =C2=A0 if (!nfsd41_copy_replay_data(resp, entry)) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /*
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* Not enough roo=

m to use the replay rpc header, send the

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* cached header.=

Release all the allocated result pages.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*/
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 svc_free_res_pages(res=

p->rqstp);

Quoted 3 lines
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_copy_pages(resp-=

rqstp->rq_respages, entry->ce_respages,

Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 entry->ce_resused);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 } else {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* Release all but the=

first allocated result page */

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 resp->rqstp->rq_resuse=

d--;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 svc_free_res_pages(res=

p->rqstp);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_copy_pages(&resp=

->rqstp->rq_respages[1],

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&entry->ce_respages[1],

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0entry->ce_resused - 1);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =C2=A0 =C2=A0 }
+ =C2=A0 =C2=A0 /* The sequence operation has been encoded, cstate=

->datap set. */

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 memcpy(resp->cstate.datap, slot->sl_data, slot->sl=

_datalen);

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

- =C2=A0 =C2=A0 resp->rqstp->rq_resused =3D entry->ce_resused;
- =C2=A0 =C2=A0 resp->opcnt =3D entry->ce_opcnt;
- =C2=A0 =C2=A0 resp->cstate.iovlen =3D entry->ce_datav.iov_len + =

entry->ce_rpchdrlen;

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 status =3D entry->ce_status;
+ =C2=A0 =C2=A0 resp->opcnt =3D slot->sl_opcnt;
+ =C2=A0 =C2=A0 resp->p =3D resp->cstate.datap + XDR_QUADLEN(slot-=

sl_datalen);

Quoted 6 lines
Quoted 6 lines

+ =C2=A0 =C2=A0 status =3D slot->sl_status;

=C2=A0 =C2=A0 =C2=A0 return status;
=C2=A0}

fs/nfsd/nfs4state.c +0 -0
@@ -1489,7 +1394,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,

=C2=A0 =C2=A0 =C2=A0 if (seq->slotid >=3D session->se_fchannel.max=

reqs)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto out;

- =C2=A0 =C2=A0 slot =3D &session->se_slots[seq->slotid];
+ =C2=A0 =C2=A0 slot =3D session->se_slots[seq->slotid]; =C2=A0 =C2=A0 =C2=A0 dprintk("%s: slotid %d\n", __func__, seq->slo=

tid);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0 =C2=A0 =C2=A0 /* We do not negotiate the number of slots ye=

t, so set the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
fs/nfsd/nfs4state.c +0 -0
@@ -1502,7 +1407,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cstate->slot =3D =

slot;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cstate->session =3D=

session;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* Return the cac=

hed reply status and set cstate->status

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* for nfsd4_svc_=

encode_compoundres processing */

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0* for nfsd4_proc=

_compound processing */

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 status =3D nfsd4_=

replay_cache_entry(resp, seq);

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cstate->status =3D=

nfserr_replay_cache;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto replay_cache=

;

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +0 -0
@@ -1513,7 +1418,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,

=C2=A0 =C2=A0 =C2=A0 /* Success! bump slot seqid */ =C2=A0 =C2=A0 =C2=A0 slot->sl_inuse =3D true;
=C2=A0 =C2=A0 =C2=A0 slot->sl_seqid =3D seq->seqid;
- =C2=A0 =C2=A0 slot->sl_cache_entry.ce_cachethis =3D seq->cacheth=

is;

Quoted 9 lines
Quoted 9 lines

fs/nfsd/nfs4xdr.c nfsd4_encode_sequence(struct nfsd4_compoundr=

Quoted 9 lines

+ =C2=A0 =C2=A0 slot->sl_cachethis =3D seq->cachethis;

=C2=A0 =C2=A0 =C2=A0 cstate->slot =3D slot;
=C2=A0 =C2=A0 =C2=A0 cstate->session =3D session;

fs/nfsd/nfs4xdr.c +0 -0
index fdf632b..49824ea 100644
@@ -3064,6 +3064,7 @@ nfsd4_encode_sequence(struct nfsd4_compoundr=

es *resp, int nfserr,

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4xdr.c nfsd4_encode_sequence(struct nfsd4_compoundr= · line 3067

Quoted 4 lines
fs/nfsd/nfs4xdr.c +1 -0
 =C2=A0 =C2=A0 =C2=A0 WRITE32(0);
 
 =C2=A0 =C2=A0 =C2=A0 ADJUST_ARGS();
+ =C2=A0 =C2=A0 resp->cstate.datap =3D p; /* DRC cache data pointe=

r */

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfsd=

Quoted 4 lines
fs/nfsd/nfs4xdr.c +0 -0
 =C2=A0 =C2=A0 =C2=A0 return 0;
 =C2=A0}
 
@@ -3166,7 +3167,7 @@ static int nfsd4_check_drc_limit(struct nfsd=

4_compoundres *resp)

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfsd= · line 3169

Quoted 4 lines
fs/nfsd/nfs4xdr.c +0 -1
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return status;
 
 =C2=A0 =C2=A0 =C2=A0 session =3D resp->cstate.session;
- =C2=A0 =C2=A0 if (session =3D=3D NULL || slot->sl_cache_entry.ce=

_cachethis =3D=3D 0)

Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfsd= · line 3170

Quoted 1 line
fs/nfsd/nfs4xdr.c +1 -0
+ =C2=A0 =C2=A0 if (session =3D=3D NULL || slot->sl_cachethis =3D=3D=

0)

Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqst *=

Quoted 4 lines
fs/nfsd/nfs4xdr.c +0 -0
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return status;
 
 =C2=A0 =C2=A0 =C2=A0 if (resp->opcnt >=3D args->opcnt)
@@ -3291,6 +3292,7 @@ nfs4svc_encode_compoundres(struct svc_rqst *=

rqstp, __be32 *p, struct nfsd4_compo

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0 =C2=A0 =C2=A0 /*
=C2=A0 =C2=A0 =C2=A0 =C2=A0* All that remains is to write the tag =

and operation count...

Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqst=

Quoted 6 lines

=C2=A0 =C2=A0 =C2=A0 =C2=A0*/
+ =C2=A0 =C2=A0 struct nfsd4_compound_state *cs =3D &resp->cstate; =C2=A0 =C2=A0 =C2=A0 struct kvec *iov;
=C2=A0 =C2=A0 =C2=A0 p =3D resp->tagp;
=C2=A0 =C2=A0 =C2=A0 *p++ =3D htonl(resp->taglen);

fs/nfsd/nfs4xdr.c +0 -0
@@ -3304,14 +3306,11 @@ nfs4svc_encode_compoundres(struct svc_rqst=

*rqstp, __be32 *p, struct nfsd4_compo

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 iov =3D &rqstp->r=

q_res.head[0];

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 iov->iov_len =3D ((char*)resp->p) - (char*)io=

v->iov_base;

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=C2=A0 =C2=A0 =C2=A0 BUG_ON(iov->iov_len > PAGE_SIZE);
- =C2=A0 =C2=A0 if (nfsd4_has_session(&resp->cstate)) {
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (resp->cstate.statu=

s =3D=3D nfserr_replay_cache &&

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 !nfsd4_not_cached(resp)) {

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 iov->iov_len =3D resp->cstate.iovlen;

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } else {
+ =C2=A0 =C2=A0 if (nfsd4_has_session(cs)) {
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (cs->status !=3D nf=

serr_replay_cache) {

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 nfsd4_store_cache_entry(resp);

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 dprintk("%s: SET SLOT STATE TO AVAILABLE\n", __func__);

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 resp->cstate.slot->sl_inuse =3D 0;

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 resp->cstate.slot->sl_inuse =3D false;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_put_session=

(resp->cstate.session);

Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfssvc.c nfsd_dispatch(struct svc_rqst *rqstp, __be32 =

Quoted 6 lines

=C2=A0 =C2=A0 =C2=A0 }

fs/nfsd/nfssvc.c +0 -0
index d68cd05..944ef01 100644
@@ -576,10 +576,6 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32 =

*statp)

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 + rqstp->rq_res.h=

ead[0].iov_len;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 rqstp->rq_res.head[0].iov_len +=3D sizeof(__b=

e32);

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

- =C2=A0 =C2=A0 /* NFSv4.1 DRC requires statp */
- =C2=A0 =C2=A0 if (rqstp->rq_vers =3D=3D 4)
- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 nfsd4_set_statp(rqstp,=

statp);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-
=C2=A0 =C2=A0 =C2=A0 /* Now call the procedure handler, and encode=

NFS status. */

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 nfserr =3D proc->pc_func(rqstp, rqstp->rq_arg=

p, rqstp->rq_resp);

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 nfserr =3D map_new_errors(rqstp->rq_vers, nfs=

err);

Quoted 1 line
Quoted 1 line

include/linux/nfsd/state=

Quoted 1 line
include/linux/nfsd/state= renamed from include/linux/nfsd/state.h +0 -0

=2Eh

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

index ff0b771..e745100 100644

include/linux/nfsd/state.h +0 -0
@@ -94,30 +94,23 @@ struct nfs4_cb_conn {
 

=C2=A0/* Maximum number of slots per session. 160 is useful for lo=

ng haul TCP */

Quoted 4 lines
Quoted 4 lines

include/linux/nfsd/state.h

Quoted 4 lines

=C2=A0#define NFSD_MAX_SLOTS_PER_SESSION =C2=A0 =C2=A0 160

include/linux/nfsd/state.h +0 -3
-/* Maximum number of pages per slot cache entry */
-#define NFSD_PAGES_PER_SLOT =C2=A01
-#define NFSD_SLOT_CACHE_SIZE =C2=A0 =C2=A0 =C2=A0 =C2=A0 PAGE_SIZ=

E

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=C2=A0/* Maximum number of operations per session compound */ =C2=A0#define NFSD_MAX_OPS_PER_COMPOUND =C2=A0 =C2=A016

include/linux/nfsd/state.h +2 -0
+/* Maximum =C2=A0session per slot cache size */
+#define NFSD_SLOT_CACHE_SIZE =C2=A0 =C2=A0 =C2=A0 =C2=A0 1024

=C2=A0/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session =

*/

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=C2=A0#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION =C2=A0 =C2=A032 =C2=A0#define NFSD_MAX_MEM_PER_SESSION =C2=A0\
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (NFSD_CACHE_SIZE_=

SLOTS_PER_SESSION * NFSD_SLOT_CACHE_SIZE)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

-struct nfsd4_cache_entry {
- =C2=A0 =C2=A0 __be32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ce_status=

;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 struct kvec =C2=A0 =C2=A0 ce_datav; /* encoded NFS=

v4.1 data in rq_res.head[0] */

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 struct page =C2=A0 =C2=A0 *ce_respages[NFSD_PAGES_=

PER_SLOT + 1];

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 int =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ce_c=

achethis;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 short =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ce_resuse=

d;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 int =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ce_o=

pcnt;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 int =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ce_r=

pchdrlen;

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
include/linux/nfsd/state.h +0 -2
-};
-

=C2=A0struct nfsd4_slot {
- =C2=A0 =C2=A0 bool =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sl_inuse;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 u32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sl_seqid;

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 struct nfsd4_cache_entry =C2=A0 =C2=A0 =C2=A0 =C2=A0=

sl_cache_entry;

Quoted 9 lines
Quoted 9 lines
Quoted 7 lines

+ =C2=A0 =C2=A0 bool =C2=A0 =C2=A0sl_inuse;
+ =C2=A0 =C2=A0 u32 =C2=A0 =C2=A0 sl_seqid;
+ =C2=A0 =C2=A0 int =C2=A0 =C2=A0 sl_cachethis;
+ =C2=A0 =C2=A0 int =C2=A0 =C2=A0 sl_opcnt;
+ =C2=A0 =C2=A0 __be32 =C2=A0sl_status;
+ =C2=A0 =C2=A0 u32 =C2=A0 =C2=A0 sl_datalen;
+ =C2=A0 =C2=A0 char =C2=A0 =C2=A0sl_data[];

Could you just move sl_inuse to the end? =C2=A0It'll save a few byt=

es in the

Quoted 1 line
Quoted 1 line

structure (because the compiler will probably stick 3 bytes after i=

t to

Quoted 6 lines
Quoted 1 line

align sl_seqid.)

=20
How about this?
=20
struct nfsd4_slot {

include/linux/nfsd/state.h +0 -1
-       bool                            sl_inuse;

include/linux/nfsd/state.h struct nfs4_cb_conn { · line 114

Quoted 10 lines
include/linux/nfsd/state.h +7 -2
-       u32                             sl_seqid;
-       struct nfsd4_cache_entry        sl_cache_entry;
+       bool    sl_inuse;
+       bool    sl_cachethis;
+       u16     sl_opcnt;
+       u32     sl_seqid;
+       __be32  sl_status;
+       u32     sl_datalen;
+       char    sl_data[];
 };

Looks OK to me! By the way, on recent distro's you should be able to run apt-get install/yum install dwarves, then

pahole fs/nfsd/nfsd.o

and look through the output for the struct of interest:

struct nfsd4_slot {
bool sl_inuse; /* 0 1=
*/

/* XXX 3 bytes hole, try to pack */

u32 sl_seqid; /* 4 4=
*/
struct nfsd4_cache_entry sl_cache_entry; /* 8 36=
*/

        /* size: 44, cachelines: 1 */
        /* sum members: 41, holes: 1, sum holes: 3 */
        /* last cacheline: 44 bytes */
};      /* definitions: 3 */

(But no point obsessing over this kind of thing; I just happened to notice the problem in this patch and figured for such a common structur= e
it might be worth mentioning.)

--b.

Quoted 5 lines

=20
-->Andy
=20

Quoted 2 lines

--b.

Quoted 8 lines
Quoted 8 lines

include/linux/nfsd/state.h struct nfsd4_session { · line 162

Quoted 8 lines

=C2=A0};

=C2=A0struct nfsd4_channel_attrs {

include/linux/nfsd/state.h +0 -1
@@ -159,7 +152,7 @@ struct nfsd4_session {
 =C2=A0 =C2=A0 =C2=A0 struct nfs4_sessionid =C2=A0 se_sessionid;
 =C2=A0 =C2=A0 =C2=A0 struct nfsd4_channel_attrs se_fchannel;
 =C2=A0 =C2=A0 =C2=A0 struct nfsd4_channel_attrs se_bchannel;
- =C2=A0 =C2=A0 struct nfsd4_slot =C2=A0 =C2=A0 =C2=A0 se_slots[];=

=C2=A0 =C2=A0 /* forward channel slots */

Quoted 1 line
Quoted 1 line

include/linux/nfsd/state.h struct nfsd4_session { · line 155

Quoted 1 line
include/linux/nfsd/state.h +1 -0
+ =C2=A0 =C2=A0 struct nfsd4_slot =C2=A0 =C2=A0 =C2=A0 *se_slots[]=

; =C2=A0 =C2=A0/* forward channel slots */

Quoted 12 lines
Quoted 12 lines
Quoted 8 lines
include/linux/nfsd/state.h +0 -0
 =C2=A0};
 

=C2=A0static inline void

include/linux/nfsd/xdr4.h +0 -0
index 3f71660..73164c2 100644
@@ -51,7 +51,7 @@ struct nfsd4_compound_state {
Quoted 4 lines

=C2=A0 =C2=A0 =C2=A0 /* For sessions DRC */
=C2=A0 =C2=A0 =C2=A0 struct nfsd4_session =C2=A0 =C2=A0*session; =C2=A0 =C2=A0 =C2=A0 struct nfsd4_slot =C2=A0 =C2=A0 =C2=A0 *slot;
- =C2=A0 =C2=A0 __be32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0*statp;

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 __be32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0*datap;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 size_t =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 =C2=A0 =C2=A0 =C2=A0iovlen;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 u32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 minorversion;

Quoted 1 line
Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 =C2=A0 u32 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=

=C2=A0 =C2=A0 =C2=A0 =C2=A0 status;

Quoted 1 line
Quoted 1 line

include/linux/nfsd/xdr4.h static inline bool nfsd4_is_solo_sequence(stru=

Quoted 1 line
include/linux/nfsd/xdr4.h +0 -0
@@ -472,8 +472,7 @@ static inline bool nfsd4_is_solo_sequence(stru=

ct nfsd4_compoundres *resp)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0static inline bool nfsd4_not_cached(struct nfsd4_compoundres=

*resp)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=C2=A0{
- =C2=A0 =C2=A0 return !resp->cstate.slot->sl_cache_entry.ce_cache=

this ||

Quoted 1 line
Quoted 1 line
Quoted 1 line

- =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=

=A0 nfsd4_is_solo_sequence(resp);

Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =C2=A0 =C2=A0 return !resp->cstate.slot->sl_cachethis || nfsd4_i=

s_solo_sequence(resp);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=C2=A0}

=C2=A0#define NFS4_SVC_XDRSIZE =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =

=C2=A0 sizeof(struct nfsd4_compoundargs)

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

--
1.6.2.5

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#13 lore raw

Re: [pnfs] [PATCH 5/5] nfsd41: replace page based DRC with buffer based DRC

Quoted 1 line · On Tue, Sep 1, 2009 at 11:14 AM, J. Bruce Fields<[email protected]> = wrote:

On Tue, Sep 01, 2009 at 09:48:41AM -0400, William A. (Andy) Adamson w=

rote:

Quoted 1 line
Quoted 1 line

On Fri, Aug 28, 2009 at 5:33 PM, J. Bruce Fields<bfields-uC3wQj2KruMpug/[email protected]=

Quoted 4 lines · g> wrote:
Quoted 4 lines
Quoted 4 lines · On Thu, Aug 27, 2009 at 12:07:44PM -0400, [email protected] wrote:
Quoted 3 lines

From: Andy Adamson <[email protected]>

Use NFSD_SLOT_CACHE_SIZE size buffers for sessions DRC instead of=

holding nfsd

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

pages in cache.

Connectathon testing has shown that 1024 bytes for encoded compou=

nd operation

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

responses past the sequence operation is sufficient, 512 bytes is=

a little too

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

small. Set NFSD_SLOT_CACHE_SIZE to 1024.

Allocate memory for the session DRC in the CREATE_SESSION operati=

on

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

to guarantee that the memory resource is available for caching re=

sponses.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Allocate each slot individually in preparation for slot table siz=

e negotiation.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Remove struct nfsd4_cache_entry and helper functions for the old =

page-based

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

DRC.

The iov_len calculation in nfs4svc_encode_compoundres is now alwa=

ys

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

correct, clean up the nfs4svc_encode_compoundres session logic.

The nfsd4_compound_state statp pointer is also not used. Remove nfsd4_set_statp().

Move useful nfsd4_cache_entry fields into nfsd4_slot.

---
=A0fs/nfsd/nfs4state.c =A0 =A0 =A0 =A0| =A0207 ++++++++++++------=

--------------------------

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct sv=

Quoted 11 lines

=A0fs/nfsd/nfs4xdr.c =A0 =A0 =A0 =A0 =A0| =A0 13 ++-- =A0fs/nfsd/nfssvc.c =A0 =A0 =A0 =A0 =A0 | =A0 =A04 - =A0include/linux/nfsd/state.h | =A0 27 ++---- =A0include/linux/nfsd/xdr4.h =A0| =A0 =A05 +- =A05 files changed, 74 insertions(+), 182 deletions(-)

fs/nfsd/nfs4state.c +0 -0
index 4695cec..2d72d5c 100644
@@ -510,12 +510,22 @@ static int init_forechannel_attrs(struct sv=

c_rqst *rqstp,

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 12 lines
fs/nfsd/nfs4state.c +9 -0
 =A0 =A0 =A0 return status;
 =A0}
 
+static void
+free_session_slots(struct nfsd4_session *ses)
+{
+ =A0 =A0 int i;
+
+ =A0 =A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++)
+ =A0 =A0 =A0 =A0 =A0 =A0 kfree(ses->se_slots[i]);
+}
+
Quoted 2 lines

=A0static int
=A0alloc_init_session(struct svc_rqst *rqstp, struct nfs4_client =

*clp,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct nfsd4_create_session *c=

ses)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=A0{
=A0 =A0 =A0 struct nfsd4_session *new, tmp;
- =A0 =A0 int idx, status =3D nfserr_serverfault, slotsize;
+ =A0 =A0 struct nfsd4_slot *sp;
+ =A0 =A0 int idx, status =3D nfserr_serverfault, slotsize, cache=

size, i;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Just as a style thing: that list's getting a little long. =A0Could=

you

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

keep at least "status" on a separate line?

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, =

Quoted 4 lines

=A0 =A0 =A0 memset(&tmp, 0, sizeof(tmp));

fs/nfsd/nfs4state.c +0 -0
@@ -526,14 +536,23 @@ alloc_init_session(struct svc_rqst *rqstp, =

struct nfs4_client *clp,

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 529

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -1
 =A0 =A0 =A0 if (status)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
- =A0 =A0 /* allocate struct nfsd4_session and slot table in one =

piece */

Quoted 1 line
Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 530

Quoted 1 line
fs/nfsd/nfs4state.c +0 -1
- =A0 =A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(struct nf=

sd4_slot);

Quoted 1 line
Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 539

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 /* allocate struct nfsd4_session and slot table pointer=

s in one piece */

Quoted 1 line
Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 540

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 slotsize =3D tmp.se_fchannel.maxreqs * sizeof(struct nf=

sd4_slot *);

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 541

Quoted 1 line
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 new =3D kzalloc(sizeof(*new) + slotsize, GFP_KERNEL);

I think this is OK for now, but maybe stick something like:

=A0 =A0 =A0 =A0BUILD_BUG_ON(NFSD_MAX_SLOTS_PER_SESSION * sizeof(st=

ruct nfsd4_slot)

Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0+ sizeof(struct nfs=

d4_session) > PAGE_SIZE);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

in state.h just to warn anyone who wants to blindly bump up NFSD_MAX_SLOTS_PER_SESSION. =A0(It's not really forbidden to kmall=

oc more

Quoted 1 line
Quoted 1 line
Quoted 1 line

than a page, but it's also not reliable, and if it becomes necessa=

ry

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

then we'd rather find some way to code around it.)

Quoted 6 lines
fs/nfsd/nfs4state.c +1 -0
 =A0 =A0 =A0 if (!new)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
 =A0 =A0 =A0 memcpy(new, &tmp, sizeof(*new));
 
+ =A0 =A0 /* allocate each struct nfsd4_slot and data cache in on=

e piece */

Quoted 1 line
Quoted 1 line
Quoted 1 line

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 548

Quoted 1 line
fs/nfsd/nfs4state.c +1 -0
+ =A0 =A0 cachesize =3D new->se_fchannel.maxresp_cached - NFSD_MI=

N_HDR_SEQ_SZ;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, = · line 550

Quoted 2 lines
fs/nfsd/nfs4state.c +2 -0
+ =A0 =A0 for (i =3D 0; i < new->se_fchannel.maxreqs; i++) {
+ =A0 =A0 =A0 =A0 =A0 =A0 sp =3D kzalloc(sizeof(*sp) + cachesize,=

GFP_KERNEL);

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

fs/nfsd/nfs4state.c alloc_init_session(struct svc_rqst *rqstp, s=

Quoted 9 lines
fs/nfsd/nfs4state.c +5 -0
+ =A0 =A0 =A0 =A0 =A0 =A0 if (!sp)
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out_free;
+ =A0 =A0 =A0 =A0 =A0 =A0 new->se_slots[i] =3D sp;
+ =A0 =A0 }
+
 =A0 =A0 =A0 new->se_client =3D clp;
 =A0 =A0 =A0 gen_sessionid(new);
 =A0 =A0 =A0 idx =3D hash_sessionid(&new->se_sessionid);
@@ -550,6 +569,10 @@ alloc_init_session(struct svc_rqst *rqstp, s=

truct nfs4_client *clp,

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 status =3D nfs_ok;

=A0out:
=A0 =A0 =A0 return status;
+out_free:
+ =A0 =A0 free_session_slots(new);
+ =A0 =A0 kfree(new);
+ =A0 =A0 goto out;
=A0}

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfs4state.c release_session(struct nfsd4_session *ses) · line 595

Quoted 6 lines

=A0/* caller must hold sessionid_lock */

fs/nfsd/nfs4state.c +0 -1
@@ -592,22 +615,16 @@ release_session(struct nfsd4_session *ses)
 =A0 =A0 =A0 nfsd4_put_session(ses);
 =A0}
 
-static void nfsd4_release_respages(struct page **respages, short=

resused);

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfs4state.c +0 -1
-

=A0void
=A0free_session(struct kref *kref)
=A0{
=A0 =A0 =A0 struct nfsd4_session *ses;
- =A0 =A0 int i;

=A0 =A0 =A0 ses =3D container_of(kref, struct nfsd4_session, se_r=

ef);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =A0 =A0 for (i =3D 0; i < ses->se_fchannel.maxreqs; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 struct nfsd4_cache_entry *e =3D &ses->s=

e_slots[i].sl_cache_entry;

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_release_respages(e->ce_respages, =

e->ce_resused);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 }
=A0 =A0 =A0 spin_lock(&nfsd_drc_lock);
=A0 =A0 =A0 nfsd_drc_mem_used -=3D ses->se_fchannel.maxreqs * NFS=

D_SLOT_CACHE_SIZE;

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines

=A0 =A0 =A0 spin_unlock(&nfsd_drc_lock);
+ =A0 =A0 free_session_slots(ses);
=A0 =A0 =A0 kfree(ses);
=A0}

fs/nfsd/nfs4state.c +0 -19
@@ -964,116 +981,32 @@ out_err:
 =A0 =A0 =A0 return;
 =A0}
 
-void
-nfsd4_set_statp(struct svc_rqst *rqstp, __be32 *statp)
-{
- =A0 =A0 struct nfsd4_compoundres *resp =3D rqstp->rq_resp;
-
- =A0 =A0 resp->cstate.statp =3D statp;
-}
-
-/*
- * Dereference the result pages.
- */
-static void
-nfsd4_release_respages(struct page **respages, short resused)
-{
- =A0 =A0 int i;
-
- =A0 =A0 dprintk("--> %s\n", __func__);
- =A0 =A0 for (i =3D 0; i < resused; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 if (!respages[i])
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

fs/nfsd/nfs4state.c out_err: · line 993

Quoted 8 lines
fs/nfsd/nfs4state.c +0 -8
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
- =A0 =A0 =A0 =A0 =A0 =A0 put_page(respages[i]);
- =A0 =A0 =A0 =A0 =A0 =A0 respages[i] =3D NULL;
- =A0 =A0 }
-}
-
-static void
-nfsd4_copy_pages(struct page **topages, struct page **frompages,=

short count)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +0 -5
-{
- =A0 =A0 int i;
-
- =A0 =A0 for (i =3D 0; i < count; i++) {
- =A0 =A0 =A0 =A0 =A0 =A0 topages[i] =3D frompages[i];
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfs4state.c +0 -6
- =A0 =A0 =A0 =A0 =A0 =A0 if (!topages[i])
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 continue;
- =A0 =A0 =A0 =A0 =A0 =A0 get_page(topages[i]);
- =A0 =A0 }
-}
-

=A0/*
- * Cache the reply pages up to NFSD_PAGES_PER_SLOT + 1, clearing=

the previous

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- * pages. We add a page to NFSD_PAGES_PER_SLOT for the case wher=

e the total

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- * length of the XDR response is less than se_fmaxresp_cached
- * (NFSD_PAGES_PER_SLOT * PAGE_SIZE) but the xdr_buf pages is us=

ed for a

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

- * of the reply (e.g. readdir).
- *
- * Store the base and length of the rq_req.head[0] page
- * of the NFSv4.1 data, just past the rpc header.
+ * Cache a reply. nfsd4_check_drc_limit() has bounded the cache =

size.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

=A0 */
=A0void
=A0nfsd4_store_cache_entry(struct nfsd4_compoundres *resp) =A0{
- =A0 =A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.slot-=

Quoted 9 lines

sl_cache_entry;

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

- =A0 =A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =A0 =A0 struct kvec *resv =3D &rqstp->rq_res.head[0]; -
- =A0 =A0 dprintk("--> %s entry %p\n", __func__, entry);
+ =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot;
+ =A0 =A0 unsigned int base;

- =A0 =A0 nfsd4_release_respages(entry->ce_respages, entry->ce_re=

sused);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- =A0 =A0 entry->ce_opcnt =3D resp->opcnt;
- =A0 =A0 entry->ce_status =3D resp->cstate.status;
+ =A0 =A0 dprintk("--> %s slot %p\n", __func__, slot);

- =A0 =A0 /*
- =A0 =A0 =A0* Don't need a page to cache just the sequence opera=

tion - the slot

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- =A0 =A0 =A0* does this for us!
- =A0 =A0 =A0*/
+ =A0 =A0 slot->sl_opcnt =3D resp->opcnt;
+ =A0 =A0 slot->sl_status =3D resp->cstate.status;

=A0 =A0 =A0 if (nfsd4_not_cached(resp)) {
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused =3D 0;
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_rpchdrlen =3D 0;
- =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s Just cache SEQUENCE. ce_cac=

hethis %d\n", __func__,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_c=

ache_entry.ce_cachethis);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 slot->sl_datalen =3D 0;
=A0 =A0 =A0 =A0 =A0 =A0 =A0 return;
=A0 =A0 =A0 }
- =A0 =A0 entry->ce_resused =3D rqstp->rq_resused;
- =A0 =A0 if (entry->ce_resused > NFSD_PAGES_PER_SLOT + 1)
- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused =3D NFSD_PAGES_PER_SL=

OT + 1;

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 nfsd4_copy_pages(entry->ce_respages, rqstp->rq_respages=

,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0entry->ce_resused);
- =A0 =A0 entry->ce_datav.iov_base =3D resp->cstate.statp;
- =A0 =A0 entry->ce_datav.iov_len =3D resv->iov_len - ((char *)re=

sp->cstate.statp -

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *=

)page_address(rqstp->rq_respages[0]));

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 /* Current request rpc header length*/
- =A0 =A0 entry->ce_rpchdrlen =3D (char *)resp->cstate.statp -
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *=

)page_address(rqstp->rq_respages[0]);

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -4
-}
-
-/*
- * We keep the rpc header, but take the nfs reply from the reply=

cache.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4state.c out_err:

Quoted 4 lines

- */

fs/nfsd/nfs4state.c +0 -3
-static int
-nfsd41_copy_replay_data(struct nfsd4_compoundres *resp,
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 struct nfsd4_cache_entr=

y *entry)

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

fs/nfsd/nfs4state.c

Quoted 8 lines
fs/nfsd/nfs4state.c +0 -8
-{
- =A0 =A0 struct svc_rqst *rqstp =3D resp->rqstp;
- =A0 =A0 struct kvec *resv =3D &resp->rqstp->rq_res.head[0];
- =A0 =A0 int len;
-
- =A0 =A0 /* Current request rpc header length*/
- =A0 =A0 len =3D (char *)resp->cstate.statp -
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 (char *)page_address(rq=

stp->rq_respages[0]);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =A0 =A0 if (entry->ce_datav.iov_len + len > PAGE_SIZE) {
- =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s v41 cached reply too large =

(%Zd).\n", __func__,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_datav.iov_len=

);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 return 0;
- =A0 =A0 }
- =A0 =A0 /* copy the cached reply nfsd data past the current rpc=

header */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 memcpy((char *)resv->iov_base + len, entry->ce_datav.io=

v_base,

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

- =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_datav.iov_len);
- =A0 =A0 resv->iov_len =3D len + entry->ce_datav.iov_len;
- =A0 =A0 return 1;
+ =A0 =A0 slot->sl_datalen =3D (char *)resp->p - (char *)resp->cs=

tate.datap;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =A0 =A0 base =3D (char *)resp->cstate.datap -
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0 =A0 (char *)resp->xbuf->head[0].iov_base;

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 if (read_bytes_from_xdr_buf(resp->xbuf, base, slot->sl_=

data,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

slot->sl_datalen))

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 printk(KERN_WARNING
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 "nfsd: sessions DRC cou=

ld not cache compound\n");

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I'd make this WARN("nfsd:...") just to make it completely clear it=

's a

Quoted 1 line
Quoted 1 line
Quoted 1 line

kernel bug. =A0(This case should be caught by nfsd4_check_drc_limi=

t unless

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

we've messed something up, right?)

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_co=

Quoted 5 lines

+ =A0 =A0 return;
=A0}

=A0/*

fs/nfsd/nfs4state.c +0 -0
@@ -1091,14 +1024,14 @@ nfsd4_enc_sequence_replay(struct nfsd4_co=

mpoundargs *args,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
fs/nfsd/nfs4state.c +0 -0
 =A0 =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot;
 

=A0 =A0 =A0 dprintk("--> %s resp->opcnt %d cachethis %u \n", __fu=

nc__,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 resp->opcnt, resp->cstate.slot->sl_cach=

e_entry.ce_cachethis);

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 resp->opcnt, resp->cstate.slot->sl_cach=

ethis);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=A0 =A0 =A0 /* Encode the replayed sequence operation */ =A0 =A0 =A0 op =3D &args->ops[resp->opcnt - 1]; =A0 =A0 =A0 nfsd4_encode_operation(resp, op);

=A0 =A0 =A0 /* Return nfserr_retry_uncached_rep in next operation=

=2E */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 if (args->opcnt > 1 && slot->sl_cache_entry.ce_cachethi=

s =3D=3D 0) {

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

+ =A0 =A0 if (args->opcnt > 1 && slot->sl_cachethis =3D=3D 0) { =A0 =A0 =A0 =A0 =A0 =A0 =A0 op =3D &args->ops[resp->opcnt++]; =A0 =A0 =A0 =A0 =A0 =A0 =A0 op->status =3D nfserr_retry_uncached_=

rep;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

fs/nfsd/nfs4state.c nfsd4_enc_sequence_replay(struct nfsd4_co=

Quoted 2 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_encode_operation(resp, op);

fs/nfsd/nfs4state.c +0 -0
@@ -1107,57 +1040,29 @@ nfsd4_enc_sequence_replay(struct nfsd4_co=

mpoundargs *args,

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +0 -0
 =A0}
 

=A0/*
- * Keep the first page of the replay. Copy the NFSv4.1 data from=

the first

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- * cached page. =A0Replace any futher replay pages from the cach=

e.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ * The sequence operation is not cached because we can use the s=

lot and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

+ * session values.
=A0 */
=A0__be32
=A0nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp, =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct nfsd4_seque=

nce *seq)

Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=A0{
- =A0 =A0 struct nfsd4_cache_entry *entry =3D &resp->cstate.slot-=

sl_cache_entry;

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

+ =A0 =A0 struct nfsd4_slot *slot =3D resp->cstate.slot; =A0 =A0 =A0 __be32 status;

- =A0 =A0 dprintk("--> %s entry %p\n", __func__, entry); -
- =A0 =A0 /*
- =A0 =A0 =A0* If this is just the sequence operation, we did not=

keep

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0* a page in the cache entry because we can just use =

the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0* slot info stored in struct nfsd4_sequence that was=

checked

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0* against the slot in nfsd4_sequence().
- =A0 =A0 =A0*
- =A0 =A0 =A0* This occurs when seq->cachethis is FALSE, or when =

the client

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0* session inactivity timer fires and a solo sequence=

operation

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- =A0 =A0 =A0* is sent (lease renewal).
- =A0 =A0 =A0*/
+ =A0 =A0 dprintk("--> %s slot %p\n", __func__, slot);

=A0 =A0 =A0 /* Either returns 0 or nfserr_retry_uncached */ =A0 =A0 =A0 status =3D nfsd4_enc_sequence_replay(resp->rqstp->rq_=

argp, resp);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=A0 =A0 =A0 if (status =3D=3D nfserr_retry_uncached_rep) =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;

- =A0 =A0 if (!nfsd41_copy_replay_data(resp, entry)) {
- =A0 =A0 =A0 =A0 =A0 =A0 /*
- =A0 =A0 =A0 =A0 =A0 =A0 =A0* Not enough room to use the replay =

rpc header, send the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0* cached header. Release all the all=

ocated result pages.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0*/
- =A0 =A0 =A0 =A0 =A0 =A0 svc_free_res_pages(resp->rqstp);
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_copy_pages(resp->rqstp->rq_respag=

es, entry->ce_respages,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 entry->ce_resused);
- =A0 =A0 } else {
- =A0 =A0 =A0 =A0 =A0 =A0 /* Release all but the first allocated =

result page */

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

fs/nfsd/nfs4state.c

Quoted 5 lines
fs/nfsd/nfs4state.c +0 -5
-
- =A0 =A0 =A0 =A0 =A0 =A0 resp->rqstp->rq_resused--;
- =A0 =A0 =A0 =A0 =A0 =A0 svc_free_res_pages(resp->rqstp);
-
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_copy_pages(&resp->rqstp->rq_respa=

ges[1],

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0&ent=

ry->ce_respages[1],

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0entr=

y->ce_resused - 1);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- =A0 =A0 }
+ =A0 =A0 /* The sequence operation has been encoded, cstate->dat=

ap set. */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 memcpy(resp->cstate.datap, slot->sl_data, slot->sl_data=

len);

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

- =A0 =A0 resp->rqstp->rq_resused =3D entry->ce_resused;
- =A0 =A0 resp->opcnt =3D entry->ce_opcnt;
- =A0 =A0 resp->cstate.iovlen =3D entry->ce_datav.iov_len + entry=

->ce_rpchdrlen;

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 status =3D entry->ce_status;
+ =A0 =A0 resp->opcnt =3D slot->sl_opcnt;
+ =A0 =A0 resp->p =3D resp->cstate.datap + XDR_QUADLEN(slot->sl_d=

atalen);

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

+ =A0 =A0 status =3D slot->sl_status;

=A0 =A0 =A0 return status;
=A0}

fs/nfsd/nfs4state.c +0 -0
@@ -1489,7 +1394,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 =A0 =A0 =A0 if (seq->slotid >=3D session->se_fchannel.maxreqs)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 goto out;
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
fs/nfsd/nfs4state.c +1 -1
- =A0 =A0 slot =3D &session->se_slots[seq->slotid];
+ =A0 =A0 slot =3D session->se_slots[seq->slotid];
 =A0 =A0 =A0 dprintk("%s: slotid %d\n", __func__, seq->slotid);
 

=A0 =A0 =A0 /* We do not negotiate the number of slots yet, so se=

t the

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
fs/nfsd/nfs4state.c +0 -0
@@ -1502,7 +1407,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->slot =3D slot;
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->session =3D session;

=A0 =A0 =A0 =A0 =A0 =A0 =A0 /* Return the cached reply status and=

set cstate->status

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0* for nfsd4_svc_encode_compoundres p=

rocessing */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0* for nfsd4_proc_compound processing=

*/

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 status =3D nfsd4_replay_cache_entry(r=

esp, seq);

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 cstate->status =3D nfserr_replay_cach=

e;

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 goto replay_cache;

fs/nfsd/nfs4state.c +0 -0
@@ -1513,7 +1418,7 @@ nfsd4_sequence(struct svc_rqst *rqstp,

=A0 =A0 =A0 /* Success! bump slot seqid */
=A0 =A0 =A0 slot->sl_inuse =3D true;
=A0 =A0 =A0 slot->sl_seqid =3D seq->seqid;
- =A0 =A0 slot->sl_cache_entry.ce_cachethis =3D seq->cachethis;
+ =A0 =A0 slot->sl_cachethis =3D seq->cachethis;

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

fs/nfsd/nfs4xdr.c nfsd4_encode_sequence(struct nfsd4_compound=

Quoted 7 lines

=A0 =A0 =A0 cstate->slot =3D slot;
=A0 =A0 =A0 cstate->session =3D session;

fs/nfsd/nfs4xdr.c +0 -0
index fdf632b..49824ea 100644
@@ -3064,6 +3064,7 @@ nfsd4_encode_sequence(struct nfsd4_compound=

res *resp, int nfserr,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfs=

Quoted 8 lines
fs/nfsd/nfs4xdr.c +1 -0
 =A0 =A0 =A0 WRITE32(0);
 
 =A0 =A0 =A0 ADJUST_ARGS();
+ =A0 =A0 resp->cstate.datap =3D p; /* DRC cache data pointer */
 =A0 =A0 =A0 return 0;
 =A0}
 
@@ -3166,7 +3167,7 @@ static int nfsd4_check_drc_limit(struct nfs=

d4_compoundres *resp)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fs/nfsd/nfs4xdr.c static int nfsd4_check_drc_limit(struct nfs= · line 3169

Quoted 4 lines
fs/nfsd/nfs4xdr.c +0 -1
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;
 
 =A0 =A0 =A0 session =3D resp->cstate.session;
- =A0 =A0 if (session =3D=3D NULL || slot->sl_cache_entry.ce_cach=

ethis =3D=3D 0)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqst =

Quoted 5 lines
fs/nfsd/nfs4xdr.c +1 -0
+ =A0 =A0 if (session =3D=3D NULL || slot->sl_cachethis =3D=3D 0)
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return status;
 
 =A0 =A0 =A0 if (resp->opcnt >=3D args->opcnt)
@@ -3291,6 +3292,7 @@ nfs4svc_encode_compoundres(struct svc_rqst =

*rqstp, __be32 *p, struct nfsd4_compo

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=A0 =A0 =A0 /*
=A0 =A0 =A0 =A0* All that remains is to write the tag and operati=

on count...

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfs4xdr.c nfs4svc_encode_compoundres(struct svc_rqs=

Quoted 6 lines

=A0 =A0 =A0 =A0*/
+ =A0 =A0 struct nfsd4_compound_state *cs =3D &resp->cstate; =A0 =A0 =A0 struct kvec *iov;
=A0 =A0 =A0 p =3D resp->tagp;
=A0 =A0 =A0 *p++ =3D htonl(resp->taglen);

fs/nfsd/nfs4xdr.c +0 -0
@@ -3304,14 +3306,11 @@ nfs4svc_encode_compoundres(struct svc_rqs=

t *rqstp, __be32 *p, struct nfsd4_compo

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
fs/nfsd/nfs4xdr.c +0 -0
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iov =3D &rqstp->rq_res.head[0];

=A0 =A0 =A0 iov->iov_len =3D ((char*)resp->p) - (char*)iov->iov_b=

ase;

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=A0 =A0 =A0 BUG_ON(iov->iov_len > PAGE_SIZE);
- =A0 =A0 if (nfsd4_has_session(&resp->cstate)) {
- =A0 =A0 =A0 =A0 =A0 =A0 if (resp->cstate.status =3D=3D nfserr_r=

eplay_cache &&

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 !nfsd4_=

not_cached(resp)) {

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iov->iov_len =3D resp->=

cstate.iovlen;

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- =A0 =A0 =A0 =A0 =A0 =A0 } else {
+ =A0 =A0 if (nfsd4_has_session(cs)) {
+ =A0 =A0 =A0 =A0 =A0 =A0 if (cs->status !=3D nfserr_replay_cache=

) {

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_store_cache_ent=

ry(resp);

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 dprintk("%s: SET SLOT=

STATE TO AVAILABLE\n", __func__);

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_i=

nuse =3D 0;

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 resp->cstate.slot->sl_i=

nuse =3D false;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=A0 =A0 =A0 =A0 =A0 =A0 =A0 }
=A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_put_session(resp->cstate.sessio=

n);

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/nfsd/nfssvc.c nfsd_dispatch(struct svc_rqst *rqstp, __be32=

Quoted 6 lines

=A0 =A0 =A0 }

fs/nfsd/nfssvc.c +0 -0
index d68cd05..944ef01 100644
@@ -576,10 +576,6 @@ nfsd_dispatch(struct svc_rqst *rqstp, __be32=

*statp)

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
fs/nfsd/nfssvc.c +0 -4
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 + rqstp->rq_res.head[0].iov_len;
 =A0 =A0 =A0 rqstp->rq_res.head[0].iov_len +=3D sizeof(__be32);
 
- =A0 =A0 /* NFSv4.1 DRC requires statp */
- =A0 =A0 if (rqstp->rq_vers =3D=3D 4)
- =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_set_statp(rqstp, statp);
-

=A0 =A0 =A0 /* Now call the procedure handler, and encode NFS sta=

tus. */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

=A0 =A0 =A0 nfserr =3D proc->pc_func(rqstp, rqstp->rq_argp, rqstp=

->rq_resp);

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

include/linux/nfsd/stat=

Quoted 2 lines

=A0 =A0 =A0 nfserr =3D map_new_errors(rqstp->rq_vers, nfserr);

include/linux/nfsd/stat= renamed from include/linux/nfsd/state.h +0 -0

e.h

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

index ff0b771..e745100 100644

include/linux/nfsd/state.h +0 -0
@@ -94,30 +94,23 @@ struct nfs4_cb_conn {
 

=A0/* Maximum number of slots per session. 160 is useful for long=

haul TCP */

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

=A0#define NFSD_MAX_SLOTS_PER_SESSION =A0 =A0 160

include/linux/nfsd/state.h +0 -3
-/* Maximum number of pages per slot cache entry */
-#define NFSD_PAGES_PER_SLOT =A01
-#define NFSD_SLOT_CACHE_SIZE =A0 =A0 =A0 =A0 PAGE_SIZE
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

=A0/* Maximum number of operations per session compound */ =A0#define NFSD_MAX_OPS_PER_COMPOUND =A0 =A016

include/linux/nfsd/state.h +2 -0
+/* Maximum =A0session per slot cache size */
+#define NFSD_SLOT_CACHE_SIZE =A0 =A0 =A0 =A0 1024

=A0/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */ =A0#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION =A0 =A032 =A0#define NFSD_MAX_MEM_PER_SESSION =A0\
=A0 =A0 =A0 =A0 =A0 =A0 =A0 (NFSD_CACHE_SIZE_SLOTS_PER_SESSION * =

NFSD_SLOT_CACHE_SIZE)

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-struct nfsd4_cache_entry {
- =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0ce_status;
- =A0 =A0 struct kvec =A0 =A0 ce_datav; /* encoded NFSv4.1 data i=

n rq_res.head[0] */

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 struct page =A0 =A0 *ce_respages[NFSD_PAGES_PER_SLOT + =

1];

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_cachethis;
- =A0 =A0 short =A0 =A0 =A0 =A0 =A0 ce_resused;
- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_opcnt;
- =A0 =A0 int =A0 =A0 =A0 =A0 =A0 =A0 ce_rpchdrlen;

include/linux/nfsd/state.h +0 -2
-};
-

=A0struct nfsd4_slot {
- =A0 =A0 bool =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0sl_inuse;

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0 sl_seqid;

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines

- =A0 =A0 struct nfsd4_cache_entry =A0 =A0 =A0 =A0sl_cache_entry;
+ =A0 =A0 bool =A0 =A0sl_inuse;
+ =A0 =A0 u32 =A0 =A0 sl_seqid;
+ =A0 =A0 int =A0 =A0 sl_cachethis;
+ =A0 =A0 int =A0 =A0 sl_opcnt;
+ =A0 =A0 __be32 =A0sl_status;
+ =A0 =A0 u32 =A0 =A0 sl_datalen;
+ =A0 =A0 char =A0 =A0sl_data[];

Could you just move sl_inuse to the end? =A0It'll save a few bytes=

in the

Quoted 1 line
Quoted 1 line
Quoted 1 line

structure (because the compiler will probably stick 3 bytes after =

it to

Quoted 6 lines
Quoted 6 lines
Quoted 1 line

align sl_seqid.)

How about this?

struct nfsd4_slot {
- =A0 =A0 =A0 bool =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0sl_inuse;

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=

=A0 sl_seqid;

Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 struct nfsd4_cache_entry =A0 =A0 =A0 =A0sl_cache_entry=

;

Quoted 10 lines
Quoted 8 lines

+ =A0 =A0 =A0 bool =A0 =A0sl_inuse;
+ =A0 =A0 =A0 bool =A0 =A0sl_cachethis;
+ =A0 =A0 =A0 u16 =A0 =A0 sl_opcnt;
+ =A0 =A0 =A0 u32 =A0 =A0 sl_seqid;
+ =A0 =A0 =A0 __be32 =A0sl_status;
+ =A0 =A0 =A0 u32 =A0 =A0 sl_datalen;
+ =A0 =A0 =A0 char =A0 =A0sl_data[];
=A0};

Looks OK to me! =A0By the way, on recent distro's you should be able =

to

Quoted 8 lines

run apt-get install/yum install dwarves, then

=A0 =A0 =A0 =A0pahole fs/nfsd/nfsd.o

and look through the output for the struct of interest:

struct nfsd4_slot {
=A0 =A0 =A0 =A0bool =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 sl_in=

use; =A0 =A0 =A0 =A0 =A0 =A0 /* =A0 =A0 0 =A0 =A0 1 */

Quoted 4 lines

=A0 =A0 =A0 =A0/* XXX 3 bytes hole, try to pack */

=A0 =A0 =A0 =A0u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0sl_=

seqid; =A0 =A0 =A0 =A0 =A0 =A0 /* =A0 =A0 4 =A0 =A0 4 */

Quoted 1 line

=A0 =A0 =A0 =A0struct nfsd4_cache_entry =A0 sl_cache_entry; =A0 =A0 =A0=

/* =A0 =A0 8 =A0 =A036 */

Quoted 8 lines

=A0 =A0 =A0 =A0/* size: 44, cachelines: 1 */
=A0 =A0 =A0 =A0/* sum members: 41, holes: 1, sum holes: 3 */ =A0 =A0 =A0 =A0/* last cacheline: 44 bytes */
}; =A0 =A0 =A0/* definitions: 3 */

(But no point obsessing over this kind of thing; I just happened to notice the problem in this patch and figured for such a common struct=

ure

Quoted 1 line

it might be worth mentioning.)

Cool! - thanks for the pointer

-->Andy

Quoted 8 lines

--b.

Quoted 5 lines

-->Andy

Quoted 2 lines

--b.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

include/linux/nfsd/state.h struct nfsd4_session { · line 162

Quoted 8 lines

=A0};

=A0struct nfsd4_channel_attrs {

include/linux/nfsd/state.h +0 -1
@@ -159,7 +152,7 @@ struct nfsd4_session {
 =A0 =A0 =A0 struct nfs4_sessionid =A0 se_sessionid;
 =A0 =A0 =A0 struct nfsd4_channel_attrs se_fchannel;
 =A0 =A0 =A0 struct nfsd4_channel_attrs se_bchannel;
- =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 se_slots[]; =A0 =A0 /* fo=

rward channel slots */

Quoted 1 line
Quoted 1 line
Quoted 1 line

include/linux/nfsd/state.h struct nfsd4_session { · line 155

Quoted 1 line
include/linux/nfsd/state.h +1 -0
+ =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 *se_slots[]; =A0 =A0/* fo=

rward channel slots */

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

include/linux/nfsd/xdr4.=

Quoted 4 lines
include/linux/nfsd/state.h +0 -0
 =A0};
 

=A0static inline void

include/linux/nfsd/xdr4.= renamed from include/linux/nfsd/xdr4.h +0 -0

h

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

index 3f71660..73164c2 100644

include/linux/nfsd/xdr4.h +0 -0
@@ -51,7 +51,7 @@ struct nfsd4_compound_state {

=A0 =A0 =A0 /* For sessions DRC */
=A0 =A0 =A0 struct nfsd4_session =A0 =A0*session;
=A0 =A0 =A0 struct nfsd4_slot =A0 =A0 =A0 *slot;
- =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*statp;
+ =A0 =A0 __be32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*datap; =A0 =A0 =A0 size_t =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0iovlen; =A0 =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 minorvers=

ion;

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

include/linux/nfsd/xdr4.h static inline bool nfsd4_is_solo_sequence(str=

Quoted 2 lines

=A0 =A0 =A0 u32 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 status;

include/linux/nfsd/xdr4.h +0 -0
@@ -472,8 +472,7 @@ static inline bool nfsd4_is_solo_sequence(str=

uct nfsd4_compoundres *resp)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=A0static inline bool nfsd4_not_cached(struct nfsd4_compoundres *=

resp)

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=A0{
- =A0 =A0 return !resp->cstate.slot->sl_cache_entry.ce_cachethis =

||

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfsd4_is_solo_sequence(=

resp);

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

+ =A0 =A0 return !resp->cstate.slot->sl_cachethis || nfsd4_is_sol=

o_sequence(resp);

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

=A0}

=A0#define NFS4_SVC_XDRSIZE =A0 =A0 =A0 =A0 =A0 =A0 sizeof(struct=

nfsd4_compoundargs)

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

--
1.6.2.5

_______________________________________________
pNFS mailing list
[email protected]
http://linux-nfs.org/cgi-bin/mailman/listinfo/pnfs

#14 lore raw

Re: [PATCH 2/5] nfsd41: bound forechannel drc size by memory usage

In reply to [email protected]
Quoted 3 lines · On Thu, Aug 27, 2009 at 12:07 PM, <[email protected]> wrote:

From: Andy Adamson <[email protected]>

By using the requested ca_maxresponsesize_cached * ca_maxresponses to=

bound

Quoted 1 line

a forechannel drc request size, clients can tailor a session to usage=

=2E

Quoted 3 lines

For example, an I/O session (READ/WRITE only) can have a much smaller ca_maxresponsesize_cached (for only WRITE compound responses) and a l=

ot larger

Quoted 5 lines

ca_maxresponses to service a large in-flight data window.

---
=A0fs/nfsd/nfs4state.c =A0 =A0 =A0 =A0| =A0 60 ++++++++++++++++++++++=

+++++++++------------

Quoted 10 lines

=A0include/linux/nfsd/state.h | =A0 =A08 ++++- =A02 files changed, 49 insertions(+), 19 deletions(-)

fs/nfsd/nfs4state.c +0 -0
index ddfd36f..a691139 100644
@@ -414,34 +414,60 @@ gen_sessionid(struct nfsd4_session *ses)
 =A0}
 
Quoted 4 lines

=A0/*
- * Give the client the number of slots it requests bound by
- * NFSD_MAX_SLOTS_PER_SESSION and by nfsd_drc_max_mem.
+ * 32 bytes of RPC header and 44 bytes of sequence operation respons=

e

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses)

Quoted 6 lines

+ * not included in NFSD_SLOT_CACHE_SIZE
+ * */

fs/nfsd/nfs4state.c +4 -0
+#define NFSD_MIN_HDR_SEQ_SZ =A0(32 + 44)
+
+/*
+ * Give the client the number of ca_maxresponsesize_cached slots it =

requests

Quoted 2 lines

+ * bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by
+ * nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSI=

ON.

Quoted 3 lines

+ *
+ * The ca_maxresponssize_cached definition includes the size
+ * of the rpc header with the variable length security flavor creden=

tial

Quoted 4 lines

+ * plus verifier (32 bytes with AUTH_SYS and NULL verifier)
+ * as well as the encoded SEQUENCE operation response (44 bytes)
+ * which are not included in NFSD_SLOT_CACHE_SIZE.
+ * We err on the side of being a bit small when AUTH_SYS/NULL verifi=

er

Quoted 3 lines

+ * is not used.
=A0*
=A0* If we run out of reserved DRC memory we should (up to a point) r=

e-negotiate

Quoted 1 line

=A0* active sessions and reduce their slot usage to make rooom for ne=

w

Quoted 3 lines

=A0* connections. For now we just fail the create session. =A0*/
-static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan=

)

Quoted 1 line

+static int set_forechannel_drc_size(struct nfsd4_channel_attrs *fcha=

n)

Quoted 3 lines

=A0{
- =A0 =A0 =A0 int mem;
+ =A0 =A0 =A0 int mem, size =3D fchan->maxresp_cached - NFSD_MIN_HDR_=

SEQ_SZ;

Quoted 6 lines

- =A0 =A0 =A0 if (fchan->maxreqs < 1)
+ =A0 =A0 =A0 if (fchan->maxreqs < 1 || size <=3D 0)
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0return nfserr_inval;
- =A0 =A0 =A0 else if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 fchan->maxreqs =3D NFSD_MAX_SLOTS_PER_S=

ESSION;

Quoted 9 lines

- =A0 =A0 =A0 mem =3D fchan->maxreqs * NFSD_SLOT_CACHE_SIZE;
+ =A0 =A0 =A0 if (size > NFSD_SLOT_CACHE_SIZE)
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 size =3D NFSD_SLOT_CACHE_SIZE; +
+ =A0 =A0 =A0 /* bound the maxreqs by NFSD_MAX_MEM_PER_SESSION */
+ =A0 =A0 =A0 mem =3D fchan->maxreqs * size;
+ =A0 =A0 =A0 if (mem > NFSD_MAX_MEM_PER_SESSION) {
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 fchan->maxreqs =3D NFSD_MAX_MEM_PER_SES=

SION / size;

Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (fchan->maxreqs > NFSD_MAX_SLOTS_PER=

_SESSION)

Quoted 1 line

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 fchan->maxreqs =3D NFSD=

_MAX_SLOTS_PER_SESSION;

Quoted 6 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 mem =3D fchan->maxreqs * size;
+ =A0 =A0 =A0 }

=A0 =A0 =A0 =A0spin_lock(&nfsd_drc_lock);
- =A0 =A0 =A0 if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem)
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 mem =3D ((nfsd_drc_max_mem - nfsd_drc_m=

em_used) /

Quoted 1 line

- =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 NFSD_SL=

OT_CACHE_SIZE) * NFSD_SLOT_CACHE_SIZE;

Quoted 3 lines

+ =A0 =A0 =A0 /* bound the total session drc memory ussage */
+ =A0 =A0 =A0 if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem) {
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 fchan->maxreqs =3D (nfsd_drc_max_mem - =

nfsd_drc_mem_used) / size;

Quoted 9 lines

+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 mem =3D fchan->maxreqs * size;
+ =A0 =A0 =A0 }
=A0 =A0 =A0 =A0nfsd_drc_mem_used +=3D mem;
=A0 =A0 =A0 =A0spin_unlock(&nfsd_drc_lock);

- =A0 =A0 =A0 fchan->maxreqs =3D mem / NFSD_SLOT_CACHE_SIZE; =A0 =A0 =A0 =A0if (fchan->maxreqs =3D=3D 0)
- =A0 =A0 =A0 =A0 =A0 =A0 =A0 return nfserr_resource;
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 return nfserr_serverfault;

               ^^^^^^^^^^^^^^^^^^^

This change belongs in patch [PATCH 4/5] nfsd41: replace nfserr_resource in pure nfs41 responses

-->Andy

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_rqst=

Quoted 6 lines

+
+ =A0 =A0 =A0 fchan->maxresp_cached =3D size + NFSD_MIN_HDR_SEQ_SZ; =A0 =A0 =A0 =A0return 0;
=A0}

fs/nfsd/nfs4state.c +0 -0
@@ -466,9 +492,6 @@ static int init_forechannel_attrs(struct svc_rqst=

*rqstp,

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_rqst= · line 470

Quoted 5 lines
fs/nfsd/nfs4state.c +0 -2
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fchan->maxresp_sz =3D maxcount;
 =A0 =A0 =A0 =A0session_fchan->maxresp_sz =3D fchan->maxresp_sz;
 
- =A0 =A0 =A0 session_fchan->maxresp_cached =3D NFSD_SLOT_CACHE_SIZE;
- =A0 =A0 =A0 fchan->maxresp_cached =3D session_fchan->maxresp_cached=

;

Quoted 4 lines
fs/nfsd/nfs4state.c +0 -1
-

=A0 =A0 =A0 =A0/* Use the client's maxops if possible */ =A0 =A0 =A0 =A0if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0fchan->maxops =3D NFSD_MAX_OPS_PER_COM=

POUND;

fs/nfsd/nfs4state.c static int init_forechannel_attrs(struct svc_rqs=

Quoted 1 line
fs/nfsd/nfs4state.c +0 -0
@@ -478,9 +501,12 @@ static int init_forechannel_attrs(struct svc_rqs=

t *rqstp,

Quoted 1 line

=A0 =A0 =A0 =A0 * recover pages from existing sessions. For now fail =

session

Quoted 6 lines

=A0 =A0 =A0 =A0 * creation.
=A0 =A0 =A0 =A0 */
- =A0 =A0 =A0 status =3D set_forechannel_maxreqs(fchan);
+ =A0 =A0 =A0 status =3D set_forechannel_drc_size(fchan);

+ =A0 =A0 =A0 session_fchan->maxresp_cached =3D fchan->maxresp_cached=

;

Quoted 11 lines

=A0 =A0 =A0 =A0session_fchan->maxreqs =3D fchan->maxreqs; +
+ =A0 =A0 =A0 dprintk("%s status %d\n", __func__, status); =A0 =A0 =A0 =A0return status;
=A0}

include/linux/nfsd/state.h +0 -0
index fb0c404..ff0b771 100644

include/linux/nfsd/state.h struct nfs4_cb_conn { · line 95

Quoted 4 lines
include/linux/nfsd/state.h +0 -1
@@ -92,13 +92,17 @@ struct nfs4_cb_conn {
 =A0 =A0 =A0 =A0struct rpc_cred * =A0 =A0 =A0 cb_cred;
 =A0};
 
-/* Maximum number of slots per session. 128 is useful for long haul =

TCP */

include/linux/nfsd/state.h struct nfs4_cb_conn { · line 95

Quoted 2 lines
include/linux/nfsd/state.h +1 -1
-#define NFSD_MAX_SLOTS_PER_SESSION =A0 =A0 128
+/* Maximum number of slots per session. 160 is useful for long haul =

TCP */

include/linux/nfsd/state.h

Quoted 10 lines
include/linux/nfsd/state.h +1 -0
+#define NFSD_MAX_SLOTS_PER_SESSION =A0 =A0 160

=A0/* Maximum number of pages per slot cache entry */ =A0#define NFSD_PAGES_PER_SLOT =A0 =A01
=A0#define NFSD_SLOT_CACHE_SIZE =A0 =A0 =A0 =A0 =A0 PAGE_SIZE =A0/* Maximum number of operations per session compound */ =A0#define NFSD_MAX_OPS_PER_COMPOUND =A0 =A0 =A016

include/linux/nfsd/state.h +4 -0
+/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
+#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION =A0 =A0 =A032
+#define NFSD_MAX_MEM_PER_SESSION =A0\
+ =A0 =A0 =A0 =A0 =A0 =A0 =A0 (NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NF=

SD_SLOT_CACHE_SIZE)

Quoted 8 lines

=A0struct nfsd4_cache_entry {
=A0 =A0 =A0 =A0__be32 =A0 =A0 =A0 =A0 =A0ce_status; --
1.6.2.5

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" =

in

Quoted 3 lines

the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html

#15 lore raw

Re: [PATCH 2/5] nfsd41: bound forechannel drc size by memory usage

In reply to [email protected]

Looks basically fine, but there are a few nitpicky problems:

Quoted 20 lines · On Thu, Aug 27, 2009 at 12:07:41PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

By using the requested ca_maxresponsesize_cached * ca_maxresponses to bound a forechannel drc request size, clients can tailor a session to usage.

For example, an I/O session (READ/WRITE only) can have a much smaller ca_maxresponsesize_cached (for only WRITE compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

---
 fs/nfsd/nfs4state.c        |   60 +++++++++++++++++++++++++++++++------------
 include/linux/nfsd/state.h |    8 ++++-
 2 files changed, 49 insertions(+), 19 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index ddfd36f..a691139 100644

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 420

Quoted 9 lines
fs/nfsd/nfs4state.c +4 -2
@@ -414,34 +414,60 @@ gen_sessionid(struct nfsd4_session *ses)
 }
 
 /*
- * Give the client the number of slots it requests bound by
- * NFSD_MAX_SLOTS_PER_SESSION and by nfsd_drc_max_mem.
+ * 32 bytes of RPC header and 44 bytes of sequence operation response
+ * not included in NFSD_SLOT_CACHE_SIZE
+ * */
+#define NFSD_MIN_HDR_SEQ_SZ  (32 + 44)

I took a look at a trace and got 24 for the rpc header (xid through accept state), 12 for compound header (status, empty tag, opcount), and 44 for the sequence response (opcode through status flags), 80 together--could you double check this?

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 429

Quoted 9 lines
fs/nfsd/nfs4state.c +9 -0
+
+/*
+ * Give the client the number of ca_maxresponsesize_cached slots it requests
+ * bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by
+ * nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.
+ *
+ * The ca_maxresponssize_cached definition includes the size
+ * of the rpc header with the variable length security flavor credential
+ * plus verifier (32 bytes with AUTH_SYS and NULL verifier)

Note there's no credential in the response, just a verifier. My
attempt at these two comments:

/*
* The protocol defines ca_maxresponssize_cached to include the size of
* the rpc header, but all we need to cache is the data starting after
* the end of the initial SEQUENCE operation--the rest we regenerate
* each time. Therefore we can advertise a ca_maxresponssize_cached
* value that is the number of bytes in our cache plus a few additional
* bytes. In order to stay on the safe side, and not promise more than
* we can cache, those additional bytes must be the minimum possible: 24
* bytes of rpc header (xid through accept state, with AUTH_NULL
* verifier), 12 for the compound header (with zero-length tag), and 44
* for the SEQUENCE op response:
*/
#define NFSD_MIN_HDR_SEQ_SZ (24 + 12 + 44)

/*
* Give the client the number of ca_maxresponsesize_cached slots it
* requests, of size bounded by NFSD_SLOT_CACHE_SIZE,
* NFSD_MAX_MEM_PER_SESSION, and nfsd_drc_max_mem. Do not allow more
* than NFSD_MAX_SLOTS_PER_SESSION.
*
* If we run out of reserved DRC memory we should (up to a point)
* re-negotiate active sessions and reduce their slot usage to make
* rooom for new connections. For now we just fail the create session. */

Does that look right?

Quoted 8 lines
fs/nfsd/nfs4state.c +4 -0
+ * as well as the encoded SEQUENCE operation response (44 bytes)
+ * which are not included in NFSD_SLOT_CACHE_SIZE.
+ * We err on the side of being a bit small when AUTH_SYS/NULL verifier
+ * is not used.
  *
  * If we run out of reserved DRC memory we should (up to a point) re-negotiate
  * active sessions and reduce their slot usage to make rooom for new
  * connections. For now we just fail the create session.

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 444

Quoted 10 lines
fs/nfsd/nfs4state.c +3 -3
  */
-static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
+static int set_forechannel_drc_size(struct nfsd4_channel_attrs *fchan)
 {
-	int mem;
+	int mem, size = fchan->maxresp_cached - NFSD_MIN_HDR_SEQ_SZ;
 
-	if (fchan->maxreqs < 1)
+	if (fchan->maxreqs < 1 || size <= 0)
 		return nfserr_inval;

Is it really illegal for the client to request a maxresp_cached less than NFSD_MIN_HDR_SEQ_SIZE? I think this should just be rounded up to zero.

Also watch out for overflow: if the client gives an extremely large value for maxresp_cached then we should round it down rather than doing arithmetic that might result in overflow.

Simplest might be to first round maxresp_cached up to NFSD_MIN_HDR_SEQ_SIZE, if it's less. *Then* subtract NFSD_MIN_HDR_SEQ_SIZE. Then round down to NFSD_SLOT_CACHE_SIZE, if it's too large. And keep all of this signed.

Quoted 22 lines
fs/nfsd/nfs4state.c +13 -6
-	else if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
-		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
 
-	mem = fchan->maxreqs * NFSD_SLOT_CACHE_SIZE;
+	if (size > NFSD_SLOT_CACHE_SIZE)
+		size = NFSD_SLOT_CACHE_SIZE;
+
+	/* bound the maxreqs by NFSD_MAX_MEM_PER_SESSION */
+	mem = fchan->maxreqs * size;
+	if (mem > NFSD_MAX_MEM_PER_SESSION) {
+		fchan->maxreqs = NFSD_MAX_MEM_PER_SESSION / size;
+		if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
+			fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
+		mem = fchan->maxreqs * size;
+	}
 
 	spin_lock(&nfsd_drc_lock);
-	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem)
-		mem = ((nfsd_drc_max_mem - nfsd_drc_mem_used) /
-				NFSD_SLOT_CACHE_SIZE) * NFSD_SLOT_CACHE_SIZE;
+	/* bound the total session drc memory ussage */
+	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem) {

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 468

Quoted 10 lines
fs/nfsd/nfs4state.c +4 -2
+		fchan->maxreqs = (nfsd_drc_max_mem - nfsd_drc_mem_used) / size;
+		mem = fchan->maxreqs * size;
+	}
 	nfsd_drc_mem_used += mem;
 	spin_unlock(&nfsd_drc_lock);
 
-	fchan->maxreqs = mem / NFSD_SLOT_CACHE_SIZE;
 	if (fchan->maxreqs == 0)
-		return nfserr_resource;
+		return nfserr_serverfault;

Remind me why serverfault and not resource here?

--b.

Quoted 56 lines
fs/nfsd/nfs4state.c +6 -4
+
+	fchan->maxresp_cached = size + NFSD_MIN_HDR_SEQ_SZ;
 	return 0;
 }
 
@@ -466,9 +492,6 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 		fchan->maxresp_sz = maxcount;
 	session_fchan->maxresp_sz = fchan->maxresp_sz;
 
-	session_fchan->maxresp_cached = NFSD_SLOT_CACHE_SIZE;
-	fchan->maxresp_cached = session_fchan->maxresp_cached;
-
 	/* Use the client's maxops if possible */
 	if (fchan->maxops > NFSD_MAX_OPS_PER_COMPOUND)
 		fchan->maxops = NFSD_MAX_OPS_PER_COMPOUND;
@@ -478,9 +501,12 @@ static int init_forechannel_attrs(struct svc_rqst *rqstp,
 	 * recover pages from existing sessions. For now fail session
 	 * creation.
 	 */
-	status = set_forechannel_maxreqs(fchan);
+	status = set_forechannel_drc_size(fchan);
 
+	session_fchan->maxresp_cached = fchan->maxresp_cached;
 	session_fchan->maxreqs = fchan->maxreqs;
+
+	dprintk("%s status %d\n", __func__, status);
 	return status;
 }
 
include/linux/nfsd/state.h +6 -2
index fb0c404..ff0b771 100644
@@ -92,13 +92,17 @@ struct nfs4_cb_conn {
 	struct rpc_cred	*	cb_cred;
 };
 
-/* Maximum number of slots per session. 128 is useful for long haul TCP */
-#define NFSD_MAX_SLOTS_PER_SESSION	128
+/* Maximum number of slots per session. 160 is useful for long haul TCP */
+#define NFSD_MAX_SLOTS_PER_SESSION     160
 /* Maximum number of pages per slot cache entry */
 #define NFSD_PAGES_PER_SLOT	1
 #define NFSD_SLOT_CACHE_SIZE		PAGE_SIZE
 /* Maximum number of operations per session compound */
 #define NFSD_MAX_OPS_PER_COMPOUND	16
+/* Maximum number of NFSD_SLOT_CACHE_SIZE slots per session */
+#define NFSD_CACHE_SIZE_SLOTS_PER_SESSION	32
+#define NFSD_MAX_MEM_PER_SESSION  \
+		(NFSD_CACHE_SIZE_SLOTS_PER_SESSION * NFSD_SLOT_CACHE_SIZE)
 
 struct nfsd4_cache_entry {
 	__be32		ce_status;
Signature · 1 line
1.6.2.5
#16 lore raw

Re: [PATCH 2/5] nfsd41: bound forechannel drc size by memory usage

Quoted 137 lines · bfields

Looks basically fine, but there are a few nitpicky problems:

Quoted 20 lines · On Thu, Aug 27, 2009 at 12:07:41PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

By using the requested ca_maxresponsesize_cached * ca_maxresponses to bound a forechannel drc request size, clients can tailor a session to usage.

For example, an I/O session (READ/WRITE only) can have a much smaller ca_maxresponsesize_cached (for only WRITE compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

---
 fs/nfsd/nfs4state.c        |   60 +++++++++++++++++++++++++++++++------------
 include/linux/nfsd/state.h |    8 ++++-
 2 files changed, 49 insertions(+), 19 deletions(-)
fs/nfsd/nfs4state.c +0 -0
index ddfd36f..a691139 100644

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 420

Quoted 9 lines
fs/nfsd/nfs4state.c +4 -2
@@ -414,34 +414,60 @@ gen_sessionid(struct nfsd4_session *ses)
 }
 
 /*
- * Give the client the number of slots it requests bound by
- * NFSD_MAX_SLOTS_PER_SESSION and by nfsd_drc_max_mem.
+ * 32 bytes of RPC header and 44 bytes of sequence operation response
+ * not included in NFSD_SLOT_CACHE_SIZE
+ * */
+#define NFSD_MIN_HDR_SEQ_SZ  (32 + 44)

I took a look at a trace and got 24 for the rpc header (xid through accept state), 12 for compound header (status, empty tag, opcount), and 44 for the sequence response (opcode through status flags), 80 together--could you double check this?

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 429

Quoted 9 lines
fs/nfsd/nfs4state.c +9 -0
+
+/*
+ * Give the client the number of ca_maxresponsesize_cached slots it requests
+ * bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by
+ * nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.
+ *
+ * The ca_maxresponssize_cached definition includes the size
+ * of the rpc header with the variable length security flavor credential
+ * plus verifier (32 bytes with AUTH_SYS and NULL verifier)

Note there's no credential in the response, just a verifier. My
attempt at these two comments:

/*
* The protocol defines ca_maxresponssize_cached to include the size of
* the rpc header, but all we need to cache is the data starting after
* the end of the initial SEQUENCE operation--the rest we regenerate
* each time. Therefore we can advertise a ca_maxresponssize_cached
* value that is the number of bytes in our cache plus a few additional
* bytes. In order to stay on the safe side, and not promise more than
* we can cache, those additional bytes must be the minimum possible: 24
* bytes of rpc header (xid through accept state, with AUTH_NULL
* verifier), 12 for the compound header (with zero-length tag), and 44
* for the SEQUENCE op response:
*/
#define NFSD_MIN_HDR_SEQ_SZ (24 + 12 + 44)

/*
* Give the client the number of ca_maxresponsesize_cached slots it
* requests, of size bounded by NFSD_SLOT_CACHE_SIZE,
* NFSD_MAX_MEM_PER_SESSION, and nfsd_drc_max_mem. Do not allow more
* than NFSD_MAX_SLOTS_PER_SESSION.
*
* If we run out of reserved DRC memory we should (up to a point)
* re-negotiate active sessions and reduce their slot usage to make
* rooom for new connections. For now we just fail the create session. */

Does that look right?

Quoted 8 lines
fs/nfsd/nfs4state.c +4 -0
+ * as well as the encoded SEQUENCE operation response (44 bytes)
+ * which are not included in NFSD_SLOT_CACHE_SIZE.
+ * We err on the side of being a bit small when AUTH_SYS/NULL verifier
+ * is not used.
  *
  * If we run out of reserved DRC memory we should (up to a point) re-negotiate
  * active sessions and reduce their slot usage to make rooom for new
  * connections. For now we just fail the create session.

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 444

Quoted 10 lines
fs/nfsd/nfs4state.c +3 -3
  */
-static int set_forechannel_maxreqs(struct nfsd4_channel_attrs *fchan)
+static int set_forechannel_drc_size(struct nfsd4_channel_attrs *fchan)
 {
-	int mem;
+	int mem, size = fchan->maxresp_cached - NFSD_MIN_HDR_SEQ_SZ;
 
-	if (fchan->maxreqs < 1)
+	if (fchan->maxreqs < 1 || size <= 0)
 		return nfserr_inval;

Is it really illegal for the client to request a maxresp_cached less than NFSD_MIN_HDR_SEQ_SIZE? I think this should just be rounded up to zero.

Also watch out for overflow: if the client gives an extremely large value for maxresp_cached then we should round it down rather than doing arithmetic that might result in overflow.

Simplest might be to first round maxresp_cached up to NFSD_MIN_HDR_SEQ_SIZE, if it's less. *Then* subtract NFSD_MIN_HDR_SEQ_SIZE. Then round down to NFSD_SLOT_CACHE_SIZE, if it's too large. And keep all of this signed.

Quoted 28 lines
fs/nfsd/nfs4state.c +16 -6
-	else if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
-		fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
 
-	mem = fchan->maxreqs * NFSD_SLOT_CACHE_SIZE;
+	if (size > NFSD_SLOT_CACHE_SIZE)
+		size = NFSD_SLOT_CACHE_SIZE;
+
+	/* bound the maxreqs by NFSD_MAX_MEM_PER_SESSION */
+	mem = fchan->maxreqs * size;
+	if (mem > NFSD_MAX_MEM_PER_SESSION) {
+		fchan->maxreqs = NFSD_MAX_MEM_PER_SESSION / size;
+		if (fchan->maxreqs > NFSD_MAX_SLOTS_PER_SESSION)
+			fchan->maxreqs = NFSD_MAX_SLOTS_PER_SESSION;
+		mem = fchan->maxreqs * size;
+	}
 
 	spin_lock(&nfsd_drc_lock);
-	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem)
-		mem = ((nfsd_drc_max_mem - nfsd_drc_mem_used) /
-				NFSD_SLOT_CACHE_SIZE) * NFSD_SLOT_CACHE_SIZE;
+	/* bound the total session drc memory ussage */
+	if (mem + nfsd_drc_mem_used > nfsd_drc_max_mem) {
+		fchan->maxreqs = (nfsd_drc_max_mem - nfsd_drc_mem_used) / size;
+		mem = fchan->maxreqs * size;
+	}
 	nfsd_drc_mem_used += mem;
 	spin_unlock(&nfsd_drc_lock);
 
Quoted 6 lines

fs/nfsd/nfs4state.c gen_sessionid(struct nfsd4_session *ses) · line 468

Quoted 4 lines
fs/nfsd/nfs4state.c +1 -2
-	fchan->maxreqs = mem / NFSD_SLOT_CACHE_SIZE;
 	if (fchan->maxreqs == 0)
-		return nfserr_resource;
+		return nfserr_serverfault;

Remind me why serverfault and not resource here?

Whoops, OK, I see that one's answered later--it's not a legal error any more.

--b.

#17 lore raw

Re: [pnfs] [PATCH 2/5] nfsd41: bound forechannel drc size by memory usage

In reply to J. Bruce Fields
Quoted 4 lines · J. Bruce Fields

Remind me why serverfault and not resource here?

--b.

NFS4ERR_RESOURCE is not a valid nfsv4.1 error.

Benny

#18 lore raw

Re: [PATCH 1/5] nfsd41: expand solo sequence check

In reply to [email protected]
Quoted 4 lines · On Thu, Aug 27, 2009 at 12:07:40PM -0400, [email protected] wrote:

From: Andy Adamson <[email protected]>

Failed sequenece == solo sequence.
Remove redundant sequence operation cache checks.

Applied, thanks.

--b.

Quoted 54 lines
---
 fs/nfsd/nfs4state.c       |    9 ---------
 include/linux/nfsd/xdr4.h |    2 +-
 2 files changed, 1 insertions(+), 10 deletions(-)
fs/nfsd/nfs4state.c +0 -9
index d2a0524..ddfd36f 100644
@@ -991,16 +991,10 @@ nfsd4_store_cache_entry(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_cache_entry *entry = &resp->cstate.slot->sl_cache_entry;
 	struct svc_rqst *rqstp = resp->rqstp;
-	struct nfsd4_compoundargs *args = rqstp->rq_argp;
-	struct nfsd4_op *op = &args->ops[resp->opcnt];
 	struct kvec *resv = &rqstp->rq_res.head[0];
 
 	dprintk("--> %s entry %p\n", __func__, entry);
 
-	/* Don't cache a failed OP_SEQUENCE. */
-	if (resp->opcnt == 1 && op->opnum == OP_SEQUENCE && resp->cstate.status)
-		return;
-
 	nfsd4_release_respages(entry->ce_respages, entry->ce_resused);
 	entry->ce_opcnt = resp->opcnt;
 	entry->ce_status = resp->cstate.status;
@@ -1490,9 +1484,6 @@ nfsd4_sequence(struct svc_rqst *rqstp,
 	slot->sl_inuse = true;
 	slot->sl_seqid = seq->seqid;
 	slot->sl_cache_entry.ce_cachethis = seq->cachethis;
-	/* Always set the cache entry cachethis for solo sequence */
-	if (nfsd4_is_solo_sequence(resp))
-		slot->sl_cache_entry.ce_cachethis = 1;
 
 	cstate->slot = slot;
 	cstate->session = session;
include/linux/nfsd/xdr4.h +1 -1
index 5e4beb0..3f71660 100644
@@ -467,7 +467,7 @@ struct nfsd4_compoundres {
 static inline bool nfsd4_is_solo_sequence(struct nfsd4_compoundres *resp)
 {
 	struct nfsd4_compoundargs *args = resp->rqstp->rq_argp;
-	return args->opcnt == 1;
+	return resp->opcnt == 1 && args->ops[0].opnum == OP_SEQUENCE;
 }
 
 static inline bool nfsd4_not_cached(struct nfsd4_compoundres *resp)
Signature · 1 line
1.6.2.5
#19 lore raw
In reply to [email protected]
Quoted 6 lines · On Thu, Aug 27, 2009 at 12:07:39PM -0400, [email protected] wrote:

NFSv4.1 DRC Rewrite Version 6

These patches apply against git://linux-nfs.org/~bfields/linux: for-2.6.32 branch and continue to rewrite the NFSv4.1 Sessions DRC. Besides some bug fixes:

Thanks for following up on this. It looks fine except for a few nits noted on replies to individual patches. Could you fix those up and resend? And then I think #7 should be it....

--b.

Quoted 32 lines

1) The bound on the fore channel per-session DRC size are rewritten. Instead of just using a maximum number of slots to bound the size, The server gives the client the number of ca_maxresponsesize_cached slots it requests bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.

This allows clients to tailor a session to usage.
For example, an I/O session (READ/WRITE/COMMIT only) can have a much smaller ca_maxresponsesize_cached (for only WRITE/COMMIt compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

2) the page-based DRC is replaced with a buffer based DRC with each slot table entry (struct nfsd4_slot + cache) allocated separately. This allocation prepares us for slot size re-negotiation via the SEQUENCE operation target and high slot id arguments.

Testing:

NFSv4.1 mount: pynfs tests - including the SEQUENCE replay cache tests. connectathon tests.

NFSv4.0 mount: connectathon tests.

-->Andy

0001-nfsd41-expand-solo-sequence-check.patch 0002-nfsd41-bound-forechannel-drc-size-by-memory-usage.patch 0003-nfsd41-use-session-maxreqs-for-sequence-target-and.patch 0004-nfsd41-replace-nfserr_resource-in-pure-nfs41-respon.patch 0005-nfsd41-replace-page-based-DRC-with-buffer-based-DRC.patch

#20 lore raw
In reply to [email protected]
Quoted 23 lines · On Thu, Aug 27, 2009 at 12:07:39PM -0400, [email protected] wrote:

NFSv4.1 DRC Rewrite Version 6

These patches apply against git://linux-nfs.org/~bfields/linux: for-2.6.32 branch and continue to rewrite the NFSv4.1 Sessions DRC. Besides some bug fixes:

1) The bound on the fore channel per-session DRC size are rewritten. Instead of just using a maximum number of slots to bound the size, The server gives the client the number of ca_maxresponsesize_cached slots it requests bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.

This allows clients to tailor a session to usage.
For example, an I/O session (READ/WRITE/COMMIT only) can have a much smaller ca_maxresponsesize_cached (for only WRITE/COMMIt compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

2) the page-based DRC is replaced with a buffer based DRC with each slot table entry (struct nfsd4_slot + cache) allocated separately. This allocation prepares us for slot size re-negotiation via the SEQUENCE operation target and high slot id arguments.

Quoted 4 lines

Testing:

NFSv4.1 mount: pynfs tests - including the SEQUENCE replay cache tests. connectathon tests.

By the way, any hints on getting the 4.1 pynfs stuff running?

./nfs4.1/testserver.py pearlet1:/pynfs41test-root --rundeps --maketree Traceback (most recent call last):
File "./nfs4.1/testserver.py", line 38, in <module>
import server41tests.environment as environment
File "/root/pynfs41/nfs4.1/server41tests/environment.py", line 15, in <module>
import rpc
File "/root/pynfs41/rpc/__init__.py", line 1, in <module>
from rpc import *
File "/root/pynfs41/rpc/rpc.py", line 14, in <module>
import security
File "/root/pynfs41/rpc/security.py", line 11, in <module>
import gssapi
ImportError: No module named gssapi

With the 4.0 pynfs I've always run

	/setup.py build_ext --inplace

before. That gives me:

This is currently broken...everything should already be set up. Just go into nfs4 and play.

The setup.py in nfs4.1/ complains about not finding xdrgen.

--b.

#21 lore raw
Quoted 24 lines · J. Bruce Fields
Quoted 23 lines · On Thu, Aug 27, 2009 at 12:07:39PM -0400, [email protected] wrote:

NFSv4.1 DRC Rewrite Version 6

These patches apply against git://linux-nfs.org/~bfields/linux: for-2.6.32 branch and continue to rewrite the NFSv4.1 Sessions DRC. Besides some bug fixes:

1) The bound on the fore channel per-session DRC size are rewritten. Instead of just using a maximum number of slots to bound the size, The server gives the client the number of ca_maxresponsesize_cached slots it requests bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.

This allows clients to tailor a session to usage.
For example, an I/O session (READ/WRITE/COMMIT only) can have a much smaller ca_maxresponsesize_cached (for only WRITE/COMMIt compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

2) the page-based DRC is replaced with a buffer based DRC with each slot table entry (struct nfsd4_slot + cache) allocated separately. This allocation prepares us for slot size re-negotiation via the SEQUENCE operation target and high slot id arguments.

Quoted 6 lines
Quoted 4 lines

Testing:

NFSv4.1 mount: pynfs tests - including the SEQUENCE replay cache tests. connectathon tests.

By the way, any hints on getting the 4.1 pynfs stuff running?

OK, solution seemed to be

	cd gssapi
	python setup.py build_ext --inplace

But now almost everything fails.with SERVERFAULT. Bah.

--b.

#22 lore raw
Quoted 30 lines · bfields
Quoted 29 lines · J. Bruce Fields
Quoted 27 lines · On Thu, Aug 27, 2009 at 12:07:39PM -0400, [email protected] wrote:

NFSv4.1 DRC Rewrite Version 6

These patches apply against git://linux-nfs.org/~bfields/linux: for-2.6.32 branch and continue to rewrite the NFSv4.1 Sessions DRC. Besides some bug fixes:

1) The bound on the fore channel per-session DRC size are rewritten. Instead of just using a maximum number of slots to bound the size, The server gives the client the number of ca_maxresponsesize_cached slots it requests bounded by NFSD_SLOT_CACHE_SIZE, NFSD_MAX_MEM_PER_SESSION and by nfsd_drc_max_mem. Do not allow more than NFSD_MAX_SLOTS_PER_SESSION.

This allows clients to tailor a session to usage.
For example, an I/O session (READ/WRITE/COMMIT only) can have a much smaller ca_maxresponsesize_cached (for only WRITE/COMMIt compound responses) and a lot larger ca_maxresponses to service a large in-flight data window.

2) the page-based DRC is replaced with a buffer based DRC with each slot table entry (struct nfsd4_slot + cache) allocated separately. This allocation prepares us for slot size re-negotiation via the SEQUENCE operation target and high slot id arguments.

Testing:

NFSv4.1 mount: pynfs tests - including the SEQUENCE replay cache tests. connectathon tests.

Quoted 8 lines
Quoted 1 line

By the way, any hints on getting the 4.1 pynfs stuff running?

OK, solution seemed to be

	cd gssapi
	python setup.py build_ext --inplace

But now almost everything fails.with SERVERFAULT. Bah.

Does anyone have a list of which tests are expected to pass at this point? If I ask for "all" tests, I get "4 skipped, 103 failed, 0 warned, 26 passed--full list available if anyone wants. I seem to be passing connectathon tests OK over 4.1.

--b.

#23 lore raw
Quoted 9 lines · bfields
Quoted 8 lines · bfields
Quoted 1 line · J. Bruce Fields

By the way, any hints on getting the 4.1 pynfs stuff running?

OK, solution seemed to be

	cd gssapi
	python setup.py build_ext --inplace
Quoted 6 lines
Quoted 1 line

But now almost everything fails.with SERVERFAULT. Bah.

Does anyone have a list of which tests are expected to pass at this point? If I ask for "all" tests, I get "4 skipped, 103 failed, 0 warned, 26 passed--full list available if anyone wants. I seem to be passing connectathon tests OK over 4.1.

If someone could just send a copy of what they consider succesful (for now) pynfs output, that would be useful.

--b.