nfsd: supports read buffer from multiples pages

ltp fsync02 will cause nfs sending LAYOUTCOMMIT with length larger than two pages. nfsd returns NFSERR_BAD_XDR right now.

This patch lets nfsd supports read buffer from multiples pages.

Signed-off-by Kinglong Mee <[email protected]>

---
 fs/nfsd/nfs4xdr.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

fs/nfsd/nfs4xdr.c +14 -4

index d6ef095..fcf399f 100644
@@ -143,11 +143,11 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	 * Maybe we need a new page, maybe we have just run out
 	 */
 	unsigned int avail = (char *)argp->end - (char *)argp->p;
+	unsigned int copied = 0;
 	__be32 *p;
+
 	if (avail + argp->pagelen < nbytes)
 		return NULL;
-	if (avail + PAGE_SIZE < nbytes) /* need more than a page !! */
-		return NULL;
 	/* ok, we can do it with the current plus the next page */
 	if (nbytes <= sizeof(argp->tmp))
 		p = argp->tmp;

@@ -164,9 +164,19 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	 * guarantee p points to at least nbytes bytes.
 	 */
 	memcpy(p, argp->p, avail);
+	copied += avail;
+	nbytes -= avail;
+
+	while (nbytes > PAGE_SIZE) {
+		next_decode_page(argp);
+		memcpy(((char*)p) + copied, argp->p, PAGE_SIZE);
+		copied += PAGE_SIZE;
+		nbytes -= PAGE_SIZE;
+	}
+
 	next_decode_page(argp);
-	memcpy(((char*)p)+avail, argp->p, (nbytes - avail));
-	argp->p += XDR_QUADLEN(nbytes - avail);
+	memcpy(((char*)p) + copied, argp->p, nbytes);
+	argp->p += XDR_QUADLEN(nbytes);
 	return p;
 }
 

Signature · 1 line

2.5.0

Quoted 2 lines · Kinglong Mee

ltp fsync02 will cause nfs sending LAYOUTCOMMIT with length larger than two pages. nfsd returns NFSERR_BAD_XDR right now.

This is with the xfs block layout?

Christoph, do we know anything about average or worst-case sizes for that layout update field?

Quoted 1 line

This patch lets nfsd supports read buffer from multiples pages.

Hm. We'll end up kmalloc()ing the passed-in field length:

		p = argp->tmpp = kmalloc(nbytes, GFP_KERNEL);

We still do still have that (avail + argp->pagelen) limit, so we're not going to pass arbitrarily large nbytes straight from the network to kmalloc. But we do try to avoid depending on higher-order allocations.

Quoted 2 lines · Kinglong Mee

ltp fsync02 will cause nfs sending LAYOUTCOMMIT with length larger than two pages. nfsd returns NFSERR_BAD_XDR right now.

This is with the xfs block layout?

Christoph, do we know anything about average or worst-case sizes for that layout update field?

Quoted 1 line

This patch lets nfsd supports read buffer from multiples pages.

Hm. We'll end up kmalloc()ing the passed-in field length:

		p = argp->tmpp = kmalloc(nbytes, GFP_KERNEL);

We still do still have that (avail + argp->pagelen) limit, so we're not going to pass arbitrarily large nbytes straight from the network to kmalloc. But we do try to avoid depending on higher-order allocations.

--b.

Quoted 48 lines

Signed-off-by Kinglong Mee <[email protected]>

---
 fs/nfsd/nfs4xdr.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

fs/nfsd/nfs4xdr.c +14 -4

index d6ef095..fcf399f 100644
@@ -143,11 +143,11 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	 * Maybe we need a new page, maybe we have just run out
 	 */
 	unsigned int avail = (char *)argp->end - (char *)argp->p;
+	unsigned int copied = 0;
 	__be32 *p;
+
 	if (avail + argp->pagelen < nbytes)
 		return NULL;
-	if (avail + PAGE_SIZE < nbytes) /* need more than a page !! */
-		return NULL;
 	/* ok, we can do it with the current plus the next page */
 	if (nbytes <= sizeof(argp->tmp))
 		p = argp->tmp;

@@ -164,9 +164,19 @@ static __be32 *read_buf(struct nfsd4_compoundargs *argp, u32 nbytes)
 	 * guarantee p points to at least nbytes bytes.
 	 */
 	memcpy(p, argp->p, avail);
+	copied += avail;
+	nbytes -= avail;
+
+	while (nbytes > PAGE_SIZE) {
+		next_decode_page(argp);
+		memcpy(((char*)p) + copied, argp->p, PAGE_SIZE);
+		copied += PAGE_SIZE;
+		nbytes -= PAGE_SIZE;
+	}
+
 	next_decode_page(argp);
-	memcpy(((char*)p)+avail, argp->p, (nbytes - avail));
-	argp->p += XDR_QUADLEN(nbytes - avail);
+	memcpy(((char*)p) + copied, argp->p, nbytes);
+	argp->p += XDR_QUADLEN(nbytes);
 	return p;
 }
 

Signature · 1 line

2.5.0

This is with the xfs block layout?

Christoph, do we know anything about average or worst-case sizes for that layout update field?

Quoted 4 lines · J. Bruce Fields

This is with the xfs block layout?

Christoph, do we know anything about average or worst-case sizes for that layout update field?

The average is rather small and fits into a single page, the worst case is basically unlimited:

(file size / block size) * sizeof(pnfs_block_extent)

by the protocol, and about half that for a non-stupid client as it would merge consecutive blocks and only trigger something close to the worst case for a "block allocated, block hole, block allocated, ..." pattern.

OK. And what's the failure mode if the layoutcommit fails? I guess the client returns the layout and redoes IO through the MDS. For a rare failure maybe that's not so terrible.

So I guess the right thing to do is take Kinglong's patch for now.

After that, it wouldn't be that hard for nfsd4_block_proc_layoutcommit to decode from an array of pages. But does the iomaps array end up being just as big?