Skip to contentSkip to conversation

Please pull NFS client changes

PATCH In review Trond Myklebust 5 participants 15 messages Jul 31, 2011 – Aug 4, 2011

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Linus,

Please pull from the "nfs-for-3.1" branch of the repository at

git pull git://git.linux-nfs.org/projects/trondmy/linux-nfs.git nfs-for-3.1

This will update the following files through the appended changesets.

Cheers,
Trond

----

 fs/nfs/Kconfig                      |    8 +-
 fs/nfs/Makefile                     |    1 +
 fs/nfs/blocklayout/Makefile         |    5 +
 fs/nfs/blocklayout/blocklayout.c    | 1019 +++++++++++++++++++++++++++++++++++
 fs/nfs/blocklayout/blocklayout.h    |  207 +++++++
 fs/nfs/blocklayout/blocklayoutdev.c |  410 ++++++++++++++
 fs/nfs/blocklayout/blocklayoutdm.c  |  111 ++++
 fs/nfs/blocklayout/extents.c        |  935 ++++++++++++++++++++++++++++++++
 fs/nfs/client.c                     |   11 +-
 fs/nfs/dir.c                        |   57 ++-
 fs/nfs/nfs4_fs.h                    |    2 +-
 fs/nfs/nfs4filelayout.c             |    2 +-
 fs/nfs/nfs4proc.c                   |   62 ++-
 fs/nfs/nfs4xdr.c                    |  233 ++++++++-
 fs/nfs/pnfs.c                       |   86 ++--
 fs/nfs/pnfs.h                       |   28 +-
 include/linux/nfs.h                 |    2 +
 include/linux/nfs4.h                |    1 +
 include/linux/nfs_fs.h              |    3 +-
 include/linux/nfs_fs_sb.h           |    4 +-
 include/linux/nfs_xdr.h             |   17 +-
 21 files changed, 3113 insertions(+), 91 deletions(-)

commit 1c46c073ba7b1333d2d736056a081d4f17f5f1d3

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:56 2011 -0400

For invalid extents, find other pages in the same fsblock and write them out.

commit 579f69df9c99794f02878f7562a6906e8b2ac316

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:55 2011 -0400

commit 0593a6123b6736f06823624813a94c863af7be87

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:54 2011 -0400

Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.

When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.

[pnfsblock: bl_write_pagelist support functions]
[pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new write_pagelist api]
[SQUASHME: pnfsblock: mds_offset is set in the generic layer]
[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]
[pnfsblock: SQUASHME: adjust to API change]
[pnfsblock: fixup blksize alignment in bl_setup_layoutcommit]
[pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new write_pagelist api]

commit 1371c564cf844d3f479527b623565ac262e6ebaa

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:53 2011 -0400

Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.

When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.

[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]
[pnfsblock: read path error handling]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new read_pagelist api]

commit 5dd59231aa9930bc3b217539da6d5eff1120173e

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:52 2011 -0400

In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.

In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:

1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)

2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup

3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.

[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]
[pnfsblock: introduce bl_committing list]
[pnfsblock: SQUASHME: adjust to API change]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: fix bug setting up layoutcommit.]
[pnfsblock: cleanup_layoutcommit wants a status parameter]

commit 936a675fe578d40919046bf0a26f1776960a504f

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:51 2011 -0400

In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.

In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:

1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)

2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup

3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.

[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]
[pnfsblock: get rid of deprecated xdr macros]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: fix bug setting up layoutcommit.]
[pnfsblock: prevent commit list corruption]
[pnfsblock: fix layoutcommit with an empty opaque]

commit 84e5fe06f95c8ad330a525a5b11d21d8e371fbfb

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:50 2011 -0400

commit 426b7d89de75cef8b308969e809f6242c72f620e

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:49 2011 -0400

Adds working implementations of various support functions to handle INVAL extents, needed by writes, such as bl_mark_sectors_init and bl_is_sector_init.

[pnfsblock: fix 64-bit compiler warnings for extent manipulation]
[Implement release_inval_marks]

commit cef25cef97ff6799c82fa467f030146aea8876dc

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:48 2011 -0400

Implement bl_find_get_extent(), one of the core extent manipulation routines.

[pnfsblock: Lookup list entry of layouts and tags in reverse order]

pnfsblock: fix print format warnings for sector_t and size_t

gcc spews warnings about these on x86_64, e.g.:
fs/nfs/blocklayout/blocklayout.c:74: warning: format ‘%Lu’ expects type ‘long long unsigned int’, but argument 2 has type ‘sector_t’ fs/nfs/blocklayout/blocklayout.c:388: warning: format ‘%d’ expects type ‘int’, but argument 5 has type ‘size_t’

commit 6cb996c2b07c4e0c48d5e67a5cc24e8211d89add

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:47 2011 -0400

XDR decodes the block layout payload sent in LAYOUTGET result, storing the result in an extent list.

[pnfsblock: get rid of deprecated xdr macros]
[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]

commit 4d9a1fcdcbf13e6e65a4b0475af7d4fb139d1842

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:46 2011 -0400

Call GETDEVICELIST during mount, then call and parse GETDEVICEINFO for each device returned.

[pnfsblock: get rid of deprecated xdr macros]
[pnfsblock: fix pnfs_deviceid references]
[pnfsblock: fix print format warnings for sector_t and size_t]
[pnfs-block: #include <linux/vmalloc.h>]
[pnfsblock: no PNFS_NFS_SERVER]
[pnfsblock: fix bug determining size of striped volume]
[pnfsblock: fix oops when using multiple devices]
[pnfsblock: get rid of vmap and deviceid->area structure]

commit f0f45cd2693863faa19f0edfc3f16d77159f7838

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:45 2011 -0400

Replace a stub, so that extents underlying the layouts are properly added, merged, or ignored as necessary.

commit 03e023d1027458d13c0007f0e4456b65ee2977f1

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:44 2011 -0400
[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]

commit be4619cf2c94aad7a9853e4ab32ce31565b9335c

Author: Jim Rees <[email protected]>
Date:   Sat Jul 30 20:52:43 2011 -0400

commit db1905c6f31ac745573ffb3bf860e07088dd8713

Author: Jim Rees <[email protected]>
Date:   Sat Jul 30 20:52:42 2011 -0400

commit 7c8b2c6c3f8bd394401da844badd28990261dd2a

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:41 2011 -0400

Adds structures and basic create/delete code for extents.

commit be79608be504916f9534af194ca8d5b612606484

Author: Benny Halevy <[email protected]>
Date:   Sat Jul 30 20:52:40 2011 -0400
[pnfsblock: use pnfs_generic_pg_init_read/write]

commit 8f7e205d20c74fbaab3e4942e5daa09f0787fdd7

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:39 2011 -0400

Define a configuration variable to enable/disable compilation of the block driver code.

Add the minimal structure for a pnfs block layout driver, and empty list-heads that will hold the extent data

[pnfsblock: make NFS_V4_1 select PNFS_BLOCK]
[pnfs-block: fix CONFIG_PNFS_BLOCK dependencies]
[pnfsblock: SQUASHME: adjust to API change]
[pnfs: move pnfs_layout_type inline in nfs_inode]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: layout alloc and free]
[pnfs: move pnfs_layout_type inline in nfs_inode]
[pnfsblock: define module alias]
[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]

commit bec3f732e912295462c795d9947a7ed59f85caee

Author: Andy Adamson <[email protected]>
Date:   Sat Jul 30 20:52:38 2011 -0400

This gives layout driver a chance to cleanup structures they put in at encode_layoutcommit.

[fixup layout header pointer for layoutcommit]
[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]

commit 009354f2e7d197a29ca08d5f2c0bbe6e806855ec

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:37 2011 -0400

Block layout needs it to determine IO size.

commit 8d619fb13ecf6f56eabf88d7fa023cefb9832d2e

Author: Benny Halevy <[email protected]>
Date:   Sat Jul 30 20:52:36 2011 -0400

To allow layout driver to issue getdevicelist at mount time, and clean up at umount time.

[fixup non NFS_V4_1 set_pnfs_layoutdriver definition]
[pnfs: pass mntfh down the init_pnfs path]

commit 3c8507c7447b671ce14c7fc250c3cd7d9f338706

Author: Andy Adamson <[email protected]>
Date:   Sat Jul 30 20:52:35 2011 -0400

The block driver uses GETDEVICELIST

[pass struct nfs_server * to getdevicelist]
[get machince creds for getdevicelist]
[fix getdevicelist decode sizing]

commit 53e8c77ea157bb1888cf33a99de11c3d967a58df

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:34 2011 -0400

Using NFS4_MAX_UINT64 will break current protocol.

[Needed in v3.0]
CC Stable Tree <[email protected]>

commit eb425aadaed93b8499bebd8802c669b1b3d592f6

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:33 2011 -0400

There can be multiple lseg per file, so layoutcommit should be able to handle it.

commit d7060761432676d0249f680ddb4d130afb0bd5c9

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:32 2011 -0400

No need to save it for every lseg.
No need to save it at every pnfs_set_layoutcommit.

commit 8b7116b52046b95874f38c519b6e8923d9d1b113

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:31 2011 -0400

No need to save it for every lseg.

[Needed in v3.0]
CC Stable Tree <[email protected]>

commit 374e4e3ec347d4f70e1417ce3e1c454f133dab72

Author: Bryan Schumaker <[email protected]>
Date:   Fri Jul 29 11:49:06 2011 -0400

Additional readdir cookie loop information

Print out the name of the file that triggers the cookie loop message to make it slightly easier to track down the cause.

commit 0c0308066ca53fdf1423895f3a42838b67b3a5a8

Author: Trond Myklebust <[email protected]>
Date:   Sat Jul 30 12:45:35 2011 -0400

NFS: Fix spurious readdir cookie loop messages

If the directory contents change, then we have to accept that the file->f_pos value may shrink if we do a 'search-by-cookie'. In that case, we should turn off the loop detection and let the NFS client try to recover.

The patch also fixes a second loop detection bug by ensuring that after turning on the ctx->duped flag, we read at least one new cookie into ctx->dir_cookie before attempting to match with ctx->dup_cookie.

Signature · 6 lines
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#1 lore raw

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 8 lines

-----Original Message-----
From: Christoph Hellwig [mailto:[email protected]]
Sent: Sunday, July 31, 2011 2:24 PM
To: Myklebust, Trond
Cc: Linus Torvalds; [email protected]; linux- [email protected]; [email protected]
Subject: Re: Please pull NFS client changes

Quoted 5 lines

How does this manage to include the pnfs block code, which only was posted
on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

The majority of that code has been ready for several weeks and has been posted several times for review. The main stuff that was addressed in the last week were smaller issues such as the naming of functions that are globally visible (i.e. not polluting the kernel namespace), a few bisectability issues and (in the last iteraction) the lack of a
signed-off-by: from Jim.

I agree that we have to address the bug that Peng found, but that appears to be linked to the NFSv4.1 back channel, which these patches did not change.

The other point to note is that the patches do not touch the main read/write code and only lightly change the pNFS code paths. The impact of any residual bugs is therefore going to be limited.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
In reply to Christoph Hellwig
Quoted 4 lines · Christoph Hellwig

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

And it doesn't even compile:

In file included from fs/nfs/client.c:51:0:
fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’
fs/nfs/client.c: At top level:
fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1

Signature · 6 lines
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw
Quoted 8 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

And it doesn't even compile:

Quoted 9 lines

In file included from fs/nfs/client.c:51:0:
fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’
fs/nfs/client.c: At top level:
fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1

Sigh. This should fix it... 8<------------------------------------------------------------------------------------- From b74e1d98b2027aa3ec46f3612143d0d35db92cf5 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH] NFS: Fix a typo in fs/nfs/pnfs.h

---
 fs/nfs/pnfs.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
fs/nfs/pnfs.h +1 -1
index e0b5d80..01cbfd5 100644
@@ -380,7 +380,7 @@ pnfs_roc_drain(struct inode *ino, u32 *barrier)
 }
 
 static inline void set_pnfs_layoutdriver(struct nfs_server *s,
-					 const struct nfs_fh *mntfh, u32 id);
+					 const struct nfs_fh *mntfh, u32 id)
 {
 }
 
Signature · 15 lines
1.7.6


-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#5 lore raw
Quoted 19 lines · Trond Myklebust
Quoted 8 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

And it doesn't even compile:

Quoted 9 lines

In file included from fs/nfs/client.c:51:0:
fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’
fs/nfs/client.c: At top level:
fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1

Quoted 1 line

Sigh. This should fix it...

Only the first one, there are two independent places it broke. You missed this bit:

fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named

Signature · 6 lines
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#6 lore raw
Quoted 20 lines · Jens Axboe
Quoted 19 lines · Trond Myklebust
Quoted 17 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

And it doesn't even compile:

In file included from fs/nfs/client.c:51:0:
fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’
fs/nfs/client.c: At top level:
fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1

Quoted 6 lines
Quoted 1 line

Sigh. This should fix it...

Only the first one, there are two independent places it broke. You missed this bit:

fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named

How's this?

8<-------------------------------------------------------------------------------------- From e01e56eb7013a11f3e09bd8a8562332777f9e100 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH v2] NFS: Re-enable compilation of nfs with !CONFIG_NFS_V4 ||
!CONFIG_NFS_V4_1

Fix a typo in fs/nfs/pnfs.h
Move the pnfs_blksize declaration outside the CONFIG_NFS_V4 section in struct nfs_server.

---
 fs/nfs/pnfs.h             |    2 +-
 include/linux/nfs_fs_sb.h |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
fs/nfs/pnfs.h +1 -1
index e0b5d80..01cbfd5 100644
@@ -380,7 +380,7 @@ pnfs_roc_drain(struct inode *ino, u32 *barrier)
 }
 
 static inline void set_pnfs_layoutdriver(struct nfs_server *s,
-					 const struct nfs_fh *mntfh, u32 id);
+					 const struct nfs_fh *mntfh, u32 id)
 {
 }
 
include/linux/nfs_fs_sb.h +1 -1
index 82fdfc7..b5479df 100644
@@ -131,6 +131,7 @@ struct nfs_server {
 	struct fscache_cookie	*fscache;	/* superblock cookie */
 #endif
 
+	u32			pnfs_blksize;	/* layout_blksize attr */
 #ifdef CONFIG_NFS_V4
 	u32			attr_bitmask[3];/* V4 bitmask representing the set
 						   of attributes supported on this
@@ -145,7 +146,6 @@ struct nfs_server {
 						   filesystem */
 	struct pnfs_layoutdriver_type  *pnfs_curr_ld; /* Active layout driver */
 	struct rpc_wait_queue	roc_rpcwaitq;
-	u32			pnfs_blksize;	/* layout_blksize attr */
 	void			*pnfs_ld_data;	/* per mount point data */
 
 	/* the following fields are protected by nfs_client->cl_lock */
Signature · 10 lines
1.7.6


-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email protected]
www.netapp.com
#7 lore raw
Quoted 21 lines · Trond Myklebust
Quoted 20 lines · Jens Axboe
Quoted 19 lines · Trond Myklebust
Quoted 17 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig

How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?

And it doesn't even compile:

In file included from fs/nfs/client.c:51:0:
fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’
fs/nfs/client.c: At top level:
fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Sigh. This should fix it...

Only the first one, there are two independent places it broke. You missed this bit:

fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named

How's this?

Yup, that gets both.

Signature · 6 lines
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#8 lore raw
In reply to Trond Myklebust

Hi all,

Quoted 10 lines · On Sun, 31 Jul 2011 15:58:45 -0400 Trond Myklebust…

How's this?

8<-------------------------------------------------------------------------------------- From e01e56eb7013a11f3e09bd8a8562332777f9e100 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH v2] NFS: Re-enable compilation of nfs with !CONFIG_NFS_V4 ||
!CONFIG_NFS_V4_1

Quoted 6 lines

Fix a typo in fs/nfs/pnfs.h
Move the pnfs_blksize declaration outside the CONFIG_NFS_V4 section in struct nfs_server.

I have applied this the linux-next today ..

Signature · 3 lines
Cheers,
Stephen Rothwell                    [email protected]
http://www.canb.auug.org.au/~sfr/
#9 lore raw
Quoted 2 lines · On Mon, 1 Aug 2011 10:57:39 +1000 Stephen Rothwell…

I have applied this the linux-next today ..

And then I checked Linus' tree again to find it already there, so I have started again from the his newer tree.

Signature · 3 lines
Cheers,
Stephen Rothwell                    [email protected]
http://www.canb.auug.org.au/~sfr/
#10 lore raw
In reply to Trond Myklebust
Quoted 11 lines · Trond Myklebust

Hi Linus,

Please pull from the "nfs-for-3.1" branch of the repository at

git pull git://git.linux-nfs.org/projects/trondmy/linux-nfs.git nfs-for-3.1

This will update the following files through the appended changesets.

Cheers,
Trond

Trond hi.

You have not included the two patches for the objects-layout bug-fixes I sent you. (And I don't see them in your linux-next branch). Please include them in the next round of fixes. (They are also for stable)

Do you need that I resend them?

Thanks
Boaz

Quoted 555 lines

----

 fs/nfs/Kconfig                      |    8 +-
 fs/nfs/Makefile                     |    1 +
 fs/nfs/blocklayout/Makefile         |    5 +
 fs/nfs/blocklayout/blocklayout.c    | 1019 +++++++++++++++++++++++++++++++++++
 fs/nfs/blocklayout/blocklayout.h    |  207 +++++++
 fs/nfs/blocklayout/blocklayoutdev.c |  410 ++++++++++++++
 fs/nfs/blocklayout/blocklayoutdm.c  |  111 ++++
 fs/nfs/blocklayout/extents.c        |  935 ++++++++++++++++++++++++++++++++
 fs/nfs/client.c                     |   11 +-
 fs/nfs/dir.c                        |   57 ++-
 fs/nfs/nfs4_fs.h                    |    2 +-
 fs/nfs/nfs4filelayout.c             |    2 +-
 fs/nfs/nfs4proc.c                   |   62 ++-
 fs/nfs/nfs4xdr.c                    |  233 ++++++++-
 fs/nfs/pnfs.c                       |   86 ++--
 fs/nfs/pnfs.h                       |   28 +-
 include/linux/nfs.h                 |    2 +
 include/linux/nfs4.h                |    1 +
 include/linux/nfs_fs.h              |    3 +-
 include/linux/nfs_fs_sb.h           |    4 +-
 include/linux/nfs_xdr.h             |   17 +-
 21 files changed, 3113 insertions(+), 91 deletions(-)

commit 1c46c073ba7b1333d2d736056a081d4f17f5f1d3

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:56 2011 -0400

For invalid extents, find other pages in the same fsblock and write them out.

commit 579f69df9c99794f02878f7562a6906e8b2ac316

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:55 2011 -0400

commit 0593a6123b6736f06823624813a94c863af7be87

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:54 2011 -0400

Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.

When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.

[pnfsblock: bl_write_pagelist support functions]
[pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new write_pagelist api]
[SQUASHME: pnfsblock: mds_offset is set in the generic layer]
[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]
[pnfsblock: SQUASHME: adjust to API change]
[pnfsblock: fixup blksize alignment in bl_setup_layoutcommit]
[pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new write_pagelist api]

commit 1371c564cf844d3f479527b623565ac262e6ebaa

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:53 2011 -0400

Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.

When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.

[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]
[pnfsblock: read path error handling]
[pnfsblock: handle errors when read or write pagelist.]
[pnfs-block: use new read_pagelist api]

commit 5dd59231aa9930bc3b217539da6d5eff1120173e

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:52 2011 -0400

In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.

In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:

1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)

2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup

3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.

[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]
[pnfsblock: introduce bl_committing list]
[pnfsblock: SQUASHME: adjust to API change]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: fix bug setting up layoutcommit.]
[pnfsblock: cleanup_layoutcommit wants a status parameter]

commit 936a675fe578d40919046bf0a26f1776960a504f

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:51 2011 -0400

In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.

In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:

1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)

2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup

3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.

[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]
[pnfsblock: get rid of deprecated xdr macros]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: fix bug setting up layoutcommit.]
[pnfsblock: prevent commit list corruption]
[pnfsblock: fix layoutcommit with an empty opaque]

commit 84e5fe06f95c8ad330a525a5b11d21d8e371fbfb

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:50 2011 -0400

commit 426b7d89de75cef8b308969e809f6242c72f620e

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:49 2011 -0400

Adds working implementations of various support functions to handle INVAL extents, needed by writes, such as bl_mark_sectors_init and bl_is_sector_init.

[pnfsblock: fix 64-bit compiler warnings for extent manipulation]
[Implement release_inval_marks]

commit cef25cef97ff6799c82fa467f030146aea8876dc

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:48 2011 -0400

Implement bl_find_get_extent(), one of the core extent manipulation routines.

[pnfsblock: Lookup list entry of layouts and tags in reverse order]

pnfsblock: fix print format warnings for sector_t and size_t

gcc spews warnings about these on x86_64, e.g.:
fs/nfs/blocklayout/blocklayout.c:74: warning: format ‘%Lu’ expects type ‘long long unsigned int’, but argument 2 has type ‘sector_t’ fs/nfs/blocklayout/blocklayout.c:388: warning: format ‘%d’ expects type ‘int’, but argument 5 has type ‘size_t’

commit 6cb996c2b07c4e0c48d5e67a5cc24e8211d89add

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:47 2011 -0400

XDR decodes the block layout payload sent in LAYOUTGET result, storing the result in an extent list.

[pnfsblock: get rid of deprecated xdr macros]
[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]

commit 4d9a1fcdcbf13e6e65a4b0475af7d4fb139d1842

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:46 2011 -0400

Call GETDEVICELIST during mount, then call and parse GETDEVICEINFO for each device returned.

[pnfsblock: get rid of deprecated xdr macros]
[pnfsblock: fix pnfs_deviceid references]
[pnfsblock: fix print format warnings for sector_t and size_t]
[pnfs-block: #include <linux/vmalloc.h>]
[pnfsblock: no PNFS_NFS_SERVER]
[pnfsblock: fix bug determining size of striped volume]
[pnfsblock: fix oops when using multiple devices]
[pnfsblock: get rid of vmap and deviceid->area structure]

commit f0f45cd2693863faa19f0edfc3f16d77159f7838

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:45 2011 -0400

Replace a stub, so that extents underlying the layouts are properly added, merged, or ignored as necessary.

commit 03e023d1027458d13c0007f0e4456b65ee2977f1

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:44 2011 -0400
[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]

commit be4619cf2c94aad7a9853e4ab32ce31565b9335c

Author: Jim Rees <[email protected]>
Date:   Sat Jul 30 20:52:43 2011 -0400

commit db1905c6f31ac745573ffb3bf860e07088dd8713

Author: Jim Rees <[email protected]>
Date:   Sat Jul 30 20:52:42 2011 -0400

commit 7c8b2c6c3f8bd394401da844badd28990261dd2a

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:41 2011 -0400

Adds structures and basic create/delete code for extents.

commit be79608be504916f9534af194ca8d5b612606484

Author: Benny Halevy <[email protected]>
Date:   Sat Jul 30 20:52:40 2011 -0400
[pnfsblock: use pnfs_generic_pg_init_read/write]

commit 8f7e205d20c74fbaab3e4942e5daa09f0787fdd7

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:39 2011 -0400

Define a configuration variable to enable/disable compilation of the block driver code.

Add the minimal structure for a pnfs block layout driver, and empty list-heads that will hold the extent data

[pnfsblock: make NFS_V4_1 select PNFS_BLOCK]
[pnfs-block: fix CONFIG_PNFS_BLOCK dependencies]
[pnfsblock: SQUASHME: adjust to API change]
[pnfs: move pnfs_layout_type inline in nfs_inode]
[blocklayout: encode_layoutcommit implementation]
[pnfsblock: layout alloc and free]
[pnfs: move pnfs_layout_type inline in nfs_inode]
[pnfsblock: define module alias]
[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]

commit bec3f732e912295462c795d9947a7ed59f85caee

Author: Andy Adamson <[email protected]>
Date:   Sat Jul 30 20:52:38 2011 -0400

This gives layout driver a chance to cleanup structures they put in at encode_layoutcommit.

[fixup layout header pointer for layoutcommit]
[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]

commit 009354f2e7d197a29ca08d5f2c0bbe6e806855ec

Author: Fred Isaman <[email protected]>
Date:   Sat Jul 30 20:52:37 2011 -0400

Block layout needs it to determine IO size.

commit 8d619fb13ecf6f56eabf88d7fa023cefb9832d2e

Author: Benny Halevy <[email protected]>
Date:   Sat Jul 30 20:52:36 2011 -0400

To allow layout driver to issue getdevicelist at mount time, and clean up at umount time.

[fixup non NFS_V4_1 set_pnfs_layoutdriver definition]
[pnfs: pass mntfh down the init_pnfs path]

commit 3c8507c7447b671ce14c7fc250c3cd7d9f338706

Author: Andy Adamson <[email protected]>
Date:   Sat Jul 30 20:52:35 2011 -0400

The block driver uses GETDEVICELIST

[pass struct nfs_server * to getdevicelist]
[get machince creds for getdevicelist]
[fix getdevicelist decode sizing]

commit 53e8c77ea157bb1888cf33a99de11c3d967a58df

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:34 2011 -0400

Using NFS4_MAX_UINT64 will break current protocol.

[Needed in v3.0]
CC Stable Tree <[email protected]>

commit eb425aadaed93b8499bebd8802c669b1b3d592f6

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:33 2011 -0400

There can be multiple lseg per file, so layoutcommit should be able to handle it.

commit d7060761432676d0249f680ddb4d130afb0bd5c9

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:32 2011 -0400

No need to save it for every lseg.
No need to save it at every pnfs_set_layoutcommit.

commit 8b7116b52046b95874f38c519b6e8923d9d1b113

Author: Peng Tao <[email protected]>
Date:   Sat Jul 30 20:52:31 2011 -0400

No need to save it for every lseg.

[Needed in v3.0]
CC Stable Tree <[email protected]>

commit 374e4e3ec347d4f70e1417ce3e1c454f133dab72

Author: Bryan Schumaker <[email protected]>
Date:   Fri Jul 29 11:49:06 2011 -0400

Additional readdir cookie loop information

Print out the name of the file that triggers the cookie loop message to make it slightly easier to track down the cause.

commit 0c0308066ca53fdf1423895f3a42838b67b3a5a8

Author: Trond Myklebust <[email protected]>
Date:   Sat Jul 30 12:45:35 2011 -0400

NFS: Fix spurious readdir cookie loop messages

If the directory contents change, then we have to accept that the file->f_pos value may shrink if we do a 'search-by-cookie'. In that case, we should turn off the loop detection and let the NFS client try to recover.

The patch also fixes a second loop detection bug by ensuring that after turning on the ctx->duped flag, we read at least one new cookie into ctx->dir_cookie before attempting to match with ctx->dup_cookie.

#11 lore raw

PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBCb2F6IEhhcnJvc2ggW21haWx0 bzpiaGFycm9zaEBwYW5hc2FzLmNvbV0NCj4gU2VudDogV2VkbmVzZGF5LCBBdWd1c3QgMDMsIDIw MTEgNzo1MiBQTQ0KPiBUbzogTXlrbGVidXN0LCBUcm9uZA0KPiBDYzogTGludXMgVG9ydmFsZHM7 IE5GUyBsaXN0DQo+IFN1YmplY3Q6IFJlOiBQbGVhc2UgcHVsbCBORlMgY2xpZW50IGNoYW5nZXMN Cj4gDQo+IE9uIDA3LzMwLzIwMTEgMDg6MTkgUE0sIFRyb25kIE15a2xlYnVzdCB3cm90ZToNCj4g PiBIaSBMaW51cywNCj4gPg0KPiA+IFBsZWFzZSBwdWxsIGZyb20gdGhlICJuZnMtZm9yLTMuMSIg YnJhbmNoIG9mIHRoZSByZXBvc2l0b3J5IGF0DQo+ID4NCj4gPiAgICBnaXQgcHVsbCBnaXQ6Ly9n aXQubGludXgtbmZzLm9yZy9wcm9qZWN0cy90cm9uZG15L2xpbnV4LW5mcy5naXQNCj4gbmZzLWZv ci0zLjENCj4gPg0KPiA+IFRoaXMgd2lsbCB1cGRhdGUgdGhlIGZvbGxvd2luZyBmaWxlcyB0aHJv dWdoIHRoZSBhcHBlbmRlZCBjaGFuZ2VzZXRzLg0KPiA+DQo+ID4gICBDaGVlcnMsDQo+ID4gICAg IFRyb25kDQo+ID4NCj4gDQo+IFRyb25kIGhpLg0KPiANCj4gWW91IGhhdmUgbm90IGluY2x1ZGVk IHRoZSB0d28gcGF0Y2hlcyBmb3IgdGhlIG9iamVjdHMtbGF5b3V0IGJ1Zy1maXhlcw0KPiBJIHNl bnQgeW91LiAoQW5kIEkgZG9uJ3Qgc2VlIHRoZW0gaW4geW91ciBsaW51eC1uZXh0IGJyYW5jaCku IFBsZWFzZQ0KPiBpbmNsdWRlIHRoZW0gaW4gdGhlIG5leHQgcm91bmQgb2YgZml4ZXMuIChUaGV5 IGFyZSBhbHNvIGZvciBzdGFibGUpDQo+IA0KPiBEbyB5b3UgbmVlZCB0aGF0IEkgcmVzZW5kIHRo ZW0/DQoNClllcywgcGxlYXNlIHJlc2VuZCB0aGVtLCBzbyB0aGF0IEkgY2FuIHF1ZXVlIHRoZW0g dXAgZm9yIGEgcG9zdC1yYzEgYnVnZml4ZXMgcHVsbC4NCg0KQ2hlZXJzDQogIFRyb25kDQoT77+9 77+97Lm7HO+/vSbvv71+77+9Ju+/vRjvv73vv70rLe+/ve+/vd22F++/ve+/vXfvv73vv73Lm++/ ve+/ve+/vW3vv71i77+977+9Z37Ip++/vRfvv73vv73cqH3vv73vv73vv73GoHrvv70majordu+/ ve+/ve+/vQfvv73vv73vv73vv716Wivvv73vv70rembvv73vv73vv71o77+977+977+9fu+/ve+/ ve+/ve+/vWnvv73vv73vv71677+9Hu+/vXfvv73vv73vv70/77+977+977+977+9Ju+/vSnfohtm

#12 lore raw

PATCH 1/1 pnfs-obj: Fix the comp_index != 0 case

There were bugs in the case of partial layout where olo_comp_index is not zero. This used to work and was tested but one of the later cleanup SQUASHMEs broke it and was not tested since.

Also add a dprint that specify those received layout parameters. Everything else was already printed.

[Needed in v3.0]
CC Stable Tree <[email protected]>
---
 fs/nfs/objlayout/objio_osd.c        |   16 +++++++---------
 fs/nfs/objlayout/pnfs_osd_xdr_cli.c |    3 +++
 2 files changed, 10 insertions(+), 9 deletions(-)
fs/nfs/objlayout/objio_osd.c +7 -9
index aa8663a..d0cda12 100644
@@ -479,7 +479,6 @@ static int _io_check(struct objio_state *ios, bool is_write)
 	for (i = 0; i <  ios->numdevs; i++) {
 		struct osd_sense_info osi;
 		struct osd_request *or = ios->per_dev[i].or;
-		unsigned dev;
 		int ret;
 
 		if (!or)
@@ -500,9 +499,8 @@ static int _io_check(struct objio_state *ios, bool is_write)
 
 			continue; /* we recovered */
 		}
-		dev = ios->per_dev[i].dev;
-		objlayout_io_set_result(&ios->ol_state, dev,
-					&ios->layout->comps[dev].oc_object_id,
+		objlayout_io_set_result(&ios->ol_state, i,
+					&ios->layout->comps[i].oc_object_id,
 					osd_pri_2_pnfs_err(osi.osd_err_pri),
 					ios->per_dev[i].offset,
 					ios->per_dev[i].length,
@@ -648,7 +646,7 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,
 	int ret = 0;
 
 	while (length) {
-		struct _objio_per_comp *per_dev = &ios->per_dev[dev];
+		struct _objio_per_comp *per_dev = &ios->per_dev[dev - first_dev];
 		unsigned cur_len, page_off = 0;
 
 		if (!per_dev->length) {
@@ -668,8 +666,8 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,
 				cur_len = stripe_unit;
 			}
 
-			if (max_comp < dev)
-				max_comp = dev;
+			if (max_comp < dev - first_dev)
+				max_comp = dev - first_dev;
 		} else {
 			cur_len = stripe_unit;
 		}
@@ -804,7 +802,7 @@ static int _read_mirrors(struct objio_state *ios, unsigned cur_comp)
 	struct _objio_per_comp *per_dev = &ios->per_dev[cur_comp];
 	unsigned dev = per_dev->dev;
 	struct pnfs_osd_object_cred *cred =
-			&ios->layout->comps[dev];
+			&ios->layout->comps[cur_comp];
 	struct osd_obj_id obj = {
 		.partition = cred->oc_object_id.oid_partition_id,
 		.id = cred->oc_object_id.oid_object_id,
@@ -902,7 +900,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)
 	for (; cur_comp < last_comp; ++cur_comp, ++dev) {
 		struct osd_request *or = NULL;
 		struct pnfs_osd_object_cred *cred =
-					&ios->layout->comps[dev];
+					&ios->layout->comps[cur_comp];
 		struct osd_obj_id obj = {
 			.partition = cred->oc_object_id.oid_partition_id,
 			.id = cred->oc_object_id.oid_object_id,
fs/nfs/objlayout/pnfs_osd_xdr_cli.c +3 -0
index 16fc758..b3918f7 100644
@@ -170,6 +170,9 @@ int pnfs_osd_xdr_decode_layout_map(struct pnfs_osd_layout *layout,
 	p = _osd_xdr_decode_data_map(p, &layout->olo_map);
 	layout->olo_comps_index = be32_to_cpup(p++);
 	layout->olo_num_comps = be32_to_cpup(p++);
+	dprintk("%s: olo_comps_index=%d olo_num_comps=%d\n", __func__,
+		layout->olo_comps_index, layout->olo_num_comps);
+
 	iter->total_comps = layout->olo_num_comps;
 	return 0;
 }
Signature · 1 line
1.7.6
#13 lore raw

Re: [PATCH] pnfs-obj: Fix the comp_index != 0 case

Opps: [PATCH 2/2] pnfs-obj: Fix the comp_index != 0 case

Thanks

Quoted 92 lines · Boaz Harrosh

There were bugs in the case of partial layout where olo_comp_index is not zero. This used to work and was tested but one of the later cleanup SQUASHMEs broke it and was not tested since.

Also add a dprint that specify those received layout parameters. Everything else was already printed.

[Needed in v3.0]
CC Stable Tree <[email protected]>
---
 fs/nfs/objlayout/objio_osd.c        |   16 +++++++---------
 fs/nfs/objlayout/pnfs_osd_xdr_cli.c |    3 +++
 2 files changed, 10 insertions(+), 9 deletions(-)
fs/nfs/objlayout/objio_osd.c +7 -9
index aa8663a..d0cda12 100644
@@ -479,7 +479,6 @@ static int _io_check(struct objio_state *ios, bool is_write)
 	for (i = 0; i <  ios->numdevs; i++) {
 		struct osd_sense_info osi;
 		struct osd_request *or = ios->per_dev[i].or;
-		unsigned dev;
 		int ret;
 
 		if (!or)
@@ -500,9 +499,8 @@ static int _io_check(struct objio_state *ios, bool is_write)
 
 			continue; /* we recovered */
 		}
-		dev = ios->per_dev[i].dev;
-		objlayout_io_set_result(&ios->ol_state, dev,
-					&ios->layout->comps[dev].oc_object_id,
+		objlayout_io_set_result(&ios->ol_state, i,
+					&ios->layout->comps[i].oc_object_id,
 					osd_pri_2_pnfs_err(osi.osd_err_pri),
 					ios->per_dev[i].offset,
 					ios->per_dev[i].length,
@@ -648,7 +646,7 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,
 	int ret = 0;
 
 	while (length) {
-		struct _objio_per_comp *per_dev = &ios->per_dev[dev];
+		struct _objio_per_comp *per_dev = &ios->per_dev[dev - first_dev];
 		unsigned cur_len, page_off = 0;
 
 		if (!per_dev->length) {
@@ -668,8 +666,8 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,
 				cur_len = stripe_unit;
 			}
 
-			if (max_comp < dev)
-				max_comp = dev;
+			if (max_comp < dev - first_dev)
+				max_comp = dev - first_dev;
 		} else {
 			cur_len = stripe_unit;
 		}
@@ -804,7 +802,7 @@ static int _read_mirrors(struct objio_state *ios, unsigned cur_comp)
 	struct _objio_per_comp *per_dev = &ios->per_dev[cur_comp];
 	unsigned dev = per_dev->dev;
 	struct pnfs_osd_object_cred *cred =
-			&ios->layout->comps[dev];
+			&ios->layout->comps[cur_comp];
 	struct osd_obj_id obj = {
 		.partition = cred->oc_object_id.oid_partition_id,
 		.id = cred->oc_object_id.oid_object_id,
@@ -902,7 +900,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)
 	for (; cur_comp < last_comp; ++cur_comp, ++dev) {
 		struct osd_request *or = NULL;
 		struct pnfs_osd_object_cred *cred =
-					&ios->layout->comps[dev];
+					&ios->layout->comps[cur_comp];
 		struct osd_obj_id obj = {
 			.partition = cred->oc_object_id.oid_partition_id,
 			.id = cred->oc_object_id.oid_object_id,
fs/nfs/objlayout/pnfs_osd_xdr_cli.c +3 -0
index 16fc758..b3918f7 100644
@@ -170,6 +170,9 @@ int pnfs_osd_xdr_decode_layout_map(struct pnfs_osd_layout *layout,
 	p = _osd_xdr_decode_data_map(p, &layout->olo_map);
 	layout->olo_comps_index = be32_to_cpup(p++);
 	layout->olo_num_comps = be32_to_cpup(p++);
+	dprintk("%s: olo_comps_index=%d olo_num_comps=%d\n", __func__,
+		layout->olo_comps_index, layout->olo_num_comps);
+
 	iter->total_comps = layout->olo_num_comps;
 	return 0;
 }
#14 lore raw

PATCH 1/2 pnfs-obj: Bug when we are running out of bio

In reply to Myklebust, Trond

When we have a situation that the number of pages we want to encode is bigger then the size of the bio. (Which can currently happen only when all IO is going to a single device .e.g group_width==1) then the IO is submitted short and we report back only the amount of bytes we actually wrote/read and all is fine. BUT ...

There was a bug that the current length counter was advanced before the fail to add the extra page, and we come to a situation that the CDB length was one-page longer then the actual bio size, which is of course rejected by the osd-target.

While here also fix the bio size calculation, in the case that we received more then one group of devices.

---
 fs/nfs/objlayout/objio_osd.c |   12 +++++-------
 1 files changed, 5 insertions(+), 7 deletions(-)
fs/nfs/objlayout/objio_osd.c +5 -7
index 9383ca7..aa8663a 100644
@@ -589,22 +589,19 @@ static void _calc_stripe_info(struct objio_state *ios, u64 file_offset,
 }
 
 static int _add_stripe_unit(struct objio_state *ios,  unsigned *cur_pg,
-		unsigned pgbase, struct _objio_per_comp *per_dev, int cur_len,
+		unsigned pgbase, struct _objio_per_comp *per_dev, int len,
 		gfp_t gfp_flags)
 {
 	unsigned pg = *cur_pg;
+	int cur_len = len;
 	struct request_queue *q =
 			osd_request_queue(_io_od(ios, per_dev->dev));
 
-	per_dev->length += cur_len;
-
 	if (per_dev->bio == NULL) {
-		unsigned stripes = ios->layout->num_comps /
-						     ios->layout->mirrors_p1;
-		unsigned pages_in_stripe = stripes *
+		unsigned pages_in_stripe = ios->layout->group_width *
 				      (ios->layout->stripe_unit / PAGE_SIZE);
 		unsigned bio_size = (ios->ol_state.nr_pages + pages_in_stripe) /
-				    stripes;
+				    ios->layout->group_width;
 
 		if (BIO_MAX_PAGES_KMALLOC < bio_size)
 			bio_size = BIO_MAX_PAGES_KMALLOC;
@@ -632,6 +629,7 @@ static int _add_stripe_unit(struct objio_state *ios,  unsigned *cur_pg,
 	}
 	BUG_ON(cur_len);
 
+	per_dev->length += len;
 	*cur_pg = pg;
 	return 0;
 }
Signature · 1 line
1.7.6