Please pull NFS client changes
Subsystems (3)
Hi Linus,
Please pull from the "nfs-for-3.1" branch of the repository at
git pull git://git.linux-nfs.org/projects/trondmy/linux-nfs.git nfs-for-3.1
This will update the following files through the appended changesets.
Cheers,
Trond
----
fs/nfs/Kconfig | 8 +- fs/nfs/Makefile | 1 + fs/nfs/blocklayout/Makefile | 5 + fs/nfs/blocklayout/blocklayout.c | 1019 +++++++++++++++++++++++++++++++++++ fs/nfs/blocklayout/blocklayout.h | 207 +++++++ fs/nfs/blocklayout/blocklayoutdev.c | 410 ++++++++++++++ fs/nfs/blocklayout/blocklayoutdm.c | 111 ++++ fs/nfs/blocklayout/extents.c | 935 ++++++++++++++++++++++++++++++++ fs/nfs/client.c | 11 +- fs/nfs/dir.c | 57 ++- fs/nfs/nfs4_fs.h | 2 +- fs/nfs/nfs4filelayout.c | 2 +- fs/nfs/nfs4proc.c | 62 ++- fs/nfs/nfs4xdr.c | 233 ++++++++- fs/nfs/pnfs.c | 86 ++-- fs/nfs/pnfs.h | 28 +- include/linux/nfs.h | 2 + include/linux/nfs4.h | 1 + include/linux/nfs_fs.h | 3 +- include/linux/nfs_fs_sb.h | 4 +- include/linux/nfs_xdr.h | 17 +- 21 files changed, 3113 insertions(+), 91 deletions(-)
commit 1c46c073ba7b1333d2d736056a081d4f17f5f1d3
Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:56 2011 -0400
For invalid extents, find other pages in the same fsblock and write them out.
commit 579f69df9c99794f02878f7562a6906e8b2ac316
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:55 2011 -0400
commit 0593a6123b6736f06823624813a94c863af7be87
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:54 2011 -0400
Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.
When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.
commit 1371c564cf844d3f479527b623565ac262e6ebaa
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:53 2011 -0400
Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.
When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.
commit 5dd59231aa9930bc3b217539da6d5eff1120173e
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:52 2011 -0400
In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.
In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:
1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)
2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup
3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.
commit 936a675fe578d40919046bf0a26f1776960a504f
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:51 2011 -0400
In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.
In the new system where actual xdr encoding is done in encode_layoutcommit() directly into xdr buffer, these are the new commit stages:
1. On setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication with bl_encode_layoutcommit && bl_cleanup_layoutcommit (Generic layer provides a void-star to hang it on)
2. bl_encode_layoutcommit is called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup
3. On cleanup the commit-extent-list is put back by a call to set_to_rw() as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.
commit 84e5fe06f95c8ad330a525a5b11d21d8e371fbfb
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:50 2011 -0400
commit 426b7d89de75cef8b308969e809f6242c72f620e
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:49 2011 -0400
Adds working implementations of various support functions to handle INVAL extents, needed by writes, such as bl_mark_sectors_init and bl_is_sector_init.
commit cef25cef97ff6799c82fa467f030146aea8876dc
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:48 2011 -0400
Implement bl_find_get_extent(), one of the core extent manipulation routines.
pnfsblock: fix print format warnings for sector_t and size_t
gcc spews warnings about these on x86_64, e.g.:
fs/nfs/blocklayout/blocklayout.c:74: warning: format ‘%Lu’ expects type ‘long long unsigned int’, but argument 2 has type ‘sector_t’ fs/nfs/blocklayout/blocklayout.c:388: warning: format ‘%d’ expects type ‘int’, but argument 5 has type ‘size_t’
commit 6cb996c2b07c4e0c48d5e67a5cc24e8211d89add
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:47 2011 -0400
XDR decodes the block layout payload sent in LAYOUTGET result, storing the result in an extent list.
commit 4d9a1fcdcbf13e6e65a4b0475af7d4fb139d1842
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:46 2011 -0400
Call GETDEVICELIST during mount, then call and parse GETDEVICEINFO for each device returned.
commit f0f45cd2693863faa19f0edfc3f16d77159f7838
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:45 2011 -0400
Replace a stub, so that extents underlying the layouts are properly added, merged, or ignored as necessary.
commit 03e023d1027458d13c0007f0e4456b65ee2977f1
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:44 2011 -0400
commit be4619cf2c94aad7a9853e4ab32ce31565b9335c
Author: Jim Rees <[email protected]> Date: Sat Jul 30 20:52:43 2011 -0400
commit db1905c6f31ac745573ffb3bf860e07088dd8713
Author: Jim Rees <[email protected]> Date: Sat Jul 30 20:52:42 2011 -0400
commit 7c8b2c6c3f8bd394401da844badd28990261dd2a
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:41 2011 -0400
Adds structures and basic create/delete code for extents.
commit be79608be504916f9534af194ca8d5b612606484
Author: Benny Halevy <[email protected]> Date: Sat Jul 30 20:52:40 2011 -0400
commit 8f7e205d20c74fbaab3e4942e5daa09f0787fdd7
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:39 2011 -0400
Define a configuration variable to enable/disable compilation of the block driver code.
Add the minimal structure for a pnfs block layout driver, and empty list-heads that will hold the extent data
commit bec3f732e912295462c795d9947a7ed59f85caee
Author: Andy Adamson <[email protected]> Date: Sat Jul 30 20:52:38 2011 -0400
This gives layout driver a chance to cleanup structures they put in at encode_layoutcommit.
commit 009354f2e7d197a29ca08d5f2c0bbe6e806855ec
Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:37 2011 -0400
Block layout needs it to determine IO size.
commit 8d619fb13ecf6f56eabf88d7fa023cefb9832d2e
Author: Benny Halevy <[email protected]> Date: Sat Jul 30 20:52:36 2011 -0400
To allow layout driver to issue getdevicelist at mount time, and clean up at umount time.
commit 3c8507c7447b671ce14c7fc250c3cd7d9f338706
Author: Andy Adamson <[email protected]> Date: Sat Jul 30 20:52:35 2011 -0400
The block driver uses GETDEVICELIST
commit 53e8c77ea157bb1888cf33a99de11c3d967a58df
Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:34 2011 -0400
Using NFS4_MAX_UINT64 will break current protocol.
commit eb425aadaed93b8499bebd8802c669b1b3d592f6
Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:33 2011 -0400
There can be multiple lseg per file, so layoutcommit should be able to handle it.
commit d7060761432676d0249f680ddb4d130afb0bd5c9
Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:32 2011 -0400
No need to save it for every lseg.
No need to save it at every pnfs_set_layoutcommit.
commit 8b7116b52046b95874f38c519b6e8923d9d1b113
Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:31 2011 -0400
No need to save it for every lseg.
commit 374e4e3ec347d4f70e1417ce3e1c454f133dab72
Author: Bryan Schumaker <[email protected]> Date: Fri Jul 29 11:49:06 2011 -0400
Additional readdir cookie loop information
Print out the name of the file that triggers the cookie loop message to make it slightly easier to track down the cause.
commit 0c0308066ca53fdf1423895f3a42838b67b3a5a8
Author: Trond Myklebust <[email protected]> Date: Sat Jul 30 12:45:35 2011 -0400
NFS: Fix spurious readdir cookie loop messages
If the directory contents change, then we have to accept that the file->f_pos value may shrink if we do a 'search-by-cookie'. In that case, we should turn off the loop detection and let the NFS client try to recover.
The patch also fixes a second loop detection bug by ensuring that after turning on the ctx->duped flag, we read at least one new cookie into ctx->dir_cookie before attempting to match with ctx->dup_cookie.
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines
-----Original Message-----
From: Christoph Hellwig [mailto:[email protected]]
Sent: Sunday, July 31, 2011 2:24 PM
To: Myklebust, Trond
Cc: Linus Torvalds; [email protected]; linux- [email protected]; [email protected]
Subject: Re: Please pull NFS client changes
Quoted 5 lines
How does this manage to include the pnfs block code, which only was posted
on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
The majority of that code has been ready for several weeks and has been posted several times for review. The main stuff that was addressed in the last week were smaller issues such as the naming of functions that are globally visible (i.e. not polluting the kernel namespace), a few bisectability issues and (in the last iteraction) the lack of a
signed-off-by: from Jim.
I agree that we have to address the bug that Peng found, but that appears to be linked to the NFSv4.1 back channel, which these patches did not change.
The other point to note is that the patches do not touch the main read/write code and only lightly change the pNFS code paths. The impact of any residual bugs is therefore going to be limited.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Christoph Hellwig
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
And it doesn't even compile:
In file included from fs/nfs/client.c:51:0:fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ token fs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’fs/nfs/client.c: At top level:fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1
Signature · 6 lines
Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
And it doesn't even compile:
Quoted 9 lines
In file included from
fs/nfs/client.c:51:0:fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ tokenfs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’fs/nfs/client.c: At top level:fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1
Sigh. This should fix it... 8<------------------------------------------------------------------------------------- From b74e1d98b2027aa3ec46f3612143d0d35db92cf5 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH] NFS: Fix a typo in fs/nfs/pnfs.h
--- fs/nfs/pnfs.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/nfs/pnfs.h +1 -1
@@ -380,7 +380,7 @@ pnfs_roc_drain(struct inode *ino, u32 *barrier) }static inline void set_pnfs_layoutdriver (struct nfs_server *s, -const struct nfs_fh *mntfh,u32 id); +const struct nfs_fh *mntfh,u32 id) { }
Signature · 15 lines
1.7.6 -- Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 19 lines · Trond Myklebust
Quoted 8 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
And it doesn't even compile:
Quoted 9 lines
In file included from
fs/nfs/client.c:51:0:fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ tokenfs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’fs/nfs/client.c: At top level:fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1
Quoted 1 line
Sigh. This should fix it...
Only the first one, there are two independent places it broke. You missed this bit:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named
Signature · 6 lines
Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 20 lines · Jens Axboe
Quoted 19 lines · Trond Myklebust
Quoted 17 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
And it doesn't even compile:
In file included from
fs/nfs/client.c:51:0:fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ tokenfs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’fs/nfs/client.c: At top level:fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1
Quoted 6 lines
Quoted 1 line
Sigh. This should fix it...
Only the first one, there are two independent places it broke. You missed this bit:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named
How's this?
8<-------------------------------------------------------------------------------------- From e01e56eb7013a11f3e09bd8a8562332777f9e100 Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH v2] NFS: Re-enable compilation of nfs with !CONFIG_NFS_V4 ||
!CONFIG_NFS_V4_1
Fix a typo in fs/nfs/pnfs.h
Move the pnfs_blksize declaration outside the CONFIG_NFS_V4 section in struct nfs_server.
--- fs/nfs/pnfs.h | 2 +- include/linux/nfs_fs_sb.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/nfs/pnfs.h +1 -1
@@ -380,7 +380,7 @@ pnfs_roc_drain(struct inode *ino, u32 *barrier) }static inline void set_pnfs_layoutdriver (struct nfs_server *s, -const struct nfs_fh *mntfh,u32 id); +const struct nfs_fh *mntfh,u32 id) { }
include/linux/nfs_fs_sb.h +1 -1
@@ -131,6 +131,7 @@ struct nfs_server {struct fscache_cookie *fscache;/* superblock cookie */ #endif +u32 pnfs_blksize;/* layout_blksize attr */ #ifdef CONFIG_NFS_V4 u32 attr_bitmask[3 ];/* V4 bitmask representing the set of attributes supported on this
@@ -145,7 +146,6 @@ struct nfs_server {filesystem */ struct pnfs_layoutdriver_type *pnfs_curr_ld;/* Active layout driver */ struct rpc_wait_queue roc_rpcwaitq; -u32 pnfs_blksize;/* layout_blksize attr */ void *pnfs_ld_data;/* per mount point data */ /* the following fields are protected by nfs_client->cl_lock */
Signature · 10 lines
1.7.6 -- Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
Quoted 21 lines · Trond Myklebust
Quoted 20 lines · Jens Axboe
Quoted 19 lines · Trond Myklebust
Quoted 17 lines · Jens Axboe
Quoted 4 lines · Christoph Hellwig
How does this manage to include the pnfs block code, which only was posted on the list a few hours before the pull request, and which doesn't appear to past testing very well (although the major issue seems to be core nfs code)?
And it doesn't even compile:
In file included from
fs/nfs/client.c:51:0:fs/nfs/pnfs.h:384:1: error: expected identifier or ‘(’ before ‘{’ tokenfs/nfs/client.c: In function ‘nfs_server_set_fsinfo’:fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member named ‘pnfs_blksize’fs/nfs/client.c: At top level:fs/nfs/pnfs.h:382:20: warning: ‘set_pnfs_layoutdriver’ used but never defined [enabled by default]
make[2]: *** [fs/nfs/client.o] Error 1
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Sigh. This should fix it...
Only the first one, there are two independent places it broke. You missed this bit:
fs/nfs/client.c:939:8: error: ‘struct nfs_server’ has no member namedHow's this?
Yup, that gets both.
Signature · 6 lines
Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi all,
Quoted 10 lines · On Sun, 31 Jul 2011 15:58:45 -0400 Trond Myklebust…
How's this?
8<-------------------------------------------------------------------------------------- From
e01e56eb7013a11f3e09bd8a8562332777f9e100Mon Sep 17 00:00:00 2001
From: Trond Myklebust <[email protected]>
Date: Sun, 31 Jul 2011 15:33:26 -0400
Subject: [PATCH v2] NFS: Re-enable compilation of nfs with !CONFIG_NFS_V4 ||
!CONFIG_NFS_V4_1
Quoted 6 lines
Fix a typo in
fs/nfs/pnfs.h
Move thepnfs_blksizedeclaration outside theCONFIG_NFS_V4section in structnfs_server.Reported-by Jens Axboe <[email protected]>Signed-off-by Trond Myklebust <[email protected]>
I have applied this the linux-next today ..
Signature · 3 lines
Cheers, Stephen Rothwell [email protected] http://www.canb.auug.org.au/~sfr/
Quoted 2 lines · On Mon, 1 Aug 2011 10:57:39 +1000 Stephen Rothwell…
I have applied this the linux-next today ..
And then I checked Linus' tree again to find it already there, so I have started again from the his newer tree.
Signature · 3 lines
Cheers, Stephen Rothwell [email protected] http://www.canb.auug.org.au/~sfr/
Quoted 11 lines · Trond Myklebust
Hi Linus,
Please pull from the "nfs-for-3.1" branch of the repository at
git pull
git://git.linux-nfs.org/projects/trondmy/linux-nfs.gitnfs-for-3.1This will update the following files through the appended changesets.
Cheers,
Trond
Trond hi.
You have not included the two patches for the objects-layout bug-fixes I sent you. (And I don't see them in your linux-next branch). Please include them in the next round of fixes. (They are also for stable)
Do you need that I resend them?
Thanks
Boaz
Quoted 555 lines
----
fs/nfs/Kconfig | 8 +- fs/nfs/Makefile | 1 + fs/nfs/blocklayout/Makefile | 5 + fs/nfs/blocklayout/blocklayout.c | 1019 +++++++++++++++++++++++++++++++++++ fs/nfs/blocklayout/blocklayout.h | 207 +++++++ fs/nfs/blocklayout/blocklayoutdev.c | 410 ++++++++++++++ fs/nfs/blocklayout/blocklayoutdm.c | 111 ++++ fs/nfs/blocklayout/extents.c | 935 ++++++++++++++++++++++++++++++++ fs/nfs/client.c | 11 +- fs/nfs/dir.c | 57 ++- fs/nfs/nfs4_fs.h | 2 +- fs/nfs/nfs4filelayout.c | 2 +- fs/nfs/nfs4proc.c | 62 ++- fs/nfs/nfs4xdr.c | 233 ++++++++- fs/nfs/pnfs.c | 86 ++-- fs/nfs/pnfs.h | 28 +- include/linux/nfs.h | 2 + include/linux/nfs4.h | 1 + include/linux/nfs_fs.h | 3 +- include/linux/nfs_fs_sb.h | 4 +- include/linux/nfs_xdr.h | 17 +- 21 files changed, 3113 insertions(+), 91 deletions(-)commit
1c46c073ba7b1333d2d736056a081d4f17f5f1d3Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:56 2011 -0400For invalid extents, find other pages in the same fsblock and write them out.
[pnfsblock: write_begin]Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
579f69df9c99794f02878f7562a6906e8b2ac316Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:55 2011 -0400Signed-off-by Peng Tao <[email protected]>Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
0593a6123b6736f06823624813a94c863af7be87Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:54 2011 -0400Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.
When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.
[pnfsblock: bl_write_pagelist support functions][pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]Signed-off-by Fred Isaman <[email protected]>[pnfsblock: handle errors when read or write pagelist.]Signed-off-by Zhang Jingwang <[email protected]>[pnfs-block: use new write_pagelist api]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>[SQUASHME: pnfsblock: mds_offset is set in the generic layer]Signed-off-by Boaz Harrosh <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]Signed-off-by Peng Tao <[email protected]>[pnfsblock: SQUASHME: adjust to API change]Signed-off-by Fred Isaman <[email protected]>[pnfsblock: fixup blksize alignment in bl_setup_layoutcommit]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS]Signed-off-by Fred Isaman <[email protected]>[pnfsblock: handle errors when read or write pagelist.]Signed-off-by Zhang Jingwang <[email protected]>[pnfs-block: use new write_pagelist api]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
1371c564cf844d3f479527b623565ac262e6ebaaAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:53 2011 -0400Note: When upper layer's read/write request cannot be fulfilled, the block layout driver shouldn't silently mark the page as error. It should do what can be done and leave the rest to the upper layer. To do so, we should set rdata/wdata->res.count properly.
When upper layer re-send the read/write request to finish the rest part of the request, pgbase is the position where we should start at.
[pnfsblock: mark IO error with NFS_LAYOUT_{RW|RO}_FAILED]Signed-off-by Peng Tao <[email protected]>[pnfsblock: read path error handling]Signed-off-by Fred Isaman <[email protected]>[pnfsblock: handle errors when read or write pagelist.]Signed-off-by Zhang Jingwang <[email protected]>[pnfs-block: use new read_pagelist api]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
5dd59231aa9930bc3b217539da6d5eff1120173eAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:52 2011 -0400In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.In the new system where actual xdr encoding is done in
encode_layoutcommit()directly into xdr buffer, these are the new commit stages:1. On
setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication withbl_encode_layoutcommit&&bl_cleanup_layoutcommit(Generic layer provides a void-star to hang it on)2.
bl_encode_layoutcommitis called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup3. On cleanup the commit-extent-list is put back by a call to
set_to_rw()as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]Signed-off-by Jim Rees <[email protected]>[pnfsblock: introduce bl_committing list]Signed-off-by Peng Tao <[email protected]>[pnfsblock: SQUASHME: adjust to API change]Signed-off-by Fred Isaman <[email protected]>[blocklayout: encode_layoutcommit implementation]Signed-off-by Boaz Harrosh <[email protected]>[pnfsblock: fix bug setting up layoutcommit.]Signed-off-by Tao Guo <[email protected]>[pnfsblock: cleanup_layoutcommit wants a status parameter]Signed-off-by Boaz Harrosh <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
936a675fe578d40919046bf0a26f1776960a504fAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:51 2011 -0400In blocklayout driver. There are two things happening while layoutcommit/cleanup.
1. the modified extents are encoded.
2. On cleanup the extents are put back on the layout rw extents list, for reads.In the new system where actual xdr encoding is done in
encode_layoutcommit()directly into xdr buffer, these are the new commit stages:1. On
setup_layoutcommit, the range is adjusted as before and a structure is allocated for communication withbl_encode_layoutcommit&&bl_cleanup_layoutcommit(Generic layer provides a void-star to hang it on)2.
bl_encode_layoutcommitis called to do the actual encoding directly into xdr. The commit-extent-list is not freed and is stored on above structure.
FIXME: The code is not yet converted to the new XDR cleanup3. On cleanup the commit-extent-list is put back by a call to
set_to_rw()as before, but with no need for XDR decoding of the list as before. And the commit-extent-list is freed. Finally allocated structure is freed.[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()][pnfsblock: get rid of deprecated xdr macros]Signed-off-by Jim Rees <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Fred Isaman <[email protected]>[blocklayout: encode_layoutcommit implementation]Signed-off-by Boaz Harrosh <[email protected]>[pnfsblock: fix bug setting up layoutcommit.]Signed-off-by Tao Guo <[email protected]>[pnfsblock: prevent commit list corruption][pnfsblock: fix layoutcommit with an empty opaque]Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
84e5fe06f95c8ad330a525a5b11d21d8e371fbfbAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:50 2011 -0400Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
426b7d89de75cef8b308969e809f6242c72f620eAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:49 2011 -0400Adds working implementations of various support functions to handle INVAL extents, needed by writes, such as
bl_mark_sectors_initandbl_is_sector_init.[pnfsblock: fix 64-bit compiler warnings for extent manipulation]Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[Implement release_inval_marks]Signed-off-by Zhang Jingwang <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
cef25cef97ff6799c82fa467f030146aea8876dcAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:48 2011 -0400Implement
bl_find_get_extent(), one of the core extent manipulation routines.[pnfsblock: Lookup list entry of layouts and tags in reverse order]Signed-off-by Zhang Jingwang <[email protected]>Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>pnfsblock: fix print format warnings for
sector_tandsize_tgcc spews warnings about these on
x86_64, e.g.:
fs/nfs/blocklayout/blocklayout.c:74: warning: format ‘%Lu’ expects type ‘long long unsigned int’, but argument 2 has type ‘sector_t’fs/nfs/blocklayout/blocklayout.c:388: warning: format ‘%d’ expects type ‘int’, but argument 5 has type ‘size_t’Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
6cb996c2b07c4e0c48d5e67a5cc24e8211d89addAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:47 2011 -0400XDR decodes the block layout payload sent in LAYOUTGET result, storing the result in an extent list.
[pnfsblock: get rid of deprecated xdr macros]Signed-off-by Jim Rees <[email protected]>Signed-off-by Fred Isaman <[email protected]>[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]Signed-off-by Tao Guo <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
4d9a1fcdcbf13e6e65a4b0475af7d4fb139d1842Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:46 2011 -0400Call GETDEVICELIST during mount, then call and parse GETDEVICEINFO for each device returned.
[pnfsblock: get rid of deprecated xdr macros]Signed-off-by Jim Rees <[email protected]>[pnfsblock: fix pnfs_deviceid references]Signed-off-by Fred Isaman <[email protected]>[pnfsblock: fix print format warnings for sector_t and size_t][pnfs-block: #include <linux/vmalloc.h>][pnfsblock: no PNFS_NFS_SERVER]Signed-off-by Benny Halevy <[email protected]>[pnfsblock: fix bug determining size of striped volume][pnfsblock: fix oops when using multiple devices]Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: get rid of vmap and deviceid->area structure]Signed-off-by Peng Tao <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
f0f45cd2693863faa19f0edfc3f16d77159f7838Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:45 2011 -0400Replace a stub, so that extents underlying the layouts are properly added, merged, or ignored as necessary.
Signed-off-by Fred Isaman <[email protected]>[pnfsblock: delete the new node before put it]Signed-off-by Mingyang Guo <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
03e023d1027458d13c0007f0e4456b65ee2977f1Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:44 2011 -0400Signed-off-by Fred Isaman <[email protected]>[pnfsblock: fix bug getting pnfs_layout_type in translate_devid().]Signed-off-by Tao Guo <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Zhang Jingwang <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
be4619cf2c94aad7a9853e4ab32ce31565b9335cAuthor: Jim Rees <[email protected]> Date: Sat Jul 30 20:52:43 2011 -0400Signed-off-by Jim Rees <[email protected]>Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[upcall bugfixes]Signed-off-by Peng Tao <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
db1905c6f31ac745573ffb3bf860e07088dd8713Author: Jim Rees <[email protected]> Date: Sat Jul 30 20:52:42 2011 -0400Signed-off-by Jim Rees <[email protected]>Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[upcall bugfixes]Signed-off-by Peng Tao <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
7c8b2c6c3f8bd394401da844badd28990261dd2aAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:41 2011 -0400Adds structures and basic create/delete code for extents.
Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Zhang Jingwang <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
be79608be504916f9534af194ca8d5b612606484Author: Benny Halevy <[email protected]> Date: Sat Jul 30 20:52:40 2011 -0400[pnfsblock: use pnfs_generic_pg_init_read/write]Signed-off-by Peng Tao <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
8f7e205d20c74fbaab3e4942e5daa09f0787fdd7Author: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:39 2011 -0400Define a configuration variable to enable/disable compilation of the block driver code.
Add the minimal structure for a pnfs block layout driver, and empty list-heads that will hold the extent data
[pnfsblock: make NFS_V4_1 select PNFS_BLOCK]Signed-off-by Peng Tao <[email protected]>Signed-off-by Fred Isaman <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfs-block: fix CONFIG_PNFS_BLOCK dependencies]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: SQUASHME: adjust to API change]Signed-off-by Fred Isaman <[email protected]>[pnfs: move pnfs_layout_type inline in nfs_inode]Signed-off-by Benny Halevy <[email protected]>[blocklayout: encode_layoutcommit implementation]Signed-off-by Boaz Harrosh <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: layout alloc and free]Signed-off-by Fred Isaman <[email protected]>[pnfs: move pnfs_layout_type inline in nfs_inode]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[pnfsblock: define module alias]Signed-off-by Peng Tao <[email protected]>[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
bec3f732e912295462c795d9947a7ed59f85caeeAuthor: Andy Adamson <[email protected]> Date: Sat Jul 30 20:52:38 2011 -0400This gives layout driver a chance to cleanup structures they put in at
encode_layoutcommit.Signed-off-by Andy Adamson <[email protected]>[fixup layout header pointer for layoutcommit]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>[rm inode and pnfs_layout_hdr args from cleanup_layoutcommit()]Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
009354f2e7d197a29ca08d5f2c0bbe6e806855ecAuthor: Fred Isaman <[email protected]> Date: Sat Jul 30 20:52:37 2011 -0400Block layout needs it to determine IO size.
Signed-off-by Fred Isaman <[email protected]>Signed-off-by Tao Guo <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
8d619fb13ecf6f56eabf88d7fa023cefb9832d2eAuthor: Benny Halevy <[email protected]> Date: Sat Jul 30 20:52:36 2011 -0400To allow layout driver to issue getdevicelist at mount time, and clean up at umount time.
[fixup non NFS_V4_1 set_pnfs_layoutdriver definition][pnfs: pass mntfh down the init_pnfs path]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
3c8507c7447b671ce14c7fc250c3cd7d9f338706Author: Andy Adamson <[email protected]> Date: Sat Jul 30 20:52:35 2011 -0400The block driver uses GETDEVICELIST
Signed-off-by Andy Adamson <[email protected]>[pass struct nfs_server * to getdevicelist][get machince creds for getdevicelist][fix getdevicelist decode sizing]Signed-off-by Benny Halevy <[email protected]>Signed-off-by Benny Halevy <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
53e8c77ea157bb1888cf33a99de11c3d967a58dfAuthor: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:34 2011 -0400Using
NFS4_MAX_UINT64will break current protocol.[Needed in v3.0]CC Stable Tree <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
eb425aadaed93b8499bebd8802c669b1b3d592f6Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:33 2011 -0400There can be multiple lseg per file, so layoutcommit should be able to handle it.
[Needed in v3.0]CC Stable Tree <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Boaz Harrosh <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
d7060761432676d0249f680ddb4d130afb0bd5c9Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:32 2011 -0400No need to save it for every lseg.
No need to save it at everypnfs_set_layoutcommit.[Needed in v3.0]CC Stable Tree <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Boaz Harrosh <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
8b7116b52046b95874f38c519b6e8923d9d1b113Author: Peng Tao <[email protected]> Date: Sat Jul 30 20:52:31 2011 -0400No need to save it for every lseg.
[Needed in v3.0]CC Stable Tree <[email protected]>Signed-off-by Peng Tao <[email protected]>Signed-off-by Jim Rees <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
374e4e3ec347d4f70e1417ce3e1c454f133dab72Author: Bryan Schumaker <[email protected]> Date: Fri Jul 29 11:49:06 2011 -0400Additional readdir cookie loop information
Print out the name of the file that triggers the cookie loop message to make it slightly easier to track down the cause.
Signed-off-by Bryan Schumaker <[email protected]>Signed-off-by Trond Myklebust <[email protected]>commit
0c0308066ca53fdf1423895f3a42838b67b3a5a8Author: Trond Myklebust <[email protected]> Date: Sat Jul 30 12:45:35 2011 -0400NFS: Fix spurious readdir cookie loop messages
If the directory contents change, then we have to accept that the
file->f_posvalue may shrink if we do a 'search-by-cookie'. In that case, we should turn off the loop detection and let the NFS client try to recover.The patch also fixes a second loop detection bug by ensuring that after turning on the
ctx->dupedflag, we read at least one new cookie intoctx->dir_cookiebefore attempting to match withctx->dup_cookie.Reported-by Petr Vandrovec <[email protected]>Cc [email protected] [2.6.39+]Signed-off-by Trond Myklebust <[email protected]>
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBCb2F6IEhhcnJvc2ggW21haWx0 bzpiaGFycm9zaEBwYW5hc2FzLmNvbV0NCj4gU2VudDogV2VkbmVzZGF5LCBBdWd1c3QgMDMsIDIw MTEgNzo1MiBQTQ0KPiBUbzogTXlrbGVidXN0LCBUcm9uZA0KPiBDYzogTGludXMgVG9ydmFsZHM7 IE5GUyBsaXN0DQo+IFN1YmplY3Q6IFJlOiBQbGVhc2UgcHVsbCBORlMgY2xpZW50IGNoYW5nZXMN Cj4gDQo+IE9uIDA3LzMwLzIwMTEgMDg6MTkgUE0sIFRyb25kIE15a2xlYnVzdCB3cm90ZToNCj4g PiBIaSBMaW51cywNCj4gPg0KPiA+IFBsZWFzZSBwdWxsIGZyb20gdGhlICJuZnMtZm9yLTMuMSIg YnJhbmNoIG9mIHRoZSByZXBvc2l0b3J5IGF0DQo+ID4NCj4gPiAgICBnaXQgcHVsbCBnaXQ6Ly9n aXQubGludXgtbmZzLm9yZy9wcm9qZWN0cy90cm9uZG15L2xpbnV4LW5mcy5naXQNCj4gbmZzLWZv ci0zLjENCj4gPg0KPiA+IFRoaXMgd2lsbCB1cGRhdGUgdGhlIGZvbGxvd2luZyBmaWxlcyB0aHJv dWdoIHRoZSBhcHBlbmRlZCBjaGFuZ2VzZXRzLg0KPiA+DQo+ID4gICBDaGVlcnMsDQo+ID4gICAg IFRyb25kDQo+ID4NCj4gDQo+IFRyb25kIGhpLg0KPiANCj4gWW91IGhhdmUgbm90IGluY2x1ZGVk IHRoZSB0d28gcGF0Y2hlcyBmb3IgdGhlIG9iamVjdHMtbGF5b3V0IGJ1Zy1maXhlcw0KPiBJIHNl bnQgeW91LiAoQW5kIEkgZG9uJ3Qgc2VlIHRoZW0gaW4geW91ciBsaW51eC1uZXh0IGJyYW5jaCku IFBsZWFzZQ0KPiBpbmNsdWRlIHRoZW0gaW4gdGhlIG5leHQgcm91bmQgb2YgZml4ZXMuIChUaGV5 IGFyZSBhbHNvIGZvciBzdGFibGUpDQo+IA0KPiBEbyB5b3UgbmVlZCB0aGF0IEkgcmVzZW5kIHRo ZW0/DQoNClllcywgcGxlYXNlIHJlc2VuZCB0aGVtLCBzbyB0aGF0IEkgY2FuIHF1ZXVlIHRoZW0g dXAgZm9yIGEgcG9zdC1yYzEgYnVnZml4ZXMgcHVsbC4NCg0KQ2hlZXJzDQogIFRyb25kDQoT77+9 77+97Lm7HO+/vSbvv71+77+9Ju+/vRjvv73vv70rLe+/ve+/vd22F++/ve+/vXfvv73vv73Lm++/ ve+/ve+/vW3vv71i77+977+9Z37Ip++/vRfvv73vv73cqH3vv73vv73vv73GoHrvv70majordu+/ ve+/ve+/vQfvv73vv73vv73vv716Wivvv73vv70rembvv73vv73vv71o77+977+977+9fu+/ve+/ ve+/ve+/vWnvv73vv73vv71677+9Hu+/vXfvv73vv73vv70/77+977+977+977+9Ju+/vSnfohtm
PATCH 1/1 pnfs-obj: Fix the comp_index != 0 case
There were bugs in the case of partial layout where olo_comp_index is not zero. This used to work and was tested but one of the later cleanup SQUASHMEs broke it and was not tested since.
Also add a dprint that specify those received layout parameters. Everything else was already printed.
--- fs/nfs/objlayout/objio_osd.c | 16 +++++++--------- fs/nfs/objlayout/pnfs_osd_xdr_cli.c | 3 +++ 2 files changed, 10 insertions(+), 9 deletions(-)
fs/nfs/objlayout/objio_osd.c +7 -9
@@ -479,7 +479,6 @@ static int _io_check(struct objio_state *ios, bool is_write)for (i =0 ; i < ios->numdevs; i++) {struct osd_sense_info osi;struct osd_request *or = ios->per_dev[i].or; -unsigned dev;int ret;if (!or)
@@ -500,9 +499,8 @@ static int _io_check(struct objio_state *ios, bool is_write)continue ;/* we recovered */ } - dev = ios->per_dev[i].dev; - objlayout_io_set_result(&ios->ol_state, dev, - &ios->layout->comps[dev].oc_object_id, + objlayout_io_set_result(&ios->ol_state, i, + &ios->layout->comps[i].oc_object_id, osd_pri_2_pnfs_err(osi.osd_err_pri), ios->per_dev[i].offset, ios->per_dev[i].length,
@@ -648,7 +646,7 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,int ret =0 ;while (length) { -struct _objio_per_comp *per_dev = &ios->per_dev[dev]; +struct _objio_per_comp *per_dev = &ios->per_dev[dev - first_dev];unsigned cur_len, page_off =0 ;if (!per_dev->length) {
@@ -668,8 +666,8 @@ static int _prepare_one_group(struct objio_state *ios, u64 length, cur_len = stripe_unit; } -if (max_comp < dev) - max_comp = dev; +if (max_comp < dev - first_dev) + max_comp = dev - first_dev; }else { cur_len = stripe_unit; }
@@ -804,7 +802,7 @@ static int _read_mirrors(struct objio_state *ios, unsigned cur_comp)struct _objio_per_comp *per_dev = &ios->per_dev[cur_comp];unsigned dev = per_dev->dev;struct pnfs_osd_object_cred *cred = - &ios->layout->comps[dev]; + &ios->layout->comps[cur_comp];struct osd_obj_id obj = { .partition = cred->oc_object_id.oid_partition_id, .id = cred->oc_object_id.oid_object_id,
@@ -902,7 +900,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)for (; cur_comp < last_comp; ++cur_comp, ++dev) {struct osd_request *or =NULL ;struct pnfs_osd_object_cred *cred = - &ios->layout->comps[dev]; + &ios->layout->comps[cur_comp];struct osd_obj_id obj = { .partition = cred->oc_object_id.oid_partition_id, .id = cred->oc_object_id.oid_object_id,
fs/nfs/objlayout/pnfs_osd_xdr_cli.c +3 -0
@@ -170,6 +170,9 @@ int pnfs_osd_xdr_decode_layout_map(struct pnfs_osd_layout *layout, p =_osd_xdr_decode_data_map (p, &layout->olo_map); layout->olo_comps_index =be32_to_cpup (p++); layout->olo_num_comps =be32_to_cpup (p++); +dprintk ("%s: olo_comps_index=%d olo_num_comps=%d\n" , __func__, + layout->olo_comps_index, layout->olo_num_comps); + iter->total_comps = layout->olo_num_comps;return 0 ; }
Signature · 1 line
1.7.6
Re: [PATCH] pnfs-obj: Fix the comp_index != 0 case
Opps: [PATCH 2/2] pnfs-obj: Fix the comp_index != 0 case
Thanks
Quoted 92 lines · Boaz Harrosh
There were bugs in the case of partial layout where
olo_comp_indexis not zero. This used to work and was tested but one of the later cleanup SQUASHMEs broke it and was not tested since.Also add a dprint that specify those received layout parameters. Everything else was already printed.
--- fs/nfs/objlayout/objio_osd.c | 16 +++++++--------- fs/nfs/objlayout/pnfs_osd_xdr_cli.c | 3 +++ 2 files changed, 10 insertions(+), 9 deletions(-)fs/nfs/objlayout/objio_osd.c +7
-9@@ -479,7 +479,6 @@ static int _io_check(struct objio_state *ios, bool is_write)for (i =0 ; i < ios->numdevs; i++) {struct osd_sense_info osi;struct osd_request *or = ios->per_dev[i].or; -unsigned dev;int ret;if (!or)@@ -500,9 +499,8 @@ static int _io_check(struct objio_state *ios, bool is_write)continue ;/* we recovered */ } - dev = ios->per_dev[i].dev; - objlayout_io_set_result(&ios->ol_state, dev, - &ios->layout->comps[dev].oc_object_id, + objlayout_io_set_result(&ios->ol_state, i, + &ios->layout->comps[i].oc_object_id, osd_pri_2_pnfs_err(osi.osd_err_pri), ios->per_dev[i].offset, ios->per_dev[i].length,@@ -648,7 +646,7 @@ static int _prepare_one_group(struct objio_state *ios, u64 length,int ret =0 ;while (length) { -struct _objio_per_comp *per_dev = &ios->per_dev[dev]; +struct _objio_per_comp *per_dev = &ios->per_dev[dev - first_dev];unsigned cur_len, page_off =0 ;if (!per_dev->length) {@@ -668,8 +666,8 @@ static int _prepare_one_group(struct objio_state *ios, u64 length, cur_len = stripe_unit; } -if (max_comp < dev) - max_comp = dev; +if (max_comp < dev - first_dev) + max_comp = dev - first_dev; }else { cur_len = stripe_unit; }@@ -804,7 +802,7 @@ static int _read_mirrors(struct objio_state *ios, unsigned cur_comp)struct _objio_per_comp *per_dev = &ios->per_dev[cur_comp];unsigned dev = per_dev->dev;struct pnfs_osd_object_cred *cred = - &ios->layout->comps[dev]; + &ios->layout->comps[cur_comp];struct osd_obj_id obj = { .partition = cred->oc_object_id.oid_partition_id, .id = cred->oc_object_id.oid_object_id,@@ -902,7 +900,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)for (; cur_comp < last_comp; ++cur_comp, ++dev) {struct osd_request *or =NULL ;struct pnfs_osd_object_cred *cred = - &ios->layout->comps[dev]; + &ios->layout->comps[cur_comp];struct osd_obj_id obj = { .partition = cred->oc_object_id.oid_partition_id, .id = cred->oc_object_id.oid_object_id,fs/nfs/objlayout/pnfs_osd_xdr_cli.c +3
-0@@ -170,6 +170,9 @@ int pnfs_osd_xdr_decode_layout_map(struct pnfs_osd_layout *layout, p =_osd_xdr_decode_data_map (p, &layout->olo_map); layout->olo_comps_index =be32_to_cpup (p++); layout->olo_num_comps =be32_to_cpup (p++); +dprintk ("%s: olo_comps_index=%d olo_num_comps=%d\n" , __func__, + layout->olo_comps_index, layout->olo_num_comps); + iter->total_comps = layout->olo_num_comps;return 0 ; }
PATCH 1/2 pnfs-obj: Bug when we are running out of bio
In reply to Myklebust, TrondWhen we have a situation that the number of pages we want to encode is bigger then the size of the bio. (Which can currently happen only when all IO is going to a single device .e.g group_width==1) then the IO is submitted short and we report back only the amount of bytes we actually wrote/read and all is fine. BUT ...
There was a bug that the current length counter was advanced before the fail to add the extra page, and we come to a situation that the CDB length was one-page longer then the actual bio size, which is of course rejected by the osd-target.
While here also fix the bio size calculation, in the case that we received more then one group of devices.
--- fs/nfs/objlayout/objio_osd.c | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-)
fs/nfs/objlayout/objio_osd.c +5 -7
@@ -589,22 +589,19 @@ static void _calc_stripe_info(struct objio_state *ios, u64 file_offset, }static int _add_stripe_unit (struct objio_state *ios,unsigned *cur_pg, -unsigned pgbase,struct _objio_per_comp *per_dev,int cur_len, +unsigned pgbase,struct _objio_per_comp *per_dev,int len,gfp_t gfp_flags) {unsigned pg = *cur_pg; +int cur_len = len;struct request_queue *q =osd_request_queue (_io_od (ios, per_dev->dev)); - per_dev->length += cur_len; -if (per_dev->bio ==NULL ) { -unsigned stripes = ios->layout->num_comps / - ios->layout->mirrors_p1; -unsigned pages_in_stripe = stripes * +unsigned pages_in_stripe = ios->layout->group_width * (ios->layout->stripe_unit /PAGE_SIZE );unsigned bio_size = (ios->ol_state.nr_pages + pages_in_stripe) / - stripes; + ios->layout->group_width;if (BIO_MAX_PAGES_KMALLOC < bio_size) bio_size =BIO_MAX_PAGES_KMALLOC ;
@@ -632,6 +629,7 @@ static int _add_stripe_unit(struct objio_state *ios, unsigned *cur_pg, }BUG_ON (cur_len); + per_dev->length += len; *cur_pg = pg;return 0 ; }
Signature · 1 line
1.7.6