pnfs git tree status pnfs-all-2.6.38-rc2-2011-01-27
Subsystems (2)
I rebased the pnfs tree onto 2.6.38-rc2 (that has the wave2 patches) and tagged the pnfs-all-latest tip as pnfs-all-2.6.38-rc2-2011-01-27
Note: spnfs-block is currently broken due to the fallocate() API change.
Cumulative patches can be generated fromgit://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.38-rc2 pnfs-all-2.6.38-rc2-2011-01-27
Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
Latest patches that went in since pnfs-all-2.6.37-rc3-2010-11-23:
spnfs-block:
Benny Halevy b4dc7fe spnfs-block: fix bl_ops.c for new blkdev_get_by_dev()
Benny Halevy a051448 FIXME: fix bl_ops use of fallocate
pnfs-submit:
Fred Isaman 021fa68 DEBUG: add some BUG_ON(refcount==0) statements Fred Isaman a9b6a3e POSTPONE: lseg refcounting Fred Isaman 5de505c debug: add assert_spin_locked Fred Isaman 24e076e pnfs-submit: wave4: fix bug dealing with commit split between DS and MDS
pnfs:
Fred Isaman 7bbc4e7 pnfs-submit: revert range simplifications made by submission patches Fred Isaman cbc79e3 POSTPONE: restore has_layout Fred Isaman 3a53a23 pnfs_submit: layoutcommit Fred Isaman e214a86 pnfs-submit: Turn off layoutcommits Fred Isaman 3c0b776 POST_SUBMIT: restore pnfs_return_layout Fred Isaman 008932b REVERT ME: "Fix callback code" Fred Isaman be9d7b0 DELETE ME: move rpcwaitq from nfs_server to nfs_inode Fred Isaman c4df508 DELETE ME: remove list_del optimization from pnfs_free_lseg_list Fred Isaman 2d67203 REVERT ME: "Fix wait bug in pnfs_roc_drain" Fred Isaman 9ca7b1b REVERT ME: "Add prefix to struct pnfs_layout_hdr fields" Fred Isaman ae88286 REVERT ME: "Add prefix to struct pnfs_layout_segment fields" Fred Isaman f84b32e REVERT ME: change roc calling convention Fred Isaman 46555cb REVERT ME: NFS_LAYOUT_DESTROYED flag Fred Isaman 3fbab3a REVERT ME: join pnfs_clear_lseg_list and nfs4_asynch_forget_layouts Fred Isaman 67901ed REVERT ME: add recall_counts Fred Isaman b5365d9 DELETE ME: outdated comment Fred Isaman b050cbe DELETE ME: pointless dprintk info Fred Isaman 6a536e9 DELETE ME: whitespace change Fred Isaman ec961df DELETE ME: incorrect comment Fred Isaman bb23404 DELETE ME: making should_free_lseg public Fred Isaman 0581f32 SQUASHME: Remove outdated WARN Benny Halevy f1ed0cc Revert "pnfs-submit: wave2: remove forgotten layoutreturn struct definitions" Benny Halevy 0f685c1 Revert "pnfs-submit: Turn off layoutcommits" Benny Halevy 8b00189 pnfs: LAYOUTRETURN code Benny Halevy b16e749 Revert "pnfs-submit: wave2: Remove LAYOUTRETURN from return on close" Benny Halevy b5d445a FIXME: roc should return layout on last close Benny Halevy 1c45467 Revert "pnfs-submit: wave2: remove cl_layoutrecalls list" Benny Halevy 7413bae Revert "pnfs-submit: wave2: Pull out all recall initiated LAYOUTRETURNS" Benny Halevy 7ca4ff5 Revert "pnfs-submit: wave2: Don't wait in layoutget" Benny Halevy f9f44b6 Revert "pnfs-submit: wave2: check that partial LAYOUTGET return is ignored" Benny Halevy f713392 SQUASHME: pnfs: call pnfs_choose_layoutget_stateid under the inode lock
pnfs-block:
Benny Halevy e834aeb SQUASHME: pnfs-block: fixup layoutcommit methods args
Jim Rees c0243f2 pnfs-block: fix blocklayoutdev.c for new blkdev_get_by_dev()
pnfs-obj-all:
Benny Halevy f5eb2e4 SQUASHME: pnfs-obj: fixup layoutcommit methods args
pnfs git tree status pnfs-all-2.6.38-rc3-2011-02-03
I rebased the pnfs tree onto 2.6.38-rc3, with Fred's lock inversion fix in place and Andy's reordering of pnfs-submit for wave3 and wave4. pnfs-all-latest tip is tagged as pnfs-all-2.6.38-rc3-2011-02-03
Note: spnfs-block is currently broken due to the fallocate() API change.
Cumulative patches can be generated fromgit://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.38-rc3 pnfs-all-2.6.38-rc3-2011-02-03
Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
A new branch, pnfs-submit-wave3 points at wave3 candidate patches and pnfs-submit points at current wave4 patches.
pnfs-submit-wave3:
Fred Isaman 1786469 pnfs-submit: wave3: lseg refcounting The pNFS Team 6d48359 pnfs_submit: add data server session to nfs4_setup_sequence The pNFS Team 3dec9b6 pnfs_submit: update nfs4_async_handle_error for data server The pNFS Team 9390f23 pnfs_submit: update state renewal for data servers Andy Adamson 790dbeb pnfs_submit: wave3 pageio-helpers Andy Adamson 54583dc pnfs_submit: wave3 associate layout segment with nfs_page The pNFS Team c85562c pnfs_submit: filelayout policy operations The pNFS Team a5437b7 pnfs_submit: filelayout i/o helpers Andy Adamson bca89961 pnfs_submit: wave3 generic read The pNFS Team a9c9c75 pnfs_submit: filelayout read The pNFS Team 60882f9 pnfs_submit: increase NFS_MAX_FILE_IO_SIZE Andy Adamson 3ad8d6c pnfs_submit: enforce requested DS only pNFS role
pnfs-submit:
Fred Isaman 3f48c62 DEBUG: add some BUG_ON(refcount==0) statements Fred Isaman 502388a debug: add assert_spin_locked Fred Isaman a655dac pnfs-submit: Add WARN_ONs removed by submission code Andy Adamson c11704d pnfs_submit: wave4 pageio-helpers Andy Adamson 8fb9114 pnfs_submit: wave4 associate layout segment with nfs_page Andy Adamson de2c655 pnfs_submit: wave4 generic read The pNFS Team 302270f pnfs_submit: generic write The pNFS Team f978c0a pnfs_submit: data server write with no getattr The pNFS Team 15e01aa pnfs_submit: filelayout write The pNFS Team f184813 pnfs_submit: signal layoutdriver commit The pNFS Team b4e299a pnfs_submit: generic commit The pNFS Team af37694 pnfs_submit: data server commit with no getattr The pNFS Team cf279a7 pnfs_submit: filelayout commit Fred Isaman 13f429d pnfs-submit: wave4: fix bug dealing with commit split between DS and MDS
Latest patches that went in since pnfs-all-2.6.38-rc2-2011-01-27:
nfs-upstream:
Fred Isaman 995a73b pnfs: avoid incorrect use of layout stateid Fred Isaman 8073b9d pnfs: do not need to clear NFS_LAYOUT_BULK_RECALL flag Fred Isaman de5f699 pnfs: fix pnfs lock inversion of i_lock and cl_lock
pnfs-all-latest:
Benny Halevy 5186ca3 SQUASHME: pnfs-block: use pnfs_layout_hdr field prefix
Benny Halevy 4c5fcc5 SQUASHME: pnfs-obj: use pnfs_layout_hdr and layout_segment field prefix
pnfs git tree status pnfs-all-2.6.38-rc5-2011-02-26
I pushed the Connectathon fixes for pnfs-all-latest to the main treegit://linux-nfs.org/~bhalevy/linux-pnfs.git
And I'll sync with Fred's wave[345] series when he's ready.
New commits submitted since last update (pnfs-all-2.6.38-rc3-2011-02-03) are listed below.
Benny
pnfs-all-2.6.37:
Benny Halevy 6d105b3 pnfsd: Initialize cb_layout clr_recall.cb_work Benny Halevy f8282df pnfsd: add default error case in nfsd4_cb_layout_done Benny Halevy 488dbb1 pnfsd: do not clear rpc_resp in nfsd4_cb_done_sequence Benny Halevy 9243514 pnfs: fix missing unlock in nfs_client_return_layouts Benny Halevy 38a4a8d pnfs: return bool status from nfs4_asynch_forget_layouts Benny Halevy 1cabadb pnfs: return NFS4ERR_DELAY from layoutrecall while waiting on lsegs Benny Halevy be52477 pnfs: fix cl_drain_notification indexing Benny Halevy 257b10f pnfs: fix nfs4_asynch_forget_layouts accounting Benny Halevy 013dffa pnfs: improve nfs4_layoutget_done debug printouts Benny Halevy 7355ec0 pnfs: fix lock inversion in pnfs_layout_process
nfs-upstream:
Trond Myklebust ec45b78 SUNRPC: Fix a memory leak in rpc_new_task() Trond Myklebust 3561b23 SUNRPC: Close a race in __rpc_wait_for_completion_task() Trond Myklebust bcaedce NFS/VFS: Fix the nfs sillyrename regression Benny Halevy 4856b2d NFSv4.1: reject zero layout with zeroed stripe unit Jim Rees b76d1d5 NFSv4.1: fix typo in filelayout_check_layout
nfsd-upstream:
Benny Halevy 767d1a0 NFSD: fix decode_cb_sequence4resok
Benny Halevy ebca1bc nfsd41: use SEQ4_STATUS_BACKCHANNEL_FAULT when cb_sequence is invalid
pnfsd:
Benny Halevy 9bb661a pnfsd: do not clear rpc_resp in nfsd4_cb_done_sequence
pnfsd-lexp:
Benny Halevy 3e947f3 pnfsd-lexp: recall layout on truncate
pnfs-submit-wave3:
Andy Adamson 733c2ca NFSv4: remove CONFIG_NFS_V4 from nfs_read_data Andy Adamson e347996 NFSv4.1: put_layout_hdr can remove nfsi->layout Andy Adamson 2e5bfda NFS move nfs_client initialization into nfs_get_client Andy Adamson 31642d8 NFSv4.1: send zero stateid seqid on v4.1 i/o Andy Adamson 078fb11 NFSv4.1: new flag for state renewal check Andy Adamson 45429c7b NFSv4.1: new flag for lease time check Andy Adamson a557a45 NFSv4.1: add MDS mount DS only check Fred Isaman 42e7c6e NFSv4.1: lseg refcounting Fred Isaman 28377e6 NFSv4.1: coelesce across layout stripes Fred Isaman f5650f9 NFSv4.1: shift pnfs_update_layout locations Andy Adamson ab7ffb5 NFSv4.1: generic read Andy Adamson 242f5ce NFSv4.1: data server connection Fred Isaman 4374a1b NFSv4.1: filelayout i/o helpers Andy Adamson fcb2ee6 NFSv4.1: filelayout read Andy Adamson 47c3fd3 NFSv4.1: filelayout async error handler Christoph Hellwig 627c22e NFSv4.1 move deviceid cache to filelayout driver Andy Adamson 29b9fde NFSv4.1: turn off pNFS on ds connection failure Fred Isaman 442d08c NFSv4.1: lseg documentation Fred Isaman 6205ddc SQUASHME wave3: address Benny's comment Fred Isaman d3ff5cd SQUASHME wave3: whitespace fix Benny Halevy 9d0fcee NFSv4.1: pnfs-submit-wave3: put_lseg_common should use list_del_init
pnfs-submit-wave4:
Fred Isaman 0479cd6 NFSv4.1: add callback to nfs4_write_done Fred Isaman 55f54f7 NFSv4.1: trigger LAYOUTGET for writes Fred Isaman 0c992aa NFSv4.1: remove GETATTR from ds writes Fred Isaman 7326c07 NFSv4.1: pnfs filelayout driver write
pnfs-submit:
Benny Halevy 34af0eb NFSv4.1: pnfs-submit-wave3: put_lseg_common should use list_del_init
pnfs:
Benny Halevy 67e864c pnfs: fix missing unlock in nfs_client_return_layouts Benny Halevy 787d5ab pnfs: return bool status from nfs4_asynch_forget_layouts Benny Halevy 5564508 pnfs: return NFS4ERR_DELAY from layoutrecall while waiting on lsegs Benny Halevy 051d190 pnfs: fix cl_drain_notification indexing Benny Halevy e60a67c pnfs: fix nfs4_asynch_forget_layouts accounting Benny Halevy a86c7ac pnfs: improve nfs4_layoutget_done debug printouts Benny Halevy a92a66b SQUASHME: pnfs: fix lock inversion in pnfs_layout_process Benny Halevy b2005e6 SQUASHME: pnfs: revert layout recall/get/return synchronization
pnfs git tree status pnfs-all-2.6.38-2011-03-25
With the release of 2.6.38 I've rebased the pnfs tree on top of it and created the pnfs-all-2.6.38 branch to hold the stream.
A new branch, pnfs-for-2.6.39, holds a cleaned-up series of the pnfs branch, to be rebased onto v2.6.39-rc* after Linus pulls the wave5 patches from nfs-for-2.6.39.
The patched in pnfs-for-2.6.39 are listed below
Benny
fs/nfs/callback.h | 20 ++ fs/nfs/callback_proc.c | 34 ++- fs/nfs/callback_xdr.c | 96 ++++++- fs/nfs/client.c | 7 +- fs/nfs/direct.c | 160 ++++++----- fs/nfs/file.c | 2 +- fs/nfs/inode.c | 11 +- fs/nfs/nfs4_fs.h | 13 +- fs/nfs/nfs4filelayout.c | 9 +- fs/nfs/nfs4proc.c | 273 +++++++++++++++++- fs/nfs/nfs4xdr.c | 396 +++++++++++++++++++++++++ fs/nfs/pnfs.c | 725 ++++++++++++++++++++++++++++++++++++++++----- fs/nfs/pnfs.h | 198 ++++++++++++- fs/nfs/read.c | 23 ++- fs/nfs/super.c | 25 ++ fs/nfs/write.c | 23 ++- include/linux/nfs4.h | 3 + include/linux/nfs_fs_sb.h | 3 + include/linux/nfs_xdr.h | 74 +++++ 19 files changed, 1919 insertions(+), 176 deletions(-)
commit 155d9fde992ff98edad95c95ea554b7ad3b6794b
Author: J. Bruce Fields <[email protected]> Date: Fri Jul 17 17:51:29 2009 -0400
pnfs: client stats
A pNFS client auto-negotiates a lot of features (minorversion level, pNFS layout type, etc.). This is convenient, but makes certain kinds of failures hard for a user to detect.
For example, if the client falls back on 4.0, or falls back to MDS IO because the user didn't connect to the right iscsi disks before mounting, the only symptoms may be reduced performance, which may not be noticed till long after the actual failure, and may be difficult for a user to diagnose.
However, such "failures" may also be perfectly normal in some cases, so we don't want to spam the system logs with them.
One approach would be to put some more information into /proc/self/mountstats.
There is a build error when CONFIG_NFS_V4 is set but CONFIG_NFS_V4_1 is *not* set. show_sessions() prototype was unbalanced between the two cases.
commit ced8029bc8f2efa0933030b47529b1df8480b1bf
Author: Benny Halevy <[email protected]> Date: Thu Feb 17 12:43:12 2011 -0500
pnfs: improve nfs4_layoutget_done debug printouts
commit 96f5c53473ce08a539d2671a94d7011cc397a5e3
Author: Benny Halevy <[email protected]> Date: Mon Jul 12 18:50:04 2010 +0300
pnfs: mark page with error in readpage_async_filler when crossing lsegs
commit fbd73560e503cc9c1b20f60d7d8588d113f3baea
Author: Benny Halevy <[email protected]> Date: Tue Sep 28 21:22:53 2010 +0200
pnfs: alloc and free layout_hdr layoutdriver methods
commit 3c5cf965115d69da8d5836229055a2440037050c
Author: Benny Halevy <[email protected]> Date: Thu May 6 20:09:02 2010 +0300
FIXME: pnfs: per mount layout driver private data
Temporary relief until we convert to use generic device cache.
commit d4ace0226c398ebd1c76790463d4e9b49e79f5dd
Author: Benny Halevy <[email protected]> Date: Wed May 5 12:10:20 2010 +0300
pnfs: pass mntfh down the init_pnfs path
To allow layout driver to issue getdevicelist at mount time.
commit 5af291ca118e7229d63883a32031f24cd237856f
Author: Andy Adamson <[email protected]> Date: Fri Apr 30 14:27:12 2010 -0400
pnfs: ds_wsize and ds_rsize
Set read/write I/O sizes for data servers
commit 5bdeaa297a843419f5c16b923f5d6a3d6a5f34db
Author: Andy Adamson <[email protected]> Date: Fri Apr 30 14:27:09 2010 -0400
pnfs: GETDEVICELIST
The block driver uses GETDEVICELIST
commit 7c9c654fb1da1d6bf7247ff285c761daef003d6a
Author: Benny Halevy <[email protected]> Date: Mon Aug 30 12:34:48 2010 +0300
pnfs: get machine creds
for getdevicelist
commit dc36f96e54aed069e73fa47adba216613cad2bc8
Author: Benny Halevy <[email protected]> Date: Mon Jun 14 11:04:23 2010 -0400
pnfs: check for non-rpc layout drivers
commit 892b1015cf26b293b7a388200e2f79589b543597
Author: Benny Halevy <[email protected]> Date: Mon Jun 14 11:04:16 2010 -0400
pnfs: Introduce pnfs_call_done and io done callbacks
To be called from layout drivers on the io done path.
commit 8d930291704eb794892112f3d90d448438838533
Author: Benny Halevy <[email protected]> Date: Mon Jun 14 11:03:48 2010 -0400
pnfs: pnfs_error
Return errors on the pnfs path in pnfs_call_data.pnfs_error
commit 2707e163a9430e70da79291113cd83047d4f0cdb
Author: Andy Adamson <[email protected]> Date: Thu Jun 17 15:34:56 2010 -0400
pnfs: setup_layoutcommit
commit 75fbbc5a44bba27fce939ed852981777873fb9a5
Author: Andy Adamson <[email protected]> Date: Thu Jun 17 15:34:55 2010 -0400
pnfs: cleanup_layoutcommit
commit 184dd8974e49fe16264dceedbaebc431af44b3b8
Author: Andy Adamson <[email protected]> Date: Thu Jun 17 15:34:54 2010 -0400
pnfs: encode_layoutcommit
commit 9fe095707fa05c1b7253dbc4bff7a8718a3ea3ae
Author: Andy Adamson <[email protected]> Date: Wed Jun 23 15:21:16 2010 -0400
pnfs: encode_layoutreturn
commit 7c0448ceee9b3ecbe34c1cea5db9cf4b8bd3f96e
Author: Benny Halevy <[email protected]> Date: Mon Jul 12 18:51:36 2010 +0300
pnfs: Use byte-range layout segments
commit f888d677ffc5c44b13ad2c74e9714670cb584607
Author: Benny Halevy <[email protected]> Date: Mon Jul 12 18:50:50 2010 +0300
pnfs: readahead_range
commit c5120038ecb4050cf2e25b33e8782640efd26e57
Author: Andy Adamson <[email protected]> Date: Wed Jul 14 15:43:57 2010 -0400
pnfs: layoutret_on_setattr
commit 752c112ff1aa66c793736595987c185bf9810116
Author: Benny Halevy <[email protected]> Date: Wed Dec 15 16:17:31 2010 +0200
pnfs: layoutreturn
commit c6024e4cb3a70642f8baf09ceffe68eac84adc0e
Author: Andy Adamson <[email protected]> Date: Fri Aug 13 17:31:59 2010 -0400
pnfs: direct i/o
commit 57ec1d2986e55ae6ea63641efdb56804a02f7420
Author: Fred Isaman <[email protected]> Date: Wed Dec 15 14:04:46 2010 -0500
pnfs: layoutcommit
commit 815bb1f7c46c93771fcea62103c1d8562c3c3f10
Author: Andy Adamson <[email protected]> Date: Fri Aug 13 17:31:37 2010 -0400
pnfs: layoutcommit helper functions
commit b4555d970d24fda018b5d3348a5160230deb8306
Author: Fred Isaman <[email protected]> Date: Mon Dec 20 11:33:54 2010 -0500
pnfs: has_layout
commit c10e3ca28266863b11fa266c13de79f5f6e83a31
Author: Benny Halevy <[email protected]> Date: Wed Sep 29 02:33:41 2010 +0200
pnfs: file needs layout commit, server attributes may be stale
commit 19c1edfde37912ca6f98f8947cd73a2248da014e
Author: Marc Eshel <[email protected]> Date: Fri Aug 13 17:32:01 2010 -0400
pnfs: CB_NOTIFY_DEVICEID
Note: This functionlaity is incomplete as all layout segments
referring to
the 'to be removed device id' need to be reaped, and all in flight
I/O drained.