pnfs git tree status pnfs-all-2.6.35-rc2-2010-06-10

I've release the tree with Alexandros and Andy's latest pnfs-submit patches as well as some fixes for pnfs-obj from Boaz. The latter, and Andy's patches went also for the pnfs-all-2.6.34 branch, tagged as pnfs-all-2.6.34-2010-06-10

Cumulative patches can be generated from
git://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.35-rc2 pnfs-all-2.6.35-rc2-2010-06-10 git diff v2.6.34 pnfs-all-2.6.34-2010-06-10

Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree

Latest patches (since 2010-05-17):

pnfs-submit:
Alexandros Batsakis (7):
pnfs-submit: clean struct nfs_inode
pnfs-submit: remove lgetcount, lretcount
pnfs-submit: change stateid to be a union
pnfs-submit: request whole-file layouts only
pnfs-submit: change layout list to be similar to other state lists
pnfs-submit: forgetful client (layouts)
pnfs-submit: support for CB_RECALL_ANY (layouts)

Andy Adamson (5):
SQUASHME: pnfs-submit: replace layoutcommit_ctx with rpc_cred SQUASHME pnfs-submit: cleanup layoutcommit call SQUASHME pnfs-submit: handle async layoutcommit errors SQUASHME pnfs remove ifdef around layoutcommit_needed SQUASHME pnfs-submit: move layoutcommit to nfs_write_inode

Ricardo Labiaga (2):
SQUASHME: pnfs-submit: Use LAYOUT_NFSV4_1_FILES instead of LAYOUT_NFSV4_FILES
pnfs-submit: Dynamically load the nfslayoutdriver

Tao Guo (2):
SQUASHME: pnfs-submit: call layoutcommit after flushing inode's data to disk.
SQUASHME: pnfs: unlock lo_lock before calling layoutdriver's setup_layoutcommit

pnfs-block:
Zhang Jingwang (1):
SQAUSHME: blocklayoutdriver: NULL pointer reference when committing too many extents

pnfsd-files:
Benny Halevy (1):
SQUASHME: pnfsd: dlm: fixup LAYOUT_NFSV4_1_FILES

Eric Anderle (2):
pnfsd: make /proc/fs/nfsd/pnfs_dlm_device report dlm device list.
SQUASHME: pnfsd: fix test in nfsd4_find_pnfs_dlm_device

Ricardo Labiaga (1):
SQUASHME: pnfs-submit: Use LAYOUT_NFSV4_1_FILES instead of LAYOUT_NFSV4_FILES

pnfsd:
Benny Halevy (2):
SQUASHME: pnfsd: cb_{set,client} moved in 2.6.35
SQUASHME: pnfsd: cl_count removed in 2.6.35

J. Bruce Fields (1):
SQUASHME: nfsd4: fix cb_recall encoding

spnfs:
Benny Halevy (1):
SQUASHME: spnfs: fixup LAYOUT_NFSV4_1_FILES

spnfs-block:
pnfsd-lexp:
Benny Halevy (1):
SQUASHME: pnfsd-lexp: fixup LAYOUT_NFSV4_1_FILES

pnfs-obj-all:
Boaz Harrosh (2):
SQUASHME: pnfs-obj: panlayout: Fix very old BUG_ONs on ol_state.status
SQUASHME: panfs_shim: Prints on Errors

pnfs-block-all:
Zhang Jingwang (1):
SQAUSHME: blocklayoutdriver: NULL pointer reference when committing too many extents

spnfs-all:
Benny Halevy (1):
SQUASHME: spnfs: fixup LAYOUT_NFSV4_1_FILES

pnfs-all-latest:
Benny Halevy (1):
DEBUG: pnfs: turn BUG_ONs in pnfs_destroy_inode to WARN_ONs

pnfs-all-2.6.34:
Andy Adamson (5):
SQUASHME: pnfs-submit: replace layoutcommit_ctx with rpc_cred
pnfs: cleanup layoutcommit call
pnfs: handle async layoutcommit errors
pnfs: remove ifdef around layoutcommit_needed
pnfs: move layoutcommit to nfs_write_inode

Benny Halevy (4):
SQUASHME: pnfsd: dlm: fixup LAYOUT_NFSV4_1_FILES
SQUASHME: pnfsd-lexp: fixup LAYOUT_NFSV4_1_FILES
SQUASHME: spnfs: fixup LAYOUT_NFSV4_1_FILES
DEBUG: pnfs: turn BUG_ONs in pnfs_destroy_inode to WARN_ONs

Boaz Harrosh (2):
SQUASHME: pnfs-obj: panlayout: Fix very old BUG_ONs on ol_state.status
panfs_shim: Prints on Errors

Eric Anderle (2):
pnfsd: make /proc/fs/nfsd/pnfs_dlm_device report dlm device list.
SQUASHME: pnfsd: fix test in nfsd4_find_pnfs_dlm_device

J. Bruce Fields (1):
SQUASHME: nfsd4: fix cb_recall encoding

Ricardo Labiaga (3):
SQUASHME: pnfs-submit: Use LAYOUT_NFSV4_1_FILES instead of LAYOUT_NFSV4_FILES
SQUASHME: pnfs-submit: Use LAYOUT_NFSV4_1_FILES instead of LAYOUT_NFSV4_FILES
pnfs-submit: Dynamically load the nfslayoutdriver

Tao Guo (2):
SQUASHME: pnfs-submit: call layoutcommit after flushing inode's data to disk.
SQUASHME: pnfs: unlock lo_lock before calling layoutdriver's setup_layoutcommit

Zhang Jingwang (1):
SQAUSHME: blocklayoutdriver: NULL pointer reference when committing too many extents

I will not be testing osd/exofs on the pnfs-all-2.6.35-rc branch from Benny. I will be sticking with pnfs-all-2.6.34. This is because of Alexandros Patches.

I have gone half way through Fred's patches and I like them a lot so far, though they are still 50% raw. But it is all in the right direction. I would not mind working on a tree with these in and fix any issues that come up.

Alexandro's, patches I have not had the time to review yet. And am reluctant to do so since they are still broken by definition. The server is still not converted, and it takes two to tango. Unlike some of us, I'm still dependent on the Linux server and that one is broken for me, if using Alexandro's client.

I have been running some tests on the Latest pnfs-all-2.6.34 branch and seeing problems with Files-layout. Obj-layout is fine.

I have a simple setup of export on localhost in a single physical machine.
* The first is using the LOCAL-EXP of an ext3, rather empty partition. I'm running my infamous test of "git clone linux" At the files checkout stage i get like 10 of:
kernel: pnfs_destroy_inode: layout.refcount 1
and a BUG_ON at nfs/inode.c:1365
The machine becomes un stable after that. I suspect because of the BUG_ON killing the kswapd thread (see below the stack trace). I told Benny that making these a WAR_ON would be better since it is a leak not a CRASH going to happen so it will be easier to fix.
But there is some layout ref-count problem hiding somewhere in files-layout.

* Same exact test over an exofs export with obj-layout gives me clean responsive machine with clean dmesg file.

See you all in Bakeathon
Boaz

---

Jun 10 18:35:34 tl1 kernel: ------------[ cut here ]------------
Jun 10 18:35:34 tl1 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/inode.c:1365!
Jun 10 18:35:34 tl1 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Jun 10 18:35:34 tl1 kernel: last sysfs file: /sys/devices/platform/host5/iscsi_host/host5/initiatorname
Jun 10 18:35:34 tl1 kernel: CPU 1 
Jun 10 18:35:34 tl1 kernel: Modules linked in: nfslayoutdriver objlayoutdriver exofs nfsd exportfs nfs lockd nfs_acl auth_rpcgss osd libosd
 crc32c sunrpc ip6_tables ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table ext2 dm_mirror
 dm_region_hash dm_log dm_multipath dm_mod i915 snd_hda_codec_via snd_hda_intel drm_kms_helper snd_hda_codec snd_hwdep snd_seq_dummy snd_se
q_oss drm snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc i2c_algo_bit v
ideo atl1c output i2c_i801 i2c_core rng_core sg button ata_generic ata_piix libata sd_mod scsi_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_
hcd [last unloaded: microcode]
Jun 10 18:35:34 tl1 kernel:
Jun 10 18:35:34 tl1 kernel: Pid: 291, comm: kswapd0 Not tainted 2.6.34-pnfs #3 G41TM-P33 (MS-7592)/MS-7592
Jun 10 18:35:34 tl1 kernel: RIP: 0010:[<ffffffffa049b9a6>]  [<ffffffffa049b9a6>] nfs_destroy_inode+0x5e/0x99 [nfs]
Jun 10 18:35:34 tl1 kernel: RSP: 0000:ffff88007d0d5c20  EFLAGS: 00010202
Jun 10 18:35:34 tl1 kernel: RAX: 0000000000000029 RBX: ffff88004516abf0 RCX: 00000000000084c1
Jun 10 18:35:34 tl1 kernel: RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
Jun 10 18:35:34 tl1 kernel: RBP: ffff88007d0d5c40 R08: ffffffffa04c5362 R09: 000000000000000a
Jun 10 18:35:34 tl1 kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff88004516a9a0
Jun 10 18:35:34 tl1 kernel: R13: ffff88004516aba8 R14: ffff88007d0d5ca0 R15: 0000000000000080
Jun 10 18:35:34 tl1 kernel: FS:  0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 10 18:35:34 tl1 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 10 18:35:34 tl1 kernel: CR2: 00007fc6fe5b0ecf CR3: 0000000076aca000 CR4: 00000000000406e0
Jun 10 18:35:34 tl1 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 10 18:35:34 tl1 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 10 18:35:34 tl1 kernel: Process kswapd0 (pid: 291, threadinfo ffff88007d0d4000, task ffff88007d0aad80)
Jun 10 18:35:34 tl1 kernel: Stack:
Jun 10 18:35:34 tl1 kernel: ffff88007d0d5c40 ffff88004516abf0 ffff88004516ac00 000000000000004e
Jun 10 18:35:34 tl1 kernel: <0> ffff88007d0d5c60 ffffffff810fbdbf ffff88007d0d5c60 ffff88004516abf0
Jun 10 18:35:34 tl1 kernel: <0> ffff88007d0d5c90 ffffffff810fc23c ffff880077917680 00000000000000b4
Jun 10 18:35:34 tl1 kernel: Call Trace:
Jun 10 18:35:34 tl1 kernel: [<ffffffff810fbdbf>] destroy_inode+0x2f/0x45
Jun 10 18:35:34 tl1 kernel: [<ffffffff810fc23c>] dispose_list+0xb6/0xe4
Jun 10 18:35:34 tl1 kernel: [<ffffffff810fc412>] shrink_icache_memory+0x1a8/0x1d8
Jun 10 18:35:34 tl1 kernel: [<ffffffff810bc3b9>] shrink_slab+0xd8/0x15c
Jun 10 18:35:34 tl1 kernel: [<ffffffff810bca11>] balance_pgdat+0x358/0x5ae
Jun 10 18:35:34 tl1 kernel: [<ffffffff810ba4c7>] ? isolate_pages_global+0x0/0x1df
Jun 10 18:35:34 tl1 kernel: [<ffffffff81057589>] ? spin_unlock_irqrestore+0xe/0x10
Jun 10 18:35:34 tl1 kernel: [<ffffffff810bce20>] kswapd+0x1b9/0x1cf
Jun 10 18:35:34 tl1 kernel: [<ffffffff8105750f>] ? autoremove_wake_function+0x0/0x39
Jun 10 18:35:34 tl1 kernel: [<ffffffff8102edea>] ? spin_unlock_irqrestore+0xe/0x10
Jun 10 18:35:34 tl1 kernel: [<ffffffff810bcc67>] ? kswapd+0x0/0x1cf
Jun 10 18:35:34 tl1 kernel: [<ffffffff810570b9>] kthread+0x7f/0x87
Jun 10 18:35:34 tl1 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
Jun 10 18:35:34 tl1 kernel: [<ffffffff8105703a>] ? kthread+0x0/0x87
Jun 10 18:35:34 tl1 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
Jun 10 18:35:34 tl1 kernel: Code: 39 6b b8 74 04 0f 0b eb fe 8b 53 a8 85 d2 74 15 48 c7 c6 a0 97 4c a0 48 c7 c7 27 dc 4c a0 31 c0 e8 86 e7 
e6 e0 83 7b a8 00 74 04 <0f> 0b eb fe 49 8d 84 24 b0 01 00 00 48 39 83 60 ff ff ff 74 04 
Jun 10 18:35:34 tl1 kernel: RIP  [<ffffffffa049b9a6>] nfs_destroy_inode+0x5e/0x99 [nfs]
Jun 10 18:35:34 tl1 kernel: RSP <ffff88007d0d5c20>
Jun 10 18:35:34 tl1 kernel: ---[ end trace 1ad113810041ecf1 ]---
Jun 10 18:36:34 tl1 ntpd[2200]: synchronized to 192.168.0.140, stratum 3
Jun 10 18:36:34 tl1 ntpd[2200]: time reset +0.338350 s
Jun 10 18:36:34 tl1 ntpd[2200]: kernel time sync status change 0001
Jun 10 18:36:41 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:41:40 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:42:48 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:42:59 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:43:47 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:44:21 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:44:39 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:45:25 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 18:46:47 tl1 ntpd[2200]: synchronized to 192.168.0.140, stratum 3
Jun 10 19:01:52 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 19:01:52 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 19:01:53 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 19:01:53 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 19:01:54 tl1 kernel: pnfs_destroy_inode: layout.refcount 1
Jun 10 19:01:54 tl1 kernel: pnfs_destroy_inode: layout.refcount 1

I know of at least one problem...the server responding with a shortIO will completely mess up the files-layout client. (It ends up trying to resend the RPC twice.) I have some pnfs-submit patches queued up to fix that, but have been holding them until some of the backlog of patches clears.

Fred

Here's the list of patches that went into the latest linux-pnfs tree as of 2010-06-17 since last update (pnfs-all-2.6.35-rc2-2010-06-10):

Note that this include Trond's nfs41 patches that I pulled from his nfs-for-2.6.36 branch.

nfs-upstream:
Andy Adamson (1):
nfs4 use mandatory attribute file type in nfs4_get_root

Trond Myklebust (15):
NFSv41: Fix a memory leak in nfs41_proc_async_sequence() NFSv4.1: Clean up nfs4_setup_sequence
NFSv4.1: Simplify nfs41_sequence_done()
NFSv4: Kill nfs4_async_handle_error() abuses by NFSv4.1 NFSv4.1: Merge the nfs41_proc_async_sequence() and nfs4_proc_sequence() NFSv4.1: Make nfs4_setup_sequence take a nfs_server argument
NFSv41: Further cleanup for nfs4_sequence_done
NFSv41: Don't store session state in the nfs_client->cl_state
NFSv41: Clean up the NFSv4.1 minor version specific operations
NFSv41: Convert the various reboot recovery ops etc to minor version ops
NFSv41: Fix nfs_async_inode_return_delegation() ugliness
NFSv41: Deprecate nfs_client->cl_minorversion
NFSv41: Clean up exclusive create
NFSv41: Cleanup for nfs4_alloc_session.
NFSv4.1: There is no need to init the session more than once...

nfs41-all:
Benny Halevy (1):
SQUASHME: DEVONLY: nfs41: improve nfs41_sequence_done debug message on NFS4ERR_DELAY

Boaz Harrosh (1):
UMLFIX: Fc13 uml brokenness

John Fastabend (1):
net: fix deliver_no_wcard regression on loopback device

pnfs-submit:
Andy Adamson (5):
pnfs-submit: add data server session to nfs4_setup_sequence SQUASHME pnfs-submit: update to new nfs41_sequence_call_done
pnfs-submit: add ds session to nfs4_async_handle_error SQUASHME pnfs-submit: export nfs_write_prepare
pnfs: trim write count to wsize when retrying via MDS

Benny Halevy (19):
SQUASHME: pnfs-submit: destroy layout rather than return on state reclaim
SQUASHME: pnfs-submit: handle NULL in pnfs_destroy_layout
SQUASHME: pnfs-submit: delete unused nfs_validate_delegation_stateid
SQUASHME: pnfs-submit: merge pnfs4_proc_setattr into nfs4_proc_setattr
SQUASHME: pnfs4_read_done status variable is unused
SQUASHME: pnfs-submit: optionally pass nfs_client to nfs4_async_handle_error
SQUASHME: pnfs-submit: merge pnfs4_read_done into nfs4_read_done - part 2
SQUASHME: pnfs-submit: merge pnfs4_read_done into nfs4_read_done - part 3
SQUASHME: pnfs-submit: get rid of pnfs4_read_done
SQUASHME: pnfs-submit: merge pnfs4_write_done into nfs4_write_done - part 1
SQUASHME: pnfs-submit: merge pnfs4_write_done into nfs4_write_done - part 3
SQUASHME: pnfs-submit: merge pnfs4_write_done into nfs4_write_done - part 4
SQUASHME: pnfs-submit: get rid of pnfs4_write_done
SQUASHME: pnfs-submit: merge pnfs4_commit_done into nfs4_commit_done - part 2
SQUASHME: pnfs-submit: merge pnfs4_commit_done into nfs4_commit_done - part 3
SQUASHME: pnfs-submit: get rid of pnfs4_commit_done
SQUASHME: pnfs-submit: merge pnfs4_proc_write_setup into nfs4_proc_write_setup
SQUASHME: pnfs-submit: merge pnfs4_proc_commit_setup into nfs4_proc_commit_setup
SQUASHME: pnfs-submit: get rid of pnfs_v4_clientops

Fred Isaman (10):
pnfs-submit: Revert "pnfs: pnfs_redirty_request"
pnfs-submit: Reorder arguments to pnfs_update_layout
SQUASHME: pnfs-submit: avoid NULL dereference in filelayout_commit dprintk
pnfs-submit: filelayout: Stop using pnfs_call_done and friends
pnfs-submit: Remove checks for non-rpc drivers
pnfs-submit: Remove pnfs_call_done and friends
pnfs-submit: Remove pnfs_error
SQUASHME: pnfs-submit: test for sending LAYOUTRETURN was reversed
SQUASHME: pnfs-submit: Remove lo_lock and use i_lock instead
pnfs-submit: filelayout: Support dense layouts

pnfs:
Benny Halevy (3):
Revert "pnfs-submit: Remove pnfs_error" Revert "pnfs-submit: Remove pnfs_call_done and friends" Revert "pnfs-submit: Remove checks for non-rpc drivers"

pnfs-block:
Fred Isaman (8):
HACK: propagate fsdata into nfs_writepage_setup
SQUASHME: pnfs: export get_lseg and put_lseg
SQUASHME: pnfsblock: write_begin adjust for removed fields
SQUASHME: pnfsblock: write_end adjust for removed ok_to_use_pnfs
SQUASHME: pnfsblock: write_end_cleanup adjust for removed ok_to_use_pnfs
SQUASHME: pnfsblock: bl_write_pagelist support functions adjust for missing PG_USE_PNFS
SQUASHME: pnfsblock: bl_write_pagelist adjust for missing PG_USE_PNFS
pnfs: Revert "pnfs_post_submit: Restore "pnfs: pnfs_do_flush" part 2"

Zhang Jingwang (1):
SQAUSHME: blocklayoutdriver: NULL pointer reference when committing too many extents

pnfsd-files:
Andy Adamson (1):
SQUASHME: pnfsd: fix dlm device naming

Eric Anderle (1):
pnfsd: Add IP address validation to nfsd4_set_pnfs_dlm_device()

pnfsd:
Benny Halevy (1):

      SQUASHME: pnfsd: fix error handling for s_pnfs_op->get_device_info

Here's the list of patches that went into the latest linux-pnfs tree as of 2010-07-01 since last update (pnfs-all-2.6.35-rc2-2010-06-17)

As usual, cumulative patches against the mainline kernel can be generated via http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree

nfsd-upstream:
Boaz Harrosh b9598af SQUASHME: nfsd41: Fix a crash when a callback is retried

nfs-upstream:
Benny Halevy a1c6087 nfs: fix possible leak in nfs_proc_symlink
Trond Myklebust 1308c3f NFSv4: Ensure that /proc/self/mountinfo displays the minor version number

nfs41-all:
Benny Halevy 20265c2 i915_dma Bug #16247

pnfs-submit:

  Benny Halevy     f88044b SQUASHME: pnfs-submit: __wait no longer used by _pnfs_update_layout
  Benny Halevy     397af9d SQUASHME: pnfs-submit: fix possible leak in nfs4_pnfs_ds_create
  Benny Halevy     970b309 SAUASHME: pnfs-submit: prevent buffer overrun in decode_and_add_ds
  Andy Adamson     4ae3ac5 SQUASHME pnfs-submit: remove struct nfs_server from stack
  Andy Adamson     fcc6499 SQUASHME: pnfs-submit: don't re-initialize i_lock
  Andy Adamson     f83fa38 SQUASHME pnfs-submit: add error handlers to layout get
  Andy Adamson     14e9066 SQUASHME pnfs-submit: handle assassinated layoutcommit
  Andy Adamson     2a1ef93 SQUASHME pnfs-submit: add error handling to layout return
  Andy Adamson     cbae7a5 SQUASHME pnfs-submit: remove encode_layoutreturn
  Andy Adamson     761f452 SQUASHME pnfs-submit: pnfs_try_to_read, write, commit using freed memory
  Andy Adamson     56a98e8 SQUASHME: pnfs-submit remove unused filelayout_mount_type
  Fred Isaman      fcbaa5e pnfs-submit: remove list_empty check from put_layout

pnfs:
Benny Halevy 5157ed0 SQUASHME: pnfs: check for read_buf error in decode_pnfs_layoutrecall_args
Andy Adamson 5637fa1 pnfs-post-submit: restore encode_layoutreturn

pnfs-block:
Benny Halevy 5d459a3 SQUASHME: pnfs-block: nfs4_blk_add_block_disk ret must be signed

pnfsd-files:

  Benny Halevy     e840874 SQUASHME: pnfsd-files: nfsd4_pnfs_dlm_shutdown should use list_for_each_entry_safe
  Benny Halevy     a9fb462 SQUASHME: pnfsd-files: verify_stateid's return status is __be32
  Benny Halevy     3895976 SQUASHME: pnfsd-files: prevent NULL deref in nfsd4_pnfs_dlm_getdevinfo
  Eric Anderle     805dec2 pnfsd: Fix num_ds bug in nfsd4_set_pnfs_dlm_device()

pnfsd:

  Boaz Harrosh     259e31d SQUASHME: pnfsd: into [SQUASHME: pnfsd: Support for cb_layout returning NFS4ERR_DELAY]
  Benny Halevy     bf5656c SQUASHME: pnfsd: fix pointer dereference in dprintk of write verifiers
  Benny Halevy     263355c pnfsd: reset status on NFS4ERR_NOMATCHING_LAYOUT
  Benny Halevy     50debee SQUASHME: pnfsd: move kfree of the cb_device rpc_args to release time
  Benny Halevy     bc3b006 SQUASHME: pnfsd: move kfree of the cb_layout rpc_args to release time

spnfs-block:
Benny Halevy 8b047ca SQUASHME: spnfs-block: kfree(res) in bl_pipe_downcall error path

pnfs-obj-all:
Benny Halevy 8ef6e62 SQUASHME: pnfs-obj: use NFSDBG_PNFS_LD also in pnfs_osd_xdr_cli.c

pnfs-exofs-all:
Boaz Harrosh 7cc0c0e SQUASHME: pnfsd-exofs: Fix exofs_inode_recall_layout() sending 2 recalls

Benny you have missed a patch from Andy's patchset I can count 9 patches by Andy but the original set had 10.
The missing patch is titled:
[PATCH 03/10] SQUASHME pnfs-submit: clear page lseg on partial i/o

Your Friday bhalevy tree had it I think. And I have on queue a patchset that conflicts with it. (Which is why I waited for your release before posting). So what is the disposition of this patch, is it in or out? Andy?

I'll post a set based on [PATCH 03/10] applied.

Boaz

Correct. Andy requested me to pull it out as it needs more work.

Let's deal with them in FIFO order.
I hope the conflicts are not too hard to resolve so if you're ready with your patches first, let's go for them.

Benny

Here's the list of patches that went into the latest linux-pnfs tree as of 2010-07-20 since last update (pnfs-all-2.6.35-rc3-2010-07-01:

Note that this include more nfs41 patches that I pulled from Trond's nfs-for-2.6.36 branch.

nfs-upstream:

  Benny Halevy     f42838f nfs: fix possible leak in nfs_proc_symlink
  Trond Myklebust  5936a2a NFSv4.1: Handle NFS4ERR_DELAY on SEQUENCE correctly
  Trond Myklebust  8df9417 NFSv4: Clean up struct nfs4_state_owner
  Trond Myklebust  8a5390e NFSv4: Ensure that we track the NFSv4 lock state in read/write requests.
  Trond Myklebust  f9f7435 NFSv4: Clean up for lockowner XDR encoding
  Trond Myklebust  ab9cd73 NFSv4: Add support for the RELEASE_LOCKOWNER operation
  Trond Myklebust  dba6dd5 NFSv4: Ensure the lockowners are labelled using the fl_owner and/or fl_pid

pnfs-submit:

  Boaz Harrosh     ad68cbb SQUASHME: pnfs-submit: Collapse pnfs_try_to_write_data code path and avoid big fat inlines
  Boaz Harrosh     f6d92c5 SQUASHME: pnfs-submit: Collapse pnfs_try_to_read_data code path and avoid big fat inlines
  Boaz Harrosh     3f15df9 SQUASHME: pnfs-submit: Collapse pnfs_try_to_commit code path and avoid big fat inlines
  Boaz Harrosh     a50ccf1 SQUASHME: pnfs-submit: Move _pnfs_clear_lseg_from_pages to pnfs.c
  Boaz Harrosh     1d58066 SQUASHME: pnfs-commit: Fix scariness in nfs4_clear_inode
  Boaz Harrosh     56dc2c0 SQUASHME: pnfs-submit: pnfs_return_layout remove surrounding #ifdef CONFIG_NFS_V4_1
  Andy Adamson     a3ae6d3 SQUASHME pnfs-submit: remove pnfs_init_once
  Andy Adamson     5969feb SQUASHME pnfs-submit remove pnfs_alloc_init_inode
  Andy Adamson     1a4f2af SQUASHME pnfs-submit: remove pnfs_destroy_inode
  Andy Adamson     a4831d8 SQUASHME pnfs-submit: set layoutcommit status after async error handler
  Andy Adamson     d2e80be SQUASHME pnfs-submit: set layoutget status after async error handler
  Fred Isaman      f13fcd2 SQUASHME pnfs-submit: add state flag for layoutcommit_needed
  Fred Isaman      5709bf4 SQUASHME: pnfs-submit: move pnfs_layout_suspend back to nfs_inode
  Andy Adamson     86c584f SQUASHME pnfs-submit embed pnfs_layout_type
  Andy Adamson     9fcd46d SQUASHME pnfs-submit: filelayout: use new alloc/free_layout API
  Andy Adamson     f4c135f SQUASHME pnfs-submit: rewrite layout allocation
  Andy Adamson     8cad754 SQUASHME pnfs-submit; fix pnfs_update_layout reference counting
  Andy Adamson     ef1af9c SQUASHME pnfs_submit: don't get a reference on boundary calculation
  Andy Adamson     24e681c SQUASHME pnfs-submit: don't reference the layout in init_lseg
  Andy Adamson     fcc3bf2 SQUASHME pnfs-submit: pnfs_update_layout always references the lseg
  Andy Adamson     9b66bb1 SQUASHME pnfs-submit: reference the layout when inserted into segs list
  Andy Adamson     971db12 SQUASHME pnfs-submit: rename put_layout to put_layout_locked
  Andy Adamson     185e07d SQUASHME pnfs-submit: reference layout across layoutcommit
  Andy Adamson     c730ea5 SQUASHME pnfs-submit: reference layout for layoutreturn
  Andy Adamson     0175ef7 SQUASHME pnfs-submit: remove put_layout from pnfs_free_layout
  Andy Adamson     ce23bd5 SQUASHME pnfs-submit: do not reference a layout in destroy_layout.
  Andy Adamson     ce7647a SQUASHME pnfs-submit: remove grab_current_layout
  Fred Isaman      a3f2877 pnfs-submit: Rely on full-file layout assumption to clean out some code
  Fred Isaman      871cac0 pnfs-submit: Remove readahead_range
  Fred Isaman      ccf3a9d pnfs-submit: Remove offset and count arguments from pnfs_update_layout
  Benny Halevy     27c4fb5 SQUASHME: pnfs-submit: check has_layout in __nfs4_close
  Benny Halevy     823ef07 SQUASHME: pnfs-submit: clean up nfs_lock_alloc_layout
  Benny Halevy     8e5ddfc SQUASHME: DEBUG: pnfs-submit: define and use BUG_ON_UNLOCKED_INO
  Andy Adamson     411990e SQUASHME pnfs-submit zero layout stateid on last layout segment removal
  Andy Adamson     d768d15 SQUASHME pnfs-submit: pnfs_try_to_commit can release nfs_write_data
  Andy Adamson     bec6c17 SQUASHME pnfs-submit remove gather_across_stripes policy
  Andy Adamson     0a2d01e SQUASHME pnfs-submit remove layoutret_on_setattr policy
  Andy Adamson     fc9abd7 SQUASHME pnfs-submit fix CONFIG_NFS_V4_1 not set compile error
  Andy Adamson     f37d86f SQUASHME pnfs-submit consolidate read_pagelist parameters
  Andy Adamson     749156d SQUASHME pnfs-submit filelayout use new read_pagelist API
  Andy Adamson     5e8daaf SQUASHME pnfs-submit consolidate write_pagelist parameters
  Andy Adamson     f1c98a4 SQUASHME pnfs-submit filelayout use new write_pagelist API
  Andy Adamson     467d776 SQUASHME pnfs-submit consolidate commit parameters
  Andy Adamson     6fbc8c2 SQUASHME pnfs-submit filelayout_commit use new API
  Andy Adamson     d2d5f50 SQUASHME pnfs-submit remove unused pnfs_return_layout_done
  Andy Adamson     ba814bf SQUASHME pnfs-submit remove unused nfs4_filelayout fields
  Andy Adamson     6ef6c01 SQUASHME pnfs-submit nfs4filelayout.c cleanup
  Andy Adamson     168ad0f8 SQUASHME pnfs-submit nfs4_filelayoutdev.c cleanup
  Andy Adamson     65f10b2 SQUASHME pnfs-submit get layoutget stateid from layout in xdr layer
  Andy Adamson     6be5ffb SQUASHME pnfs-submit get layoutreturn stateid from layout in xdr layer
  Andy Adamson     0980514 SQUASHME pnfs-submit remove unnecessary layoutreturn field
  Andy Adamson     c1edb65 SQUASHME pnfs-submit remove unnecessary layoutget field
  Benny Halevy     d8c9740 SQUASHME: pnfs-submit: add nr_pages back to write_pagelist api
  Benny Halevy     829e768 SQUASHME: pnfs-submit: filelayout: add nr_pages back to write_pagelist api
  Benny Halevy     d5b7f90 SQUASHME: pnfs-submit: use nfs_page_array_len in pnfs_try_to_read_data

pnfs:

  Andy Adamson     807eb70 pnfs-post-submit: restore and fix layoutret_on_setattr
  Andy Adamson     9c85e97 pnfs post-submit: restore gather_across_stripes policy
  Benny Halevy     0a21f6e pnfs-post-submit: reintroduce offset and count arguments from pnfs_update_layout"
  Benny Halevy     5260f76 pnfs-post-submit: Reintroduce readahead_range
  Benny Halevy     01ed291 pnfs-post-submit Reintroduce layout segment byte-range semantics
  Benny Halevy     7d3f96a pnfs-post-submit: reintroduce optional lseg referencing

pnfs-obj-all:

  Benny Halevy     e33bfbf SQUASHME: pnfs-obj: use new alloc/free_layout API
  Benny Halevy     d5ba2e9 SQUASHME: pnfs-obj: use new commit api
  Benny Halevy     c7f6fa5 SQAUSHME: pnfs-obj: use new read_pagelist api
  Benny Halevy     36bb622 SQUASHME: pnfs-obj: use new write_pagelist api

pnfs-block-all:

  Benny Halevy     d299751 SQUASHME: pnfs-block: use new alloc/free_layout API
  Benny Halevy     e8a4b7e SQUASHME: pnfs-block: use new commit api
  Benny Halevy     2f1320f SQUASHME: pnfs-block: use new read_pagelist api
  Benny Halevy     7e540f7 SQUASHME: pnfs-block: use new write_pagelist api

pnfs-all-2.6.35-2010-08-05 is based over v2.6.35
I started with the pnfs-submit branch squashing but there's a lot of work to be done there to tidy it up and prepare a series for review.
Andy has graciously offered his help with this task. Thanks!

At this point we'll hold off with new patches for submission until we're done with squashing the series and then every new SQUASHME patch sent must indicate which patch(es) it needs to be squashed into so not to create a huge back log like we have today.

Here's the list of patches that went into the latest linux-pnfs tree as of 2010-08-05 since last update pnfs-all-2.6.35-rc5-2010-07-20:

pnfs-submit:

  Andy Adamson     83c4c44 SQUASHME pnfs-submit remove pnfs4_layout_reclaim
  Andy Adamson     67282c6 SQUASHME pnfs-submit destroy all layouts upon lease expiration
  Ricardo Labiaga  f20c76e SQUASHME: pnfs-submit: Use generic layout name to load layout modules
  Benny Halevy     fa73e39 SQUASHME: pnfs-submit: reduce nesting level in set_pnfs_layoutdriver

pnfs-block:
Jim Rees b51aad4 pnfs-block: Add support for simple rpc pipefs
Jim Rees 123741a pnfs-block: Remove device creation from kernel

http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree is updated with the latest information.

Note that Ricardo's "Use generic layout name to load layout modules" patch renamed the nfslayoutdriver module to "nfs_layout_nfsv41_files" so it can be loaded automagically by the kernel. The blocks and objects modules should be renamed along the same lines (TODO).

Benny

Hit Send too soon.
Subject is correct now.

For the record, here's the list of patches that went in at pnfs-all-2.6.35-2010-08-24 since last 2.6.35 update (pnfs-all-2.6.35-2010-08-05)

nfsd-upstream:
Andy Adamson e205af4 nfsd: remove duplicate NFS4_STATEID_SIZE declaration

nfs-upstream:

  Fred Isaman      f130c18 nfs41: prevent exchange_id from sending server-only flag
  Benny Halevy     7c7966b SUNRPC: define xdr_decode_opaque_fixed
  Benny Halevy     05b7977 SUNRPC: don't shorten buflen twice in xdr_shrink_pagelen
  Benny Halevy     6a8f625 SUNRPC: clean up xdr_shrink_pagelen use of temporary pointer
  Benny Halevy     eaed6b0 SUNRPC: don't use the copy variable in nested block
  Benny Halevy     e206d65 SUNRPC: simplify xdr_shrink_pagelen use of "copy"
  Trond Myklebust  6e50a73 SUNRPC: Don't truncate tail data unnecessarily in xdr_shrink_pagelen

pnfs-submit: (revised patchset w/ updated sign-offs and naming convention)

  The pNFS Team    6b6970c pnfs_submit: pnfs and nfslayoutdriver kconfig
  The pNFS Team    5a723d9 pnfs_submit: introduce include/linux/nfs4_pnfs.h
  The pNFS Team    db1a56c pnfs_submit: introduce fs/nfs/pnfs.h
  The pNFS Team    dd0aa15 pnfs_submit: introduce fs/nfs/pnfs.c
  The pNFS Team    91162b2 pnfs_submit: register unregister pnfs module
  The pNFS Team    7e9c5a8 pnfs_submit: set and unset pnfs layoutdriver modules
  The pNFS Team    5b3b2ad pnfs_submit: generic pnfs deviceid cache
  The pNFS Team    0af7dd6 pnfs_submit: introduce nfs4layoutdriver module
  The pNFS Team    8ee80ad pnfs_submit: filelayout data server cache
  The pNFS Team    2d9fd56 pnfs_submit: filelayout deviceid cache
  The pNFS Team    70a6cdc pnfs_submit: generic getdeviceinfo
  The pNFS Team    8f3251d pnfs_submit: filelayout getdeviceinfo
  The pNFS Team    6782a46 pnfs-submit: change stateid to be a union
  The pNFS Team    7c22f02 pnfs_submit: layout header alloc,reference, and destroy
  The pNFS Team    091b28b pnfs_submit: filelayout alloc_layout and free_layout
  The pNFS Team    d9f2f32 pnfs_submit: layout segment alloc, reference, destroy
  The pNFS Team    3a64b06 pnfs_submit: layoutget
  The pNFS Team    1a33c42 pnfs_submit: layout helper functions
  The pNFS Team    01919e1 pnfs_submit: filelayout layout segment alloc and free
  The pNFS Team    aa87591 pnfs_submit: layoutcommit helper functions
  The pNFS Team    1335f72 pnfs_submit: layoutcommit
  The pNFS Team    df68077 pnfs_submit: layoutreturn helper functions
  The pNFS Team    6a50166 pnfs_submit: layoutreturn
  The pNFS Team    0b7ce58 pnfs_submit: add data server session to nfs4_setup_sequence
  The pNFS Team    a6fda6e pnfs_submit: update nfs4_async_handle_error for data server
  The pNFS Team    5c3a44e pnfs_submit: update state renewal for data servers
  Andy Adamson     6793140 pnfs_submit-pageio-helpers.patch
  Andy Adamson     d4eed6c pnfs_submit: associate layout segmennt with nfs_page
  The pNFS Team    2e0f058 pnfs_submit: filelayout policy operations
  The pNFS Team    409b04d pnfs_submit: filelayout i/o helpers
  The pNFS Team    b09d538 pnfs_submit: generic read
  The pNFS Team    9409d81 pnfs_submit: filelayout read
  The pNFS Team    e1df6dd pnfs_submit: generic write
  The pNFS Team    2ff0463 pnfs_submit: data server write with no getattr
  The pNFS Team    b9bd85c pnfs_submit: filelayout write
  The pNFS Team    05a2257 pnfs_submit: signal layoutdriver commit
  The pNFS Team    77c846f pnfs_submit: generic commit
  The pNFS Team    b8c5f02 pnfs_submit: data server commit with no getattr
  The pNFS Team    76a69f8 pnfs_submit: filelayout commit
  The pNFS Team    1a267aa pnfs_submit: cb_layoutrecall
  The pNFS Team    f5f4f98 pnfs_submit: increase NFS_MAX_FILE_IO_SIZE
  Benny Halevy     6606305 SQUASHME: pnfs-submit: rename layoutrecall related types

pnfs:

  Andy Adamson     73a08d2 SQUASHME pnfs_post_submit: direct i/o
  Andy Adamson     5cfa46f SQUASHME pnfs_post_submit: cb notify deviceid declarations
  Andy Adamson     aac7937 SQUASHME pnfs_submit: remove this unused code?

pnfs-obj:
Benny Halevy ecdcfc0 SQUASHME: pnfs-obj: apply types rename

pnfs-block:
Benny Halevy d50bd6d SQUASHME: pnfs-block: apply types rename

As mentioned in a different message this concludes pnfs-all-2.6.35 and it is now opened only for fixes.

The development branches and pnfs-all-latest in particular are now rebased onto v2.6.36-rc3.

Benny