Skip to contentSkip to conversation

a simple and scalable pNFS block layout server V2

PATCH In review Christoph Hellwig 10 participants 63 messages Jan 22, 2015 – Feb 15, 2015 21 messages not in the archive

Subsystems (8)
lore
Share & tools
lore

Christoph Hellwig

author reviewer
#0 lore raw

This series adds support for the pNFS operations in NFS v4.1, as well as a block layout driver that can export block based filesystems that implement a few additional export operations. Support for XFS is provided in this series, but other filesystems could be added easily.

The core pNFS code of course owns its heritage to the existing Linux pNFS server prototype, but except for a few bits and pieces in the XDR path nothing is left from it.

The design of this new pNFS server is fairly different from the old one - while the old one implemented very little semantics in nfsd and left almost everything to filesystems my implementation implements as much as possible in common nfsd code, then dispatches to a layout driver that still is part of nfsd and only then calls into the filesystem, thus keeping it free from intimate pNFS knowledge.

This version of the code has been rebased to the locks-for-3.20 tree which adds a new lock context structure to the inode. For now we still (ab)use the lease list like in the last version. Adding a pNFS-specific list would duplicate a lot of code without much benefit. But during this research I came up with way to associate a nfs4_file with a struct file at open time which should allow to greatly simplify the pNFS and delegation code. So stay tuned for some patches in this area!

For now this also doesn't take errata 3901 for rfc 5661 into account yet and sticks to the verified errata. The changes in 3901 seem useful in the longer run one verified and will be implemented eventually. Note that the only existing pNFS block client (Linux) would not benefit from the longer layout lifetimes anyway.

More details are document in the individual patch descriptions and code comments.

This code is also available from:

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-2

Changes since V1:
- rebased to the locks-for-3.20 tree
- fixed a memory leak in the nfsd4_encode_getdeviceinfo error path
- removed the always one lg_roc field
- added a nopnfs export option
- use a synchronous transaction in layoutget to simplify recovery
- various XFS fixes pointed out by Dave
- various documentation typo fixes

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-1-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#2 lore raw

PATCH 1/20 nfs: add LAYOUT_TYPE_MAX enum value

This gives us a nice upper bound for later use in nfѕd.

---
 include/linux/nfs4.h | 1 +
 1 file changed, 1 insertion(+)
include/linux/nfs4.h +1 -0
index 022b761..8a3589c 100644
@@ -516,6 +516,7 @@ enum pnfs_layouttype {
 	LAYOUT_NFSV4_1_FILES  = 1,
 	LAYOUT_OSD2_OBJECTS = 2,
 	LAYOUT_BLOCK_VOLUME = 3,
+	LAYOUT_TYPE_MAX
 };
 
 /* used for both layout return and recall */
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#3 lore raw

PATCH 2/20 fs: track fl_owner for leases

In reply to a message not in the archive

Just like for other lock types we should allow different owners to have a read lease on a file. Currently this can't happen, but with the addition of pNFS layout leases we'll need this feature.

---
 fs/locks.c          | 12 +++++++-----
 fs/nfsd/nfs4state.c |  2 +-
 2 files changed, 8 insertions(+), 6 deletions(-)
fs/locks.c +7 -5
index 2fc36b3..65350a23 100644
@@ -1661,7 +1661,8 @@ generic_add_lease(struct file *filp, long arg, struct file_lock **flp, void **pr
 	 */
 	error = -EAGAIN;
 	list_for_each_entry(fl, &ctx->flc_lease, fl_list) {
-		if (fl->fl_file == filp) {
+		if (fl->fl_file == filp &&
+		    fl->fl_owner == lease->fl_owner) {
 			my_fl = fl;
 			continue;
 		}
@@ -1721,7 +1722,7 @@ out:
 	return error;
 }
 
-static int generic_delete_lease(struct file *filp)
+static int generic_delete_lease(struct file *filp, void *owner)
 {
 	int error = -EAGAIN;
 	struct file_lock *fl, *victim = NULL;
@@ -1737,7 +1738,8 @@ static int generic_delete_lease(struct file *filp)
 
 	spin_lock(&ctx->flc_lock);
 	list_for_each_entry(fl, &ctx->flc_lease, fl_list) {
-		if (fl->fl_file == filp) {
+		if (fl->fl_file == filp &&
+		    fl->fl_owner == owner) {
 			victim = fl;
 			break;
 		}
@@ -1778,7 +1780,7 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp,
 
 	switch (arg) {
 	case F_UNLCK:
-		return generic_delete_lease(filp);
+		return generic_delete_lease(filp, *priv);
 	case F_RDLCK:
 	case F_WRLCK:
 		if (!(*flp)->fl_lmops->lm_break) {
@@ -1857,7 +1859,7 @@ static int do_fcntl_add_lease(unsigned int fd, struct file *filp, long arg)
 int fcntl_setlease(unsigned int fd, struct file *filp, long arg)
 {
 	if (arg == F_UNLCK)
-		return vfs_setlease(filp, F_UNLCK, NULL, NULL);
+		return vfs_setlease(filp, F_UNLCK, NULL, (void **)&filp);
 	return do_fcntl_add_lease(fd, filp, arg);
 }
 
fs/nfsd/nfs4state.c +1 -1
index 532a60c..30ff0d4 100644
@@ -693,7 +693,7 @@ static void nfs4_put_deleg_lease(struct nfs4_file *fp)
 	spin_unlock(&fp->fi_lock);
 
 	if (filp) {
-		vfs_setlease(filp, F_UNLCK, NULL, NULL);
+		vfs_setlease(filp, F_UNLCK, NULL, (void **)&fp);
 		fput(filp);
 	}
 }
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw

PATCH 3/20 fs: add FL_LAYOUT lease type

In reply to a message not in the archive

This (ab-)uses the file locking code to allow filesystems to recall outstanding pNFS layouts on a file. This new lease type is similar but not quite the same as FL_DELEG. A FL_LAYOUT lease can always be granted, an a per-filesystem lock (XFS iolock for the initial implementation) ensures not FL_LAYOUT leases granted when we would need to recall them.

Also included are changes that allow multiple outstanding read leases of different types on the same file as long as they have a differnt owner. This wasn't a problem until now as nfsd never set FL_LEASE leases, and no one else used FL_DELEG leases, but given that nfsd will also issues FL_LAYOUT leases we will have to handle it now.

---
 fs/locks.c         | 14 ++++++++++----
 include/linux/fs.h | 16 ++++++++++++++++
 2 files changed, 26 insertions(+), 4 deletions(-)
fs/locks.c +10 -4
index 65350a23..6b9772d1 100644
@@ -137,7 +137,7 @@
 
 #define IS_POSIX(fl)	(fl->fl_flags & FL_POSIX)
 #define IS_FLOCK(fl)	(fl->fl_flags & FL_FLOCK)
-#define IS_LEASE(fl)	(fl->fl_flags & (FL_LEASE|FL_DELEG))
+#define IS_LEASE(fl)	(fl->fl_flags & (FL_LEASE|FL_DELEG|FL_LAYOUT))
 #define IS_OFDLCK(fl)	(fl->fl_flags & FL_OFDLCK)
 
 static bool lease_breaking(struct file_lock *fl)
@@ -1371,6 +1371,8 @@ static void time_out_leases(struct inode *inode, struct list_head *dispose)
 
 static bool leases_conflict(struct file_lock *lease, struct file_lock *breaker)
 {
+	if ((breaker->fl_flags & FL_LAYOUT) != (lease->fl_flags & FL_LAYOUT))
+		return false;
 	if ((breaker->fl_flags & FL_DELEG) && (lease->fl_flags & FL_LEASE))
 		return false;
 	return locks_conflict(breaker, lease);
@@ -1594,11 +1596,14 @@ int fcntl_getlease(struct file *filp)
  * conflict with the lease we're trying to set.
  */
 static int
-check_conflicting_open(const struct dentry *dentry, const long arg)
+check_conflicting_open(const struct dentry *dentry, const long arg, int flags)
 {
 	int ret = 0;
 	struct inode *inode = dentry->d_inode;
 
+	if (flags & FL_LAYOUT)
+		return 0;
+
 	if ((arg == F_RDLCK) && (atomic_read(&inode->i_writecount) > 0))
 		return -EAGAIN;
 
@@ -1647,7 +1652,7 @@ generic_add_lease(struct file *filp, long arg, struct file_lock **flp, void **pr
 
 	spin_lock(&ctx->flc_lock);
 	time_out_leases(inode, &dispose);
-	error = check_conflicting_open(dentry, arg);
+	error = check_conflicting_open(dentry, arg, lease->fl_flags);
 	if (error)
 		goto out;
 
@@ -1703,7 +1708,7 @@ generic_add_lease(struct file *filp, long arg, struct file_lock **flp, void **pr
 	 * precedes these checks.
 	 */
 	smp_mb();
-	error = check_conflicting_open(dentry, arg);
+	error = check_conflicting_open(dentry, arg, lease->fl_flags);
 	if (error) {
 		locks_unlink_lock_ctx(lease, &ctx->flc_lease_cnt);
 		goto out;
@@ -1787,6 +1792,7 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp,
 			WARN_ON_ONCE(1);
 			return -ENOLCK;
 		}
+
 		return generic_add_lease(filp, arg, flp, priv);
 	default:
 		return -EINVAL;
include/linux/fs.h +16 -0
index f87cb2f..d5658f4 100644
@@ -875,6 +875,7 @@ static inline struct file *get_file(struct file *f)
 #define FL_DOWNGRADE_PENDING	256 /* Lease is being downgraded */
 #define FL_UNLOCK_PENDING	512 /* Lease is being broken */
 #define FL_OFDLCK	1024	/* lock is "owned" by struct file */
+#define FL_LAYOUT	2048	/* outstanding pNFS layout */
 
 /*
  * Special return value from posix_lock_file() and vfs_lock_file() for
@@ -2036,6 +2037,16 @@ static inline int break_deleg_wait(struct inode **delegated_inode)
 	return ret;
 }
 
+static inline int break_layout(struct inode *inode, bool wait)
+{
+	smp_mb();
+	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+		return __break_lease(inode,
+				wait ? O_WRONLY : O_WRONLY | O_NONBLOCK,
+				FL_LAYOUT);
+	return 0;
+}
+
 #else /* !CONFIG_FILE_LOCKING */
 static inline int locks_mandatory_locked(struct file *file)
 {
@@ -2091,6 +2102,11 @@ static inline int break_deleg_wait(struct inode **delegated_inode)
 	return 0;
 }
 
+static inline int break_layout(struct inode *inode, bool wait)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FILE_LOCKING */
 
 /* fs/open.c */
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Acked-by on v1 3/20
#5 lore raw

Re: [PATCH 03/20] fs: add FL_LAYOUT lease type

Quoted 12 lines · On Thu, 22 Jan 2015 12:09:49 +0100 Christoph Hellwig <[email protected]> wrote:

This (ab-)uses the file locking code to allow filesystems to recall outstanding pNFS layouts on a file. This new lease type is similar but not quite the same as FL_DELEG. A FL_LAYOUT lease can always be granted, an a per-filesystem lock (XFS iolock for the initial implementation) ensures not FL_LAYOUT leases granted when we would need to recall them.

Also included are changes that allow multiple outstanding read leases of different types on the same file as long as they have a differnt owner. This wasn't a problem until now as nfsd never set FL_LEASE leases, and no one else used FL_DELEG leases, but given that nfsd will also issues FL_LAYOUT leases we will have to handle it now.

Quoted 6 lines
---
 fs/locks.c         | 14 ++++++++++----
 include/linux/fs.h | 16 ++++++++++++++++
 2 files changed, 26 insertions(+), 4 deletions(-)

Abuse indeed. :)

I'd probably prefer to move this to a separate list within the i_flctx instead of overloading the lease stuff, but I'm not religious about it.

I can live with this for now, and if and when we get around to representing different types of file locks with different types of structures, we can do that conversion then.

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-4-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#7 lore raw

Re: [PATCH 03/20] fs: add FL_LAYOUT lease type

Quoted 5 lines · Christoph Hellwig

This (ab-)uses the file locking code to allow filesystems to recall outstanding pNFS layouts on a file. This new lease type is similar but not quite the same as FL_DELEG. A FL_LAYOUT lease can always be granted, an a per-filesystem lock (XFS iolock for the initial implementation) ensures not FL_LAYOUT leases granted when we would need to recall them.

So when there's a conflicting operation it's xfs's responsibility to call break_layout and wait for the recall?

(And what roughly is the set of conflicting operations?)

--b.

Quoted 120 lines

Also included are changes that allow multiple outstanding read leases of different types on the same file as long as they have a differnt owner. This wasn't a problem until now as nfsd never set FL_LEASE leases, and no one else used FL_DELEG leases, but given that nfsd will also issues FL_LAYOUT leases we will have to handle it now.

---
 fs/locks.c         | 14 ++++++++++----
 include/linux/fs.h | 16 ++++++++++++++++
 2 files changed, 26 insertions(+), 4 deletions(-)
fs/locks.c +10 -4
index 65350a23..6b9772d1 100644
@@ -137,7 +137,7 @@
 
 #define IS_POSIX(fl)	(fl->fl_flags & FL_POSIX)
 #define IS_FLOCK(fl)	(fl->fl_flags & FL_FLOCK)
-#define IS_LEASE(fl)	(fl->fl_flags & (FL_LEASE|FL_DELEG))
+#define IS_LEASE(fl)	(fl->fl_flags & (FL_LEASE|FL_DELEG|FL_LAYOUT))
 #define IS_OFDLCK(fl)	(fl->fl_flags & FL_OFDLCK)
 
 static bool lease_breaking(struct file_lock *fl)
@@ -1371,6 +1371,8 @@ static void time_out_leases(struct inode *inode, struct list_head *dispose)
 
 static bool leases_conflict(struct file_lock *lease, struct file_lock *breaker)
 {
+	if ((breaker->fl_flags & FL_LAYOUT) != (lease->fl_flags & FL_LAYOUT))
+		return false;
 	if ((breaker->fl_flags & FL_DELEG) && (lease->fl_flags & FL_LEASE))
 		return false;
 	return locks_conflict(breaker, lease);
@@ -1594,11 +1596,14 @@ int fcntl_getlease(struct file *filp)
  * conflict with the lease we're trying to set.
  */
 static int
-check_conflicting_open(const struct dentry *dentry, const long arg)
+check_conflicting_open(const struct dentry *dentry, const long arg, int flags)
 {
 	int ret = 0;
 	struct inode *inode = dentry->d_inode;
 
+	if (flags & FL_LAYOUT)
+		return 0;
+
 	if ((arg == F_RDLCK) && (atomic_read(&inode->i_writecount) > 0))
 		return -EAGAIN;
 
@@ -1647,7 +1652,7 @@ generic_add_lease(struct file *filp, long arg, struct file_lock **flp, void **pr
 
 	spin_lock(&ctx->flc_lock);
 	time_out_leases(inode, &dispose);
-	error = check_conflicting_open(dentry, arg);
+	error = check_conflicting_open(dentry, arg, lease->fl_flags);
 	if (error)
 		goto out;
 
@@ -1703,7 +1708,7 @@ generic_add_lease(struct file *filp, long arg, struct file_lock **flp, void **pr
 	 * precedes these checks.
 	 */
 	smp_mb();
-	error = check_conflicting_open(dentry, arg);
+	error = check_conflicting_open(dentry, arg, lease->fl_flags);
 	if (error) {
 		locks_unlink_lock_ctx(lease, &ctx->flc_lease_cnt);
 		goto out;
@@ -1787,6 +1792,7 @@ int generic_setlease(struct file *filp, long arg, struct file_lock **flp,
 			WARN_ON_ONCE(1);
 			return -ENOLCK;
 		}
+
 		return generic_add_lease(filp, arg, flp, priv);
 	default:
 		return -EINVAL;
include/linux/fs.h +16 -0
index f87cb2f..d5658f4 100644
@@ -875,6 +875,7 @@ static inline struct file *get_file(struct file *f)
 #define FL_DOWNGRADE_PENDING	256 /* Lease is being downgraded */
 #define FL_UNLOCK_PENDING	512 /* Lease is being broken */
 #define FL_OFDLCK	1024	/* lock is "owned" by struct file */
+#define FL_LAYOUT	2048	/* outstanding pNFS layout */
 
 /*
  * Special return value from posix_lock_file() and vfs_lock_file() for
@@ -2036,6 +2037,16 @@ static inline int break_deleg_wait(struct inode **delegated_inode)
 	return ret;
 }
 
+static inline int break_layout(struct inode *inode, bool wait)
+{
+	smp_mb();
+	if (inode->i_flctx && !list_empty_careful(&inode->i_flctx->flc_lease))
+		return __break_lease(inode,
+				wait ? O_WRONLY : O_WRONLY | O_NONBLOCK,
+				FL_LAYOUT);
+	return 0;
+}
+
 #else /* !CONFIG_FILE_LOCKING */
 static inline int locks_mandatory_locked(struct file *file)
 {
@@ -2091,6 +2102,11 @@ static inline int break_deleg_wait(struct inode **delegated_inode)
 	return 0;
 }
 
+static inline int break_layout(struct inode *inode, bool wait)
+{
+	return 0;
+}
+
 #endif /* CONFIG_FILE_LOCKING */
 
 /* fs/open.c */
Signature · 1 line
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw

Re: [PATCH 03/20] fs: add FL_LAYOUT lease type

Quoted 11 lines · J. Bruce Fields
Quoted 5 lines · Christoph Hellwig

This (ab-)uses the file locking code to allow filesystems to recall outstanding pNFS layouts on a file. This new lease type is similar but not quite the same as FL_DELEG. A FL_LAYOUT lease can always be granted, an a per-filesystem lock (XFS iolock for the initial implementation) ensures not FL_LAYOUT leases granted when we would need to recall them.

So when there's a conflicting operation it's xfs's responsibility to call break_layout and wait for the recall?

(And what roughly is the set of conflicting operations?)

The last patch in the series has a comment explaining this.

There's two categories:

a) operations that need to be protected to maintain filesystem integrity:
truncate, hole punch, collapse range, a well as any new operation that can deallocate blocks.
b) operations that write data locally, and we want to provide a best effort attempt at data cohrency between local use and pFNS clients. This covers writes in all variants, and should apply to shared mmap writes, but sleeping in the page fault handler is too hard to bother with for this sort of best effort coherency.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#10 lore raw

PATCH 6/20 nfsd: add fh_fsid_match helper

In reply to a message not in the archive

Add a helper to check that the fsid parts of two file handles match.

---
 fs/nfsd/nfsfh.h | 9 +++++++++
 1 file changed, 9 insertions(+)
fs/nfsd/nfsfh.h +9 -0
index e24d954..84cae20 100644
@@ -196,6 +196,15 @@ static inline bool fh_match(struct knfsd_fh *fh1, struct knfsd_fh *fh2)
 	return true;
 }
 
+static inline bool fh_fsid_match(struct knfsd_fh *fh1, struct knfsd_fh *fh2)
+{
+	if (fh1->fh_fsid_type != fh2->fh_fsid_type)
+		return false;
+	if (memcmp(fh1->fh_fsid, fh2->fh_fsid, key_len(fh1->fh_fsid_type) != 0))
+		return false;
+	return true;
+}
+
 #ifdef CONFIG_NFSD_V3
 /*
  * The wcc data stored in current_fh should be cleared
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#11 lore raw

PATCH 7/20 nfsd: make lookup/alloc/unhash_stid available outside nfs4state.c

In reply to a message not in the archive
---
 fs/nfsd/nfs4state.c | 8 ++++----
 fs/nfsd/state.h     | 6 ++++++
 2 files changed, 10 insertions(+), 4 deletions(-)
fs/nfsd/nfs4state.c +4 -4
index b4f86f8..7515207 100644
@@ -486,7 +486,7 @@ static void nfs4_file_put_access(struct nfs4_file *fp, u32 access)
 		__nfs4_file_put_access(fp, O_RDONLY);
 }
 
-static struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl,
+struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl,
 					 struct kmem_cache *slab)
 {
 	struct nfs4_stid *stid;
@@ -690,7 +690,7 @@ static void nfs4_put_deleg_lease(struct nfs4_file *fp)
 	}
 }
 
-static void unhash_stid(struct nfs4_stid *s)
+void nfs4_unhash_stid(struct nfs4_stid *s)
 {
 	s->sc_type = 0;
 }
@@ -998,7 +998,7 @@ static void unhash_lock_stateid(struct nfs4_ol_stateid *stp)
 
 	list_del_init(&stp->st_locks);
 	unhash_ol_stateid(stp);
-	unhash_stid(&stp->st_stid);
+	nfs4_unhash_stid(&stp->st_stid);
 }
 
 static void release_lock_stateid(struct nfs4_ol_stateid *stp)
@@ -4438,7 +4438,7 @@ out_unlock:
 	return status;
 }
 
-static __be32
+__be32
 nfsd4_lookup_stateid(struct nfsd4_compound_state *cstate,
 		     stateid_t *stateid, unsigned char typemask,
 		     struct nfs4_stid **s, struct nfsd_net *nn)
fs/nfsd/state.h +6 -0
index 9d3be37..534983a 100644
@@ -545,6 +545,12 @@ struct nfsd_net;
 extern __be32 nfs4_preprocess_stateid_op(struct net *net,
 		struct nfsd4_compound_state *cstate,
 		stateid_t *stateid, int flags, struct file **filp);
+__be32 nfsd4_lookup_stateid(struct nfsd4_compound_state *cstate,
+		     stateid_t *stateid, unsigned char typemask,
+		     struct nfs4_stid **s, struct nfsd_net *nn);
+struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl,
+		struct kmem_cache *slab);
+void nfs4_unhash_stid(struct nfs4_stid *s);
 void nfs4_put_stid(struct nfs4_stid *s);
 void nfs4_remove_reclaim_record(struct nfs4_client_reclaim *, struct nfsd_net *);
 extern void nfs4_release_reclaim(struct nfsd_net *);
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#12 lore raw

PATCH 8/20 nfsd: make find/get/put file available outside nfs4state.c

In reply to a message not in the archive
---
 fs/nfsd/nfs4state.c | 10 ++--------
 fs/nfsd/state.h     |  7 +++++++
 2 files changed, 9 insertions(+), 8 deletions(-)
fs/nfsd/nfs4state.c +2 -8
index 7515207..c31b8f8 100644
@@ -282,7 +282,7 @@ static void nfsd4_free_file_rcu(struct rcu_head *rcu)
 	kmem_cache_free(file_slab, fp);
 }
 
-static inline void
+void
 put_nfs4_file(struct nfs4_file *fi)
 {
 	might_lock(&state_lock);
@@ -295,12 +295,6 @@ put_nfs4_file(struct nfs4_file *fi)
 	}
 }
 
-static inline void
-get_nfs4_file(struct nfs4_file *fi)
-{
-	atomic_inc(&fi->fi_ref);
-}
-
 static struct file *
 __nfs4_get_fd(struct nfs4_file *f, int oflag)
 {
@@ -3300,7 +3294,7 @@ find_file_locked(struct knfsd_fh *fh, unsigned int hashval)
 	return NULL;
 }
 
-static struct nfs4_file *
+struct nfs4_file *
 find_file(struct knfsd_fh *fh)
 {
 	struct nfs4_file *fp;
fs/nfsd/state.h +7 -0
index 534983a..cd61e7d 100644
@@ -573,6 +573,13 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 							struct nfsd_net *nn);
 extern bool nfs4_has_reclaimed_state(const char *name, struct nfsd_net *nn);
 
+struct nfs4_file *find_file(struct knfsd_fh *fh);
+void put_nfs4_file(struct nfs4_file *fi);
+static inline void get_nfs4_file(struct nfs4_file *fi)
+{
+	atomic_inc(&fi->fi_ref);
+}
+
 /* grace period management */
 void nfsd4_end_grace(struct nfsd_net *nn);
 
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#13 lore raw

PATCH 10/20 nfsd: implement pNFS operations

In reply to a message not in the archive

Add support for the GETDEVICEINFO, LAYOUTGET, LAYOUTCOMMIT and LAYOUTRETURN NFSv4.1 operations, as well as backing code to manage outstanding layouts and devices.

Layout management is very straight forward, with a nfs4_layout_stateid structure that extends nfs4_stid to manage layout stateids as the top-level structure. It is linked into the nfs4_file and nfs4_client structures like the other stateids, and contains a linked list of layouts that hang of the stateid. The actual layout operations are implemented in layout drivers that are not part of this commit, but will be added later.

The worst part of this commit is the management of the pNFS device IDs, which suffers from a specification that is not sanely implementable due to the fact that the device-IDs are global and not bound to an export, and have a small enough size so that we can't store the fsid portion of a file handle, and must never be reused. As we still do need perform all export authentication and validation checks on a device ID passed to GETDEVICEINFO we are caught between a rock and a hard place. To work around this issue we add a new hash that maps from a 64-bit integer to a fsid so that we can look up the export to authenticate against it, a 32-bit integer as a generation that we can bump when changing the device, and a currently unused 32-bit integer that could be used in the future to handle more than a single device per export. Entries in this hash table are never deleted as we can't reuse the ids anyway, and would have a severe lifetime problem anyway as Linux export structures are temporary structures that can go away under load.

Parts of the XDR data, structures and marshaling/unmarshaling code, as well as many concepts are derived from the old pNFS server implementation from Andy Adamson, Benny Halevy, Dean Hildebrand, Marc Eshel, Fred Isaman, Mike Sager, Ricardo Labiaga and many others.

---
 fs/nfsd/Kconfig                  |  10 +
 fs/nfsd/Makefile                 |   1 +
 fs/nfsd/export.c                 |   8 +
 fs/nfsd/export.h                 |   2 +
 fs/nfsd/nfs4layouts.c            | 488 +++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4proc.c               | 266 +++++++++++++++++++++
 fs/nfsd/nfs4state.c              |  16 +-
 fs/nfsd/nfs4xdr.c                | 312 +++++++++++++++++++++++++
 fs/nfsd/nfsctl.c                 |   9 +-
 fs/nfsd/nfsd.h                   |  16 +-
 fs/nfsd/pnfs.h                   |  80 +++++++
 fs/nfsd/state.h                  |  21 ++
 fs/nfsd/xdr4.h                   |  59 +++++
 include/linux/nfs4.h             |   1 +
 include/uapi/linux/nfsd/debug.h  |   1 +
 include/uapi/linux/nfsd/export.h |   4 +-
 16 files changed, 1289 insertions(+), 5 deletions(-)
 create mode 100644 fs/nfsd/nfs4layouts.c
 create mode 100644 fs/nfsd/pnfs.h
fs/nfsd/Kconfig +10 -0
index 7339515..683bf71 100644
@@ -82,6 +82,16 @@ config NFSD_V4
 
 	  If unsure, say N.
 
+config NFSD_PNFS
+	bool "NFSv4.1 server support for Parallel NFS (pNFS)"
+	depends on NFSD_V4
+	help
+	  This option enables support for the parallel NFS features of the
+	  minor version 1 of the NFSv4 protocol (RFC5661) in the kernel's NFS
+	  server.
+
+	  If unsure, say N.
+
 config NFSD_V4_SECURITY_LABEL
 	bool "Provide Security Label support for NFSv4 server"
 	depends on NFSD_V4 && SECURITY
fs/nfsd/Makefile +1 -0
index af32ef0..5806270 100644
@@ -12,3 +12,4 @@ nfsd-$(CONFIG_NFSD_V3)	+= nfs3proc.o nfs3xdr.o
 nfsd-$(CONFIG_NFSD_V3_ACL) += nfs3acl.o
 nfsd-$(CONFIG_NFSD_V4)	+= nfs4proc.o nfs4xdr.o nfs4state.o nfs4idmap.o \
 			   nfs4acl.o nfs4callback.o nfs4recover.o
+nfsd-$(CONFIG_NFSD_PNFS) += nfs4layouts.o
fs/nfsd/export.c +8 -0
index 30a739d..c3e3b6e 100644
@@ -20,6 +20,7 @@
 #include "nfsd.h"
 #include "nfsfh.h"
 #include "netns.h"
+#include "pnfs.h"
 
 #define NFSDDBG_FACILITY	NFSDDBG_EXPORT
 
@@ -545,6 +546,7 @@ static int svc_export_parse(struct cache_detail *cd, char *mesg, int mlen)
 
 	exp.ex_client = dom;
 	exp.cd = cd;
+	exp.ex_devid_map = NULL;
 
 	/* expiry */
 	err = -EINVAL;
@@ -621,6 +623,8 @@ static int svc_export_parse(struct cache_detail *cd, char *mesg, int mlen)
 		if (!gid_valid(exp.ex_anon_gid))
 			goto out4;
 		err = 0;
+
+		nfsd4_setup_layout_type(&exp);
 	}
 
 	expp = svc_export_lookup(&exp);
@@ -703,6 +707,7 @@ static void svc_export_init(struct cache_head *cnew, struct cache_head *citem)
 	new->ex_fslocs.locations = NULL;
 	new->ex_fslocs.locations_count = 0;
 	new->ex_fslocs.migrated = 0;
+	new->ex_layout_type = 0;
 	new->ex_uuid = NULL;
 	new->cd = item->cd;
 }
@@ -717,6 +722,8 @@ static void export_update(struct cache_head *cnew, struct cache_head *citem)
 	new->ex_anon_uid = item->ex_anon_uid;
 	new->ex_anon_gid = item->ex_anon_gid;
 	new->ex_fsid = item->ex_fsid;
+	new->ex_devid_map = item->ex_devid_map;
+	item->ex_devid_map = NULL;
 	new->ex_uuid = item->ex_uuid;
 	item->ex_uuid = NULL;
 	new->ex_fslocs.locations = item->ex_fslocs.locations;
@@ -725,6 +732,7 @@ static void export_update(struct cache_head *cnew, struct cache_head *citem)
 	item->ex_fslocs.locations_count = 0;
 	new->ex_fslocs.migrated = item->ex_fslocs.migrated;
 	item->ex_fslocs.migrated = 0;
+	new->ex_layout_type = item->ex_layout_type;
 	new->ex_nflavors = item->ex_nflavors;
 	for (i = 0; i < MAX_SECINFO_LIST; i++) {
 		new->ex_flavors[i] = item->ex_flavors[i];
fs/nfsd/export.h +2 -0
index 04dc8c1..1f52bfc 100644
@@ -56,6 +56,8 @@ struct svc_export {
 	struct nfsd4_fs_locations ex_fslocs;
 	uint32_t		ex_nflavors;
 	struct exp_flavor_info	ex_flavors[MAX_SECINFO_LIST];
+	enum pnfs_layouttype	ex_layout_type;
+	struct nfsd4_deviceid_map *ex_devid_map;
 	struct cache_detail	*cd;
 };
 
fs/nfsd/nfs4layouts.c new file +488 -0

488 lines not on this page · raw message

fs/nfsd/nfs4proc.c +266 -0

284 lines not on this page · raw message

fs/nfsd/nfs4state.c +15 -1
index 444d43c..925836e 100644
@@ -48,6 +48,7 @@
 #include "current_stateid.h"
 
 #include "netns.h"
+#include "pnfs.h"
 
 #define NFSDDBG_FACILITY                NFSDDBG_PROC
 
@@ -1544,6 +1545,9 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
 	INIT_LIST_HEAD(&clp->cl_lru);
 	INIT_LIST_HEAD(&clp->cl_callbacks);
 	INIT_LIST_HEAD(&clp->cl_revoked);
+#ifdef CONFIG_NFSD_PNFS
+	INIT_LIST_HEAD(&clp->cl_lo_states);
+#endif
 	spin_lock_init(&clp->cl_lock);
 	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
 	return clp;
@@ -1648,6 +1652,7 @@ __destroy_client(struct nfs4_client *clp)
 		nfs4_get_stateowner(&oo->oo_owner);
 		release_openowner(oo);
 	}
+	nfsd4_return_all_client_layouts(clp);
 	nfsd4_shutdown_callback(clp);
 	if (clp->cl_cb_conn.cb_xprt)
 		svc_xprt_put(clp->cl_cb_conn.cb_xprt);
@@ -2131,8 +2136,11 @@ nfsd4_replay_cache_entry(struct nfsd4_compoundres *resp,
 static void
 nfsd4_set_ex_flags(struct nfs4_client *new, struct nfsd4_exchange_id *clid)
 {
-	/* pNFS is not supported */
+#ifdef CONFIG_NFSD_PNFS
+	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_PNFS_MDS;
+#else
 	new->cl_exchange_flags |= EXCHGID4_FLAG_USE_NON_PNFS;
+#endif
 
 	/* Referrals are supported, Migration is not. */
 	new->cl_exchange_flags |= EXCHGID4_FLAG_SUPP_MOVED_REFER;
@@ -3060,6 +3068,9 @@ static void nfsd4_init_file(struct knfsd_fh *fh, unsigned int hashval,
 	fp->fi_share_deny = 0;
 	memset(fp->fi_fds, 0, sizeof(fp->fi_fds));
 	memset(fp->fi_access, 0, sizeof(fp->fi_access));
+#ifdef CONFIG_NFSD_PNFS
+	INIT_LIST_HEAD(&fp->fi_lo_states);
+#endif
 	hlist_add_head_rcu(&fp->fi_hash, &file_hashtbl[hashval]);
 }
 
@@ -4846,6 +4857,9 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	update_stateid(&stp->st_stid.sc_stateid);
 	memcpy(&close->cl_stateid, &stp->st_stid.sc_stateid, sizeof(stateid_t));
 
+	nfsd4_return_all_file_layouts(stp->st_stateowner->so_client,
+				      stp->st_stid.sc_file);
+
 	nfsd4_close_open_stateid(stp);
 
 	/* put reference from nfs4_preprocess_seqid_op */
fs/nfsd/nfs4xdr.c +312 -0

358 lines not on this page · raw message

fs/nfsd/nfsctl.c +8 -1
index 19ace74..aa47d75 100644
@@ -21,6 +21,7 @@
 #include "cache.h"
 #include "state.h"
 #include "netns.h"
+#include "pnfs.h"
 
 /*
  *	We have a single directory with several nodes in it.
@@ -1258,9 +1259,12 @@ static int __init init_nfsd(void)
 	retval = nfsd4_init_slabs();
 	if (retval)
 		goto out_unregister_pernet;
-	retval = nfsd_fault_inject_init(); /* nfsd fault injection controls */
+	retval = nfsd4_init_pnfs();
 	if (retval)
 		goto out_free_slabs;
+	retval = nfsd_fault_inject_init(); /* nfsd fault injection controls */
+	if (retval)
+		goto out_exit_pnfs;
 	nfsd_stat_init();	/* Statistics */
 	retval = nfsd_reply_cache_init();
 	if (retval)
@@ -1282,6 +1286,8 @@ out_free_lockd:
 out_free_stat:
 	nfsd_stat_shutdown();
 	nfsd_fault_inject_cleanup();
+out_exit_pnfs:
+	nfsd4_exit_pnfs();
 out_free_slabs:
 	nfsd4_free_slabs();
 out_unregister_pernet:
@@ -1299,6 +1305,7 @@ static void __exit exit_nfsd(void)
 	nfsd_stat_shutdown();
 	nfsd_lockd_shutdown();
 	nfsd4_free_slabs();
+	nfsd4_exit_pnfs();
 	nfsd_fault_inject_cleanup();
 	unregister_filesystem(&nfsd_fs_type);
 	unregister_pernet_subsys(&nfsd_net_ops);
fs/nfsd/nfsd.h +14 -2
index 33a46a8..565c4da 100644
@@ -325,15 +325,27 @@ void		nfsd_lockd_shutdown(void);
 
 #define NFSD4_SUPPORTED_ATTRS_WORD2 0
 
+/* 4.1 */
+#ifdef CONFIG_NFSD_PNFS
+#define PNFSD_SUPPORTED_ATTRS_WORD1	FATTR4_WORD1_FS_LAYOUT_TYPES
+#define PNFSD_SUPPORTED_ATTRS_WORD2 \
+(FATTR4_WORD2_LAYOUT_BLKSIZE	| FATTR4_WORD2_LAYOUT_TYPES)
+#else
+#define PNFSD_SUPPORTED_ATTRS_WORD1	0
+#define PNFSD_SUPPORTED_ATTRS_WORD2	0
+#endif /* CONFIG_NFSD_PNFS */
+
 #define NFSD4_1_SUPPORTED_ATTRS_WORD0 \
 	NFSD4_SUPPORTED_ATTRS_WORD0
 
 #define NFSD4_1_SUPPORTED_ATTRS_WORD1 \
-	NFSD4_SUPPORTED_ATTRS_WORD1
+	(NFSD4_SUPPORTED_ATTRS_WORD1	| PNFSD_SUPPORTED_ATTRS_WORD1)
 
 #define NFSD4_1_SUPPORTED_ATTRS_WORD2 \
-	(NFSD4_SUPPORTED_ATTRS_WORD2 | FATTR4_WORD2_SUPPATTR_EXCLCREAT)
+	(NFSD4_SUPPORTED_ATTRS_WORD2	| PNFSD_SUPPORTED_ATTRS_WORD2 | \
+	 FATTR4_WORD2_SUPPATTR_EXCLCREAT)
 
+/* 4.2 */
 #ifdef CONFIG_NFSD_V4_SECURITY_LABEL
 #define NFSD4_2_SECURITY_ATTRS		FATTR4_WORD2_SECURITY_LABEL
 #else
fs/nfsd/pnfs.h new file +80 -0

80 lines not on this page · raw message

fs/nfsd/state.h +21 -0
index 18b5ab8..4641fbb 100644
@@ -92,6 +92,7 @@ struct nfs4_stid {
 /* For a deleg stateid kept around only to process free_stateid's: */
 #define NFS4_REVOKED_DELEG_STID 16
 #define NFS4_CLOSED_DELEG_STID 32
+#define NFS4_LAYOUT_STID 64
 	unsigned char sc_type;
 	stateid_t sc_stateid;
 	struct nfs4_client *sc_client;
@@ -297,6 +298,9 @@ struct nfs4_client {
 	struct list_head	cl_delegations;
 	struct list_head	cl_revoked;	/* unacknowledged, revoked 4.1 state */
 	struct list_head        cl_lru;         /* tail queue */
+#ifdef CONFIG_NFSD_PNFS
+	struct list_head	cl_lo_states;	/* outstanding layout states */
+#endif
 	struct xdr_netobj	cl_name; 	/* id generated by client */
 	nfs4_verifier		cl_verifier; 	/* generated by client */
 	time_t                  cl_time;        /* time of last lease renewal */
@@ -496,6 +500,9 @@ struct nfs4_file {
 	atomic_t		fi_delegees;
 	struct knfsd_fh		fi_fhandle;
 	bool			fi_had_conflict;
+#ifdef CONFIG_NFSD_PNFS
+	struct list_head	fi_lo_states;
+#endif
 };
 
 /*
@@ -528,6 +535,20 @@ static inline struct nfs4_ol_stateid *openlockstateid(struct nfs4_stid *s)
 	return container_of(s, struct nfs4_ol_stateid, st_stid);
 }
 
+struct nfs4_layout_stateid {
+	struct nfs4_stid		ls_stid;
+	struct list_head		ls_perclnt;
+	struct list_head		ls_perfile;
+	spinlock_t			ls_lock;
+	struct list_head		ls_layouts;
+	u32				ls_layout_type;
+};
+
+static inline struct nfs4_layout_stateid *layoutstateid(struct nfs4_stid *s)
+{
+	return container_of(s, struct nfs4_layout_stateid, ls_stid);
+}
+
 /* flags for preprocess_seqid_op() */
 #define RD_STATE	        0x00000010
 #define WR_STATE	        0x00000020
fs/nfsd/xdr4.h +59 -0

71 lines not on this page · raw message

include/linux/nfs4.h +1 -0
index 8a3589c..bc10d68 100644
@@ -411,6 +411,7 @@ enum lock_type4 {
 #define FATTR4_WORD1_TIME_MODIFY_SET    (1UL << 22)
 #define FATTR4_WORD1_MOUNTED_ON_FILEID  (1UL << 23)
 #define FATTR4_WORD1_FS_LAYOUT_TYPES    (1UL << 30)
+#define FATTR4_WORD2_LAYOUT_TYPES       (1UL << 0)
 #define FATTR4_WORD2_LAYOUT_BLKSIZE     (1UL << 1)
 #define FATTR4_WORD2_MDSTHRESHOLD       (1UL << 4)
 #define FATTR4_WORD2_SECURITY_LABEL     (1UL << 16)
include/uapi/linux/nfsd/debug.h +1 -0
index 1fdc95b..0bf130a 100644
@@ -32,6 +32,7 @@
 #define NFSDDBG_REPCACHE	0x0080
 #define NFSDDBG_XDR		0x0100
 #define NFSDDBG_LOCKD		0x0200
+#define NFSDDBG_PNFS		0x0400
 #define NFSDDBG_ALL		0x7FFF
 #define NFSDDBG_NOCHANGE	0xFFFF
 
include/uapi/linux/nfsd/export.h +3 -1
index 584b6ef..4742f2c 100644
@@ -47,8 +47,10 @@
  * exported filesystem.
  */
 #define	NFSEXP_V4ROOT		0x10000
+#define NFSEXP_NOPNFS		0x20000
+
 /* All flags that we claim to support.  (Note we don't support NOACL.) */
-#define NFSEXP_ALLFLAGS		0x1FE7F
+#define NFSEXP_ALLFLAGS		0x3FE7F
 
 /* The flags that may vary depending on security flavor: */
 #define NFSEXP_SECINFO_FLAGS	(NFSEXP_READONLY | NFSEXP_ROOTSQUASH \
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-11-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#15 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 12 lines · Christoph Hellwig

Add support for the GETDEVICEINFO, LAYOUTGET, LAYOUTCOMMIT and LAYOUTRETURN NFSv4.1 operations, as well as backing code to manage outstanding layouts and devices.

Layout management is very straight forward, with a nfs4_layout_stateid structure that extends nfs4_stid to manage layout stateids as the top-level structure. It is linked into the nfs4_file and nfs4_client structures like the other stateids, and contains a linked list of layouts that hang of the stateid. The actual layout operations are implemented in layout drivers that are not part of this commit, but will be added later.

Quoted 15 lines

The worst part of this commit is the management of the pNFS device IDs, which suffers from a specification that is not sanely implementable due to the fact that the device-IDs are global and not bound to an export, and have a small enough size so that we can't store the fsid portion of a file handle, and must never be reused. As we still do need perform all export authentication and validation checks on a device ID passed to GETDEVICEINFO we are caught between a rock and a hard place. To work around this issue we add a new hash that maps from a 64-bit integer to a fsid so that we can look up the export to authenticate against it, a 32-bit integer as a generation that we can bump when changing the device, and a currently unused 32-bit integer that could be used in the future to handle more than a single device per export. Entries in this hash table are never deleted as we can't reuse the ids anyway, and would have a severe lifetime problem anyway as Linux export structures are temporary structures that can go away under load.

Looks to me like that works.

...

Quoted 6 lines
fs/nfsd/nfs4layouts.c new file +0 -0
new file mode 100644
index 0000000..28c8ff2
@@ -0,0 +1,488 @@

...

Quoted 27 lines
fs/nfsd/nfs4layouts.c new file +27 -0
+__be32
+nfsd4_preprocess_layout_stateid(struct svc_rqst *rqstp,
+		struct nfsd4_compound_state *cstate, stateid_t *stateid,
+		bool create, u32 layout_type, struct nfs4_layout_stateid **lsp)
+{
+	struct nfs4_layout_stateid *ls;
+	struct nfs4_stid *stid;
+	unsigned char typemask = NFS4_LAYOUT_STID;
+	__be32 status;
+
+	if (create)
+		typemask |= (NFS4_OPEN_STID | NFS4_LOCK_STID | NFS4_DELEG_STID);
+
+	status = nfsd4_lookup_stateid(cstate, stateid, typemask, &stid,
+			net_generic(SVC_NET(rqstp), nfsd_net_id));
+	if (status)
+		goto out;
+
+	if (!fh_match(&cstate->current_fh.fh_handle,
+		      &stid->sc_file->fi_fhandle)) {
+		status = nfserr_bad_stateid;
+		goto out_put_stid;
+	}
+
+	if (stid->sc_type != NFS4_LAYOUT_STID) {
+		ls = nfsd4_alloc_layout_stateid(cstate, stid, layout_type);
+		nfs4_put_stid(stid);

fs/nfsd/nfs4layouts.c line 205

Quoted 10 lines
fs/nfsd/nfs4layouts.c new file +10 -0
+
+		status = nfserr_jukebox;
+		if (!ls)
+			goto out;
+	} else {
+		ls = container_of(stid, struct nfs4_layout_stateid, ls_stid);
+
+		status = nfserr_bad_stateid;
+		if (stateid->si_generation > stid->sc_stateid.si_generation)
+			goto out_put_stid;

Is there no old_stateid case for layout stateids? And is there any chance of wraparound? (I was comparing to check_stateid_generation and expecting the only difference to be the handling of the generation-zero case.)

Quoted 8 lines
fs/nfsd/nfs4layouts.c new file +8 -0
+		if (layout_type != ls->ls_layout_type)
+			goto out_put_stid;
+	}
+
+	*lsp = ls;
+	return 0;
+
+out_put_stid:

fs/nfsd/nfs4layouts.c line 223

Quoted 10 lines
fs/nfsd/nfs4layouts.c new file +10 -0
+	nfs4_put_stid(stid);
+out:
+	return status;
+}
+
+static inline u64
+layout_end(struct nfsd4_layout_seg *seg)
+{
+	u64 end = seg->offset + seg->length;
+	return end >= seg->offset ? seg->length : NFS4_MAX_UINT64;

Shouldn't that be

	return end >= seg->offset ? end : NFS_MAX_UINT64;

?

fs/nfsd/nfs4layouts.c line 230

Quoted 7 lines
fs/nfsd/nfs4layouts.c new file +7 -0
+}
+
+static void
+layout_update_len(struct nfsd4_layout_seg *lo, u64 end)
+{
+	if (end == NFS4_MAX_UINT64)
+		lo->length = NFS4_MAX_UINT64;

Is this case necessary?

Quoted 3 lines
fs/nfsd/nfs4layouts.c new file +3 -0
+	else
+		lo->length = end - lo->offset;
+}

...

Quoted 4 lines
fs/nfsd/nfs4proc.c +0 -0
index ac71d13..b813913 100644

...

Quoted 12 lines
fs/nfsd/nfs4proc.c +8 -0
@@ -1966,6 +2213,25 @@ static struct nfsd4_operation nfsd4_ops[] = {
 		.op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid,
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize,
 	},
+#ifdef CONFIG_NFSD_PNFS
+	[OP_GETDEVICEINFO] = {
+		.op_func = (nfsd4op_func)nfsd4_getdeviceinfo,
+		.op_flags = ALLOWED_WITHOUT_FH,
+		.op_name = "OP_GETDEVICEINFO",
+	},
+	[OP_LAYOUTGET] = {
+		.op_func = (nfsd4op_func)nfsd4_layoutget,

fs/nfsd/nfs4proc.c static struct nfsd4_operation nfsd4_ops[] = { · line 2233

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+		.op_name = "OP_LAYOUTGET",
+	},
+	[OP_LAYOUTCOMMIT] = {
+		.op_func = (nfsd4op_func)nfsd4_layoutcommit,
+		.op_name = "OP_LAYOUTCOMMIT",
+	},
+	[OP_LAYOUTRETURN] = {
+		.op_func = (nfsd4op_func)nfsd4_layoutreturn,
+		.op_name = "OP_LAYOUTRETURN",
+	},

Should any of these have OP_MODIFIES_SOMETHING set? (Basically: would we be in trouble if we succesfully completed one of these operations and then weren't able to encode the result?)

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#17 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 4 lines · J. Bruce Fields

Is there no old_stateid case for layout stateids? And is there any chance of wraparound? (I was comparing to check_stateid_generation and expecting the only difference to be the handling of the generation-zero case.)

12.5.3. explicitly mentions that LAYOUTGET and LAYOUTRETURN might be outstading and processed in parallel, and sais that pNFS operations use special stateid rules. It does not explicitly say that old stateids are ok, but the model described in there very much requires the server to not reject them.

Quoted 11 lines

fs/nfsd/nfs4layouts.c line 223

Quoted 5 lines
fs/nfsd/nfs4layouts.c new file +5 -0
+static inline u64
+layout_end(struct nfsd4_layout_seg *seg)
+{
+	u64 end = seg->offset + seg->length;
+	return end >= seg->offset ? seg->length : NFS4_MAX_UINT64;

Shouldn't that be

	return end >= seg->offset ? end : NFS_MAX_UINT64;

?

Yes. This is an interesting one that sneaked in, and it turns out besides dislabling layout merging it didn't have adverse effects.

Quoted 10 lines

fs/nfsd/nfs4layouts.c line 230

Quoted 7 lines
fs/nfsd/nfs4layouts.c new file +7 -0
+}
+
+static void
+layout_update_len(struct nfsd4_layout_seg *lo, u64 end)
+{
+	if (end == NFS4_MAX_UINT64)
+		lo->length = NFS4_MAX_UINT64;

Is this case necessary?

Quoted 2 lines

fs/nfsd/nfs4layouts.c line 232

Quoted 2 lines
fs/nfsd/nfs4layouts.c new file +2 -0
+	else
+		lo->length = end - lo->offset;

We use NFS4_MAX_UINT64 as a magic value for layouts until the field end, as specified in the standard. But because we do all kinds of calculations using the end value we need to propagate that magic from and to it.

Quoted 3 lines

Should any of these have OP_MODIFIES_SOMETHING set? (Basically: would we be in trouble if we succesfully completed one of these operations and then weren't able to encode the result?)

All but GETDEVICEINFO should get it.

I've implemented all your suggested changes and will send out and update after doing a little more testing.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#18 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 46 lines · Christoph Hellwig
Quoted 4 lines · J. Bruce Fields

Is there no old_stateid case for layout stateids? And is there any chance of wraparound? (I was comparing to check_stateid_generation and expecting the only difference to be the handling of the generation-zero case.)

12.5.3. explicitly mentions that LAYOUTGET and LAYOUTRETURN might be outstading and processed in parallel, and sais that pNFS operations use special stateid rules. It does not explicitly say that old stateids are ok, but the model described in there very much requires the server to not reject them.

Quoted 11 lines

fs/nfsd/nfs4layouts.c line 223

Quoted 5 lines
fs/nfsd/nfs4layouts.c new file +5 -0
+static inline u64
+layout_end(struct nfsd4_layout_seg *seg)
+{
+	u64 end = seg->offset + seg->length;
+	return end >= seg->offset ? seg->length : NFS4_MAX_UINT64;

Shouldn't that be

	return end >= seg->offset ? end : NFS_MAX_UINT64;

?

Yes. This is an interesting one that sneaked in, and it turns out besides dislabling layout merging it didn't have adverse effects.

Quoted 10 lines

fs/nfsd/nfs4layouts.c line 230

Quoted 7 lines
fs/nfsd/nfs4layouts.c new file +7 -0
+}
+
+static void
+layout_update_len(struct nfsd4_layout_seg *lo, u64 end)
+{
+	if (end == NFS4_MAX_UINT64)
+		lo->length = NFS4_MAX_UINT64;

Is this case necessary?

Quoted 2 lines

fs/nfsd/nfs4layouts.c line 232

Quoted 2 lines
fs/nfsd/nfs4layouts.c new file +2 -0
+	else
+		lo->length = end - lo->offset;

We use NFS4_MAX_UINT64 as a magic value for layouts until the field end, as specified in the standard. But because we do all kinds of calculations using the end value we need to propagate that magic from and to it.

Quoted 8 lines
Quoted 3 lines

Should any of these have OP_MODIFIES_SOMETHING set? (Basically: would we be in trouble if we succesfully completed one of these operations and then weren't able to encode the result?)

All but GETDEVICEINFO should get it.

I've implemented all your suggested changes and will send out and update after doing a little more testing.

Thanks!

I didn't notice anything that looked like a big problem to me, so absent any objections I'll commit the revised versions for 3.20 once we figure out how to handle the xfs stuff.

--b.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#20 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 3 lines · J. Bruce Fields

I didn't notice anything that looked like a big problem to me, so absent any objections I'll commit the revised versions for 3.20 once we figure out how to handle the xfs stuff.

Do you want the resend to be on top of Jeffs locks tree, or do you want me to be based just on the nfsd changes, which would require a fairly trivial merge once both of the trees hit mainline?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#22 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 8 lines · Christoph Hellwig
Quoted 3 lines · J. Bruce Fields

I didn't notice anything that looked like a big problem to me, so absent any objections I'll commit the revised versions for 3.20 once we figure out how to handle the xfs stuff.

Do you want the resend to be on top of Jeffs locks tree, or do you want me to be based just on the nfsd changes, which would require a fairly trivial merge once both of the trees hit mainline?

I'm planning to pull Jeff's tree and then apply these on top. (Even if the conflict's fairly trivial I'm just happier being able to test the combination exactly as they're commited.)

I'll do that now, should be pushed out in an hour or two.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 5 lines · J. Bruce Fields

I'm planning to pull Jeff's tree and then apply these on top. (Even if the conflict's fairly trivial I'm just happier being able to test the combination exactly as they're commited.)

I'll do that now, should be pushed out in an hour or two.

I've pushed out a pnfsd-for-3.20-3 branch to
git://git.infradead.org/users/hch/pnfs.git

Or do you want me to resend the whole thing?
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#26 lore raw

Re: [PATCH 10/20] nfsd: implement pNFS operations

Quoted 11 lines · Christoph Hellwig
Quoted 5 lines · J. Bruce Fields

I'm planning to pull Jeff's tree and then apply these on top. (Even if the conflict's fairly trivial I'm just happier being able to test the combination exactly as they're commited.)

I'll do that now, should be pushed out in an hour or two.

I've pushed out a pnfsd-for-3.20-3 branch to
git://git.infradead.org/users/hch/pnfs.git

Or do you want me to resend the whole thing?

No, looks fine to me as is. Thanks! I'll play around with it a little more and then push it out.

GETLAYOUT is going to be annoying if we need big layouts. We'll have to figure out what to do with that reply size estimation, and clients will have to figure out how to get the maxresponsesized_cached right, I guess? Oh well.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#27 lore raw

PATCH 11/20 nfsd: implement pNFS layout recalls

In reply to a message not in the archive

Add support to issue layout recalls to clients. For now we only support full-file recalls to get a simple and stable implementation. This allows to embedd a nfsd4_callback structure in the layout_state and thus avoid any memory allocations under spinlocks during a recall. For normal use cases that do not intent to share a single file between multiple clients this implementation is fully sufficient.

To ensure layouts are recalled on local filesystem access each layout state registers a new FL_LAYOUT lease with the kernel file locking code, which filesystems that support pNFS exports that require recalls need to break on conflicting access patterns.

The XDR code is based on the old pNFS server implementation by Andy Adamson, Benny Halevy, Boaz Harrosh, Dean Hildebrand, Fred Isaman, Marc Eshel, Mike Sager and Ricardo Labiaga.

---
 fs/nfsd/nfs4callback.c |  99 +++++++++++++++++++++++
 fs/nfsd/nfs4layouts.c  | 214 ++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4proc.c     |   4 +
 fs/nfsd/nfs4state.c    |   1 +
 fs/nfsd/state.h        |   6 ++
 fs/nfsd/xdr4cb.h       |   7 ++
 6 files changed, 330 insertions(+), 1 deletion(-)
fs/nfsd/nfs4callback.c +99 -0
index 7cbdf1b..5827785 100644
@@ -546,6 +546,102 @@ out:
 	return status;
 }
 
+#ifdef CONFIG_NFSD_PNFS
+/*
+ * CB_LAYOUTRECALL4args
+ *
+ *	struct layoutrecall_file4 {
+ *		nfs_fh4         lor_fh;
+ *		offset4         lor_offset;
+ *		length4         lor_length;
+ *		stateid4        lor_stateid;
+ *	};
+ *
+ *	union layoutrecall4 switch(layoutrecall_type4 lor_recalltype) {
+ *	case LAYOUTRECALL4_FILE:
+ *		layoutrecall_file4 lor_layout;
+ *	case LAYOUTRECALL4_FSID:
+ *		fsid4              lor_fsid;
+ *	case LAYOUTRECALL4_ALL:
+ *		void;
+ *	};
+ *
+ *	struct CB_LAYOUTRECALL4args {
+ *		layouttype4             clora_type;
+ *		layoutiomode4           clora_iomode;
+ *		bool                    clora_changed;
+ *		layoutrecall4           clora_recall;
+ *	};
+ */
+static void encode_cb_layout4args(struct xdr_stream *xdr,
+				  const struct nfs4_layout_stateid *ls,
+				  struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	BUG_ON(hdr->minorversion == 0);
+
+	p = xdr_reserve_space(xdr, 5 * 4);
+	*p++ = cpu_to_be32(OP_CB_LAYOUTRECALL);
+	*p++ = cpu_to_be32(ls->ls_layout_type);
+	*p++ = cpu_to_be32(IOMODE_ANY);
+	*p++ = cpu_to_be32(1);
+	*p = cpu_to_be32(RETURN_FILE);
+
+	encode_nfs_fh4(xdr, &ls->ls_stid.sc_file->fi_fhandle);
+
+	p = xdr_reserve_space(xdr, 2 * 8);
+	p = xdr_encode_hyper(p, 0);
+	xdr_encode_hyper(p, NFS4_MAX_UINT64);
+
+	encode_stateid4(xdr, &ls->ls_recall_sid);
+
+	hdr->nops++;
+}
+
+static void nfs4_xdr_enc_cb_layout(struct rpc_rqst *req,
+				   struct xdr_stream *xdr,
+				   const struct nfsd4_callback *cb)
+{
+	const struct nfs4_layout_stateid *ls =
+		container_of(cb, struct nfs4_layout_stateid, ls_recall);
+	struct nfs4_cb_compound_hdr hdr = {
+		.ident = 0,
+		.minorversion = cb->cb_minorversion,
+	};
+
+	encode_cb_compound4args(xdr, &hdr);
+	encode_cb_sequence4args(xdr, cb, &hdr);
+	encode_cb_layout4args(xdr, ls, &hdr);
+	encode_cb_nops(&hdr);
+}
+
+static int nfs4_xdr_dec_cb_layout(struct rpc_rqst *rqstp,
+				  struct xdr_stream *xdr,
+				  struct nfsd4_callback *cb)
+{
+	struct nfs4_cb_compound_hdr hdr;
+	enum nfsstat4 nfserr;
+	int status;
+
+	status = decode_cb_compound4res(xdr, &hdr);
+	if (unlikely(status))
+		goto out;
+	if (cb) {
+		status = decode_cb_sequence4res(xdr, cb);
+		if (unlikely(status))
+			goto out;
+	}
+	status = decode_cb_op_status(xdr, OP_CB_LAYOUTRECALL, &nfserr);
+	if (unlikely(status))
+		goto out;
+	if (unlikely(nfserr != NFS4_OK))
+		status = nfs_cb_stat_to_errno(nfserr);
+out:
+	return status;
+}
+#endif /* CONFIG_NFSD_PNFS */
+
 /*
  * RPC procedure tables
  */
@@ -563,6 +659,9 @@ out:
 static struct rpc_procinfo nfs4_cb_procedures[] = {
 	PROC(CB_NULL,	NULL,		cb_null,	cb_null),
 	PROC(CB_RECALL,	COMPOUND,	cb_recall,	cb_recall),
+#ifdef CONFIG_NFSD_PNFS
+	PROC(CB_LAYOUT,	COMPOUND,	cb_layout,	cb_layout),
+#endif
 };
 
 static struct rpc_version nfs_cb_version4 = {
fs/nfsd/nfs4layouts.c +213 -1

287 lines not on this page · raw message

fs/nfsd/nfs4proc.c +4 -0
index b813913..c051d5b 100644
@@ -1301,6 +1301,10 @@ nfsd4_layoutget(struct svc_rqst *rqstp,
 	if (nfserr)
 		goto out;
 
+	nfserr = nfserr_recallconflict;
+	if (atomic_read(&ls->ls_stid.sc_file->fi_lo_recalls))
+		goto out_put_stid;
+
 	nfserr = ops->proc_layoutget(current_fh->fh_dentry->d_inode,
 				     current_fh, lgp);
 	if (nfserr)
fs/nfsd/nfs4state.c +1 -0
index 925836e..1eba300 100644
@@ -3070,6 +3070,7 @@ static void nfsd4_init_file(struct knfsd_fh *fh, unsigned int hashval,
 	memset(fp->fi_access, 0, sizeof(fp->fi_access));
 #ifdef CONFIG_NFSD_PNFS
 	INIT_LIST_HEAD(&fp->fi_lo_states);
+	atomic_set(&fp->fi_lo_recalls, 0);
 #endif
 	hlist_add_head_rcu(&fp->fi_hash, &file_hashtbl[hashval]);
 }
fs/nfsd/state.h +6 -0
index 4641fbb..9290534 100644
@@ -502,6 +502,7 @@ struct nfs4_file {
 	bool			fi_had_conflict;
 #ifdef CONFIG_NFSD_PNFS
 	struct list_head	fi_lo_states;
+	atomic_t		fi_lo_recalls;
 #endif
 };
 
@@ -542,6 +543,10 @@ struct nfs4_layout_stateid {
 	spinlock_t			ls_lock;
 	struct list_head		ls_layouts;
 	u32				ls_layout_type;
+	struct file			*ls_file;
+	struct nfsd4_callback		ls_recall;
+	stateid_t			ls_recall_sid;
+	bool				ls_recalled;
 };
 
 static inline struct nfs4_layout_stateid *layoutstateid(struct nfs4_stid *s)
@@ -556,6 +561,7 @@ static inline struct nfs4_layout_stateid *layoutstateid(struct nfs4_stid *s)
 enum nfsd4_cb_op {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
+	NFSPROC4_CLNT_CB_LAYOUT,
 	NFSPROC4_CLNT_CB_SEQUENCE,
 };
 
fs/nfsd/xdr4cb.h +7 -0
index c5c55df..c47f6fd 100644
@@ -21,3 +21,10 @@
 #define NFS4_dec_cb_recall_sz		(cb_compound_dec_hdr_sz  +      \
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
+#define NFS4_enc_cb_layout_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
+					1 + 3 +                         \
+					enc_nfs4_fh_sz + 4)
+#define NFS4_dec_cb_layout_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
+					op_dec_sz)
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#28 lore raw

PATCH 14/20 exportfs: add methods for block layout exports

In reply to a message not in the archive

Add three methods to allow exporting pnfs block layout volumes:

- get_uuid: get a filesystem unique signature exposed to clients
- map_blocks: map and if nessecary allocate blocks for a layout
- commit_blocks: commit blocks in a layout once the client is done with them

For now we stick the external pnfs block layout interfaces into s_export_op to avoid mixing them up with the internal interface between the NFS server and the layout drivers. Once we've fully internalized the latter interface we can redecide if these methods should stay in s_export_ops.

---
 include/linux/exportfs.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
include/linux/exportfs.h +21 -0
index 41b223a..ff46bf7 100644
@@ -4,6 +4,7 @@
 #include <linux/types.h>
 
 struct dentry;
+struct iattr;
 struct inode;
 struct super_block;
 struct vfsmount;
@@ -180,6 +181,19 @@ struct fid {
  *    get_name is not (which is possibly inconsistent)
  */
 
+/* types of block ranges for multipage write mappings. */
+#define IOMAP_HOLE	0x01	/* no blocks allocated, need allocation */
+#define IOMAP_DELALLOC	0x02	/* delayed allocation blocks */
+#define IOMAP_MAPPED	0x03	/* blocks allocated @blkno */
+#define IOMAP_UNWRITTEN	0x04	/* blocks allocated @blkno in unwritten state */
+
+struct iomap {
+	sector_t	blkno;	/* first sector of mapping */
+	loff_t		offset;	/* file offset of mapping, bytes */
+	u64		length;	/* length of mapping, bytes */
+	int		type;	/* type of mapping */
+};
+
 struct export_operations {
 	int (*encode_fh)(struct inode *inode, __u32 *fh, int *max_len,
 			struct inode *parent);
@@ -191,6 +205,13 @@ struct export_operations {
 			struct dentry *child);
 	struct dentry * (*get_parent)(struct dentry *child);
 	int (*commit_metadata)(struct inode *inode);
+
+	int (*get_uuid)(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
+	int (*map_blocks)(struct inode *inode, loff_t offset,
+			  u64 len, struct iomap *iomap,
+			  bool write, u32 *device_generation);
+	int (*commit_blocks)(struct inode *inode, struct iomap *iomaps,
+			     int nr_iomaps, struct iattr *iattr);
 };
 
 extern int exportfs_encode_inode_fh(struct inode *inode, struct fid *fid,
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#29 lore raw

PATCH 16/20 xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

In reply to a message not in the archive

The code is already ready for it, and the pnfs layout commit code expects to be able to pass a larger than 32-bit argument.

---
 fs/xfs/xfs_iomap.c | 2 +-
 fs/xfs/xfs_iomap.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_iomap.c +1 -1
index c980e2a..ccb1dd0 100644
@@ -802,7 +802,7 @@ int
 xfs_iomap_write_unwritten(
 	xfs_inode_t	*ip,
 	xfs_off_t	offset,
-	size_t		count)
+	xfs_off_t	count)
 {
 	xfs_mount_t	*mp = ip->i_mount;
 	xfs_fileoff_t	offset_fsb;
fs/xfs/xfs_iomap.h +1 -1
index 411fbb8..8688e66 100644
@@ -27,6 +27,6 @@ int xfs_iomap_write_delay(struct xfs_inode *, xfs_off_t, size_t,
 			struct xfs_bmbt_irec *);
 int xfs_iomap_write_allocate(struct xfs_inode *, xfs_off_t,
 			struct xfs_bmbt_irec *);
-int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, size_t);
+int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, xfs_off_t);
 
 #endif /* __XFS_IOMAP_H__*/
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#30 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Who can give us ACKs on these last five fs/xfs patches? (And is it going to cause trouble if they go in through the nfsd tree?)

--b.

Quoted 36 lines · Christoph Hellwig

The code is already ready for it, and the pnfs layout commit code expects to be able to pass a larger than 32-bit argument.

---
 fs/xfs/xfs_iomap.c | 2 +-
 fs/xfs/xfs_iomap.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
fs/xfs/xfs_iomap.c +1 -1
index c980e2a..ccb1dd0 100644
@@ -802,7 +802,7 @@ int
 xfs_iomap_write_unwritten(
 	xfs_inode_t	*ip,
 	xfs_off_t	offset,
-	size_t		count)
+	xfs_off_t	count)
 {
 	xfs_mount_t	*mp = ip->i_mount;
 	xfs_fileoff_t	offset_fsb;
fs/xfs/xfs_iomap.h +1 -1
index 411fbb8..8688e66 100644
@@ -27,6 +27,6 @@ int xfs_iomap_write_delay(struct xfs_inode *, xfs_off_t, size_t,
 			struct xfs_bmbt_irec *);
 int xfs_iomap_write_allocate(struct xfs_inode *, xfs_off_t,
 			struct xfs_bmbt_irec *);
-int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, size_t);
+int xfs_iomap_write_unwritten(struct xfs_inode *, xfs_off_t, xfs_off_t);
 
 #endif /* __XFS_IOMAP_H__*/
Signature · 1 line
1.9.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

author reviewer
#31 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 2 lines · J. Bruce Fields

Who can give us ACKs on these last five fs/xfs patches? (And is it going to cause trouble if they go in through the nfsd tree?)

We'd need ACKs from Dave. He already has pulled in two patches so we might run into some conflicts. Maybe the best idea is to add the exportfs patch to both the XFS and nfsd trees, so each of them can pull in the rest? Or we could commit the two XFS preparation patches to both tree and get something that compiles and works in the nfsd tree.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#32 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 11 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields

Who can give us ACKs on these last five fs/xfs patches? (And is it going to cause trouble if they go in through the nfsd tree?)

We'd need ACKs from Dave. He already has pulled in two patches so we might run into some conflicts. Maybe the best idea is to add the exportfs patch to both the XFS and nfsd trees, so each of them can pull in the rest? Or we could commit the two XFS preparation patches to both tree and get something that compiles and works in the nfsd tree.

This patch has already been committed to the XFS repo.

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#33 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 6 lines · Dave Chinner
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields

Who can give us ACKs on these last five fs/xfs patches? (And is it going to cause trouble if they go in through the nfsd tree?)

Quoted 8 lines
Quoted 6 lines

We'd need ACKs from Dave. He already has pulled in two patches so we might run into some conflicts. Maybe the best idea is to add the exportfs patch to both the XFS and nfsd trees, so each of them can pull in the rest? Or we could commit the two XFS preparation patches to both tree and get something that compiles and works in the nfsd tree.

This patch has already been committed to the XFS repo.

And it looks like I missed the sync transaction on growfs patch, too, so I'll commit that one later today.

As to the pNFSD specific changes, I haven't really looked them over in any great detail yet. My main concern is that there are no specific regression tests for this yet, I'm not sure how we go about verifying it actually works properly and we don't inadvertantly break it in the future. Christoph?

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#34 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 16 lines · Dave Chinner
Quoted 6 lines · Dave Chinner
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields

Who can give us ACKs on these last five fs/xfs patches? (And is it going to cause trouble if they go in through the nfsd tree?)

Quoted 8 lines
Quoted 6 lines

We'd need ACKs from Dave. He already has pulled in two patches so we might run into some conflicts. Maybe the best idea is to add the exportfs patch to both the XFS and nfsd trees, so each of them can pull in the rest? Or we could commit the two XFS preparation patches to both tree and get something that compiles and works in the nfsd tree.

This patch has already been committed to the XFS repo.

Quoted 8 lines

And it looks like I missed the sync transaction on growfs patch, too, so I'll commit that one later today.

As to the pNFSD specific changes, I haven't really looked them over in any great detail yet. My main concern is that there are no specific regression tests for this yet, I'm not sure how we go about verifying it actually works properly and we don't inadvertantly break it in the future. Christoph?

Previously: http://lkml.kernel.org/r/[email protected]

	>       - any advice on testing?  Is there was some simple
	>       virtual setup that would allow any loser with no special
	>       hardware (e.g., me) to check whether they've broken the
	>       block server?

Run two kvm VMs that share the same disk. Create an XFS filesystem on the MDS, and export it. If the client has blkmapd running (on Debian it needs to be started manually) it will use pNFS for accessing the filesystem. Verify that using the per-operation counters in /proc/self/mounstats. Repeat with additional clients as nessecary.

Alternatively set up a simple iSCSI target using tgt or lio and connect to it from multiple clients.

Which sounds reasonable to me, but I haven't tried to incorporate this into my regression testing yet.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author reviewer
#35 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 14 lines · J. Bruce Fields

Previously: http://lkml.kernel.org/r/[email protected]

	>       - any advice on testing?  Is there was some simple
	>       virtual setup that would allow any loser with no special
	>       hardware (e.g., me) to check whether they've broken the
	>       block server?

Run two kvm VMs that share the same disk. Create an XFS filesystem on the MDS, and export it. If the client has blkmapd running (on Debian it needs to be started manually) it will use pNFS for accessing the filesystem. Verify that using the per-operation counters in /proc/self/mounstats. Repeat with additional clients as nessecary.

Quoted 5 lines

Alternatively set up a simple iSCSI target using tgt or lio and connect to it from multiple clients.

Which sounds reasonable to me, but I haven't tried to incorporate this into my regression testing yet.

Additionally I can offer the following script to generate recalls, which don't really happen during normal operation. I don't really know how to write a proper testcase that coordinates access to the exported filesystem and nfs unless it runs locally on the same node, though. It would need some higher level, network aware test harness:

----- snip -----
#!/bin/sh

set +x

# wait for grace period
touch /mnt/nfs1/foo

dd if=/dev/zero of=/mnt/nfs1/foo bs=128M count=32 conv=fdatasync oflag=direct &

sleep 2

echo "" > /mnt/test/foo && echo "recall done"

A message that is not in the archive (sent off-list or not yet received): [email protected]

#37 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 18 lines · Christoph Hellwig
Quoted 17 lines · J. Bruce Fields

Previously: http://lkml.kernel.org/r/[email protected]

	>       - any advice on testing?  Is there was some simple
	>       virtual setup that would allow any loser with no special
	>       hardware (e.g., me) to check whether they've broken the
	>       block server?

Run two kvm VMs that share the same disk. Create an XFS filesystem on the MDS, and export it. If the client has blkmapd running (on Debian it needs to be started manually) it will use pNFS for accessing the filesystem. Verify that using the per-operation counters in /proc/self/mounstats. Repeat with additional clients as nessecary.

Alternatively set up a simple iSCSI target using tgt or lio and connect to it from multiple clients.

Quoted 8 lines
Quoted 2 lines

Which sounds reasonable to me, but I haven't tried to incorporate this into my regression testing yet.

Additionally I can offer the following script to generate recalls, which don't really happen during normal operation. I don't really know how to write a proper testcase that coordinates access to the exported filesystem and nfs unless it runs locally on the same node, though. It would need some higher level, network aware test harness:

Thanks. I got as far as doing a quick manual test with vm's sharing a "disk":

	[root@f21-2]# mount -overs=4.1 f21-1:/exports/xfs-pnfs /mnt/
	[root@f21-2]# echo "hello world" >/mnt/testfile
	[root@f21-2]# grep LAYOUTGET /proc/self/mountstats 
		   LAYOUTGET: 1 1 0 236 196 0 4 4

I haven't tried to set up automated testing with recalls, but that shouldn't be hard.

--b.

Quoted 6 lines

----- snip -----
#!/bin/sh

set +x

Quoted 8 lines
# wait for grace period
touch /mnt/nfs1/foo

dd if=/dev/zero of=/mnt/nfs1/foo bs=128M count=32 conv=fdatasync oflag=direct &

sleep 2

echo "" > /mnt/test/foo && echo "recall done"

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#38 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 22 lines · J. Bruce Fields
Quoted 21 lines · Christoph Hellwig
Quoted 19 lines · J. Bruce Fields

Previously: http://lkml.kernel.org/r/[email protected]

	>       - any advice on testing?  Is there was some simple
	>       virtual setup that would allow any loser with no special
	>       hardware (e.g., me) to check whether they've broken the
	>       block server?

Run two kvm VMs that share the same disk. Create an XFS filesystem on the MDS, and export it. If the client has blkmapd running (on Debian it needs to be started manually) it will use pNFS for accessing the filesystem. Verify that using the per-operation counters in /proc/self/mounstats. Repeat with additional clients as nessecary.

Alternatively set up a simple iSCSI target using tgt or lio and connect to it from multiple clients.

Which sounds reasonable to me, but I haven't tried to incorporate this into my regression testing yet.

Quoted 8 lines
Quoted 5 lines

Additionally I can offer the following script to generate recalls, which don't really happen during normal operation. I don't really know how to write a proper testcase that coordinates access to the exported filesystem and nfs unless it runs locally on the same node, though. It would need some higher level, network aware test harness:

Thanks. I got as far as doing a quick manual test with vm's sharing a "disk":

Oh, also I forgot, on fedora:

	[root@f21-2]# systemctl enable nfs-blkmap.target
Quoted 24 lines
	[root@f21-2]# mount -overs=4.1 f21-1:/exports/xfs-pnfs /mnt/
	[root@f21-2]# echo "hello world" >/mnt/testfile
	[root@f21-2]# grep LAYOUTGET /proc/self/mountstats 
		   LAYOUTGET: 1 1 0 236 196 0 4 4

I haven't tried to set up automated testing with recalls, but that shouldn't be hard.

--b.

Quoted 14 lines

----- snip -----
#!/bin/sh

set +x

# wait for grace period
touch /mnt/nfs1/foo

dd if=/dev/zero of=/mnt/nfs1/foo bs=128M count=32 conv=fdatasync oflag=direct &

sleep 2

echo "" > /mnt/test/foo && echo "recall done"

#39 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

In reply to Dave Chinner
Quoted 5 lines · Dave Chinner

As to the pNFSD specific changes, I haven't really looked them over in any great detail yet. My main concern is that there are no specific regression tests for this yet, I'm not sure how we go about verifying it actually works properly and we don't inadvertantly break it in the future. Christoph?

Any chance you could review them this week so we can get them merged in time for 3.20? In the worst case Bruce will have to pull the xfs tree into the nfsd tree so that we have all the dependencies available.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#41 lore raw

Re: [PATCH 16/20] xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

Quoted 11 lines · Christoph Hellwig
Quoted 5 lines · Dave Chinner

As to the pNFSD specific changes, I haven't really looked them over in any great detail yet. My main concern is that there are no specific regression tests for this yet, I'm not sure how we go about verifying it actually works properly and we don't inadvertantly break it in the future. Christoph?

Any chance you could review them this week so we can get them merged in time for 3.20? In the worst case Bruce will have to pull the xfs tree into the nfsd tree so that we have all the dependencies available.

I'm working my way through them. I'm just about to pull in the growfs transaction patch (missed it last time around), and I try to have a decent look over the other two patches later today.

I'm not sure I have any bandwidth to test them yet, but perhaps if I add a one-time message "Experimental feature in use" when the code is first executed then it will be OK to merge (i.e. process similar to delayed logging and CRC introduction). Once we've got more confidence that it's all working properly, then we can remove the experimental tag from it. Does that sound like a reasonable approach to take?

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#42 lore raw

PATCH 17/20 xfs: update the superblock using a synchronous transaction in growfs

In reply to a message not in the archive

Growfs updates the secondary superblocks using synchronous unlogged buffer writes after committing the updates to the primary superblock.

Mark the transaction to the primary superblock as synchronous so that we guarantee it is committed to disk before we update the secondary superblocks.

---
 fs/xfs/xfs_fsops.c | 1 +
 1 file changed, 1 insertion(+)
fs/xfs/xfs_fsops.c +1 -0
index fdc6422..74c6211 100644
@@ -488,6 +488,7 @@ xfs_growfs_data_private(
 		xfs_trans_mod_sb(tp, XFS_TRANS_SB_FDBLOCKS, nfree);
 	if (dpct)
 		xfs_trans_mod_sb(tp, XFS_TRANS_SB_IMAXPCT, dpct);
+	xfs_trans_set_sync(tp);
 	error = xfs_trans_commit(tp, 0);
 	if (error)
 		return error;
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#43 lore raw

PATCH 19/20 xfs: implement pNFS export operations

In reply to a message not in the archive

Add operations to export pNFS block layouts from an XFS filesystem. See the previous commit adding the operations for an explanation of them.

---
 fs/xfs/Makefile     |   1 +
 fs/xfs/xfs_export.c |   6 ++
 fs/xfs/xfs_fsops.c  |   2 +
 fs/xfs/xfs_iops.c   |   2 +-
 fs/xfs/xfs_iops.h   |   1 +
 fs/xfs/xfs_mount.h  |  11 +++
 fs/xfs/xfs_pnfs.c   | 243 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_pnfs.h   |  11 +++
 8 files changed, 276 insertions(+), 1 deletion(-)
 create mode 100644 fs/xfs/xfs_pnfs.c
 create mode 100644 fs/xfs/xfs_pnfs.h
fs/xfs/Makefile +1 -0
index d617999..df68285 100644
@@ -121,3 +121,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL)	+= xfs_acl.o
 xfs-$(CONFIG_PROC_FS)		+= xfs_stats.o
 xfs-$(CONFIG_SYSCTL)		+= xfs_sysctl.o
 xfs-$(CONFIG_COMPAT)		+= xfs_ioctl32.o
+xfs-$(CONFIG_NFSD_PNFS)		+= xfs_pnfs.o
fs/xfs/xfs_export.c +6 -0
index 5eb4a14..b97359b 100644
@@ -30,6 +30,7 @@
 #include "xfs_trace.h"
 #include "xfs_icache.h"
 #include "xfs_log.h"
+#include "xfs_pnfs.h"
 
 /*
  * Note that we only accept fileids which are long enough rather than allow
@@ -245,4 +246,9 @@ const struct export_operations xfs_export_operations = {
 	.fh_to_parent		= xfs_fs_fh_to_parent,
 	.get_parent		= xfs_fs_get_parent,
 	.commit_metadata	= xfs_fs_nfs_commit_metadata,
+#ifdef CONFIG_NFSD_PNFS
+	.get_uuid		= xfs_fs_get_uuid,
+	.map_blocks		= xfs_fs_map_blocks,
+	.commit_blocks		= xfs_fs_commit_blocks,
+#endif
 };
fs/xfs/xfs_fsops.c +2 -0
index 74c6211..99465ba 100644
@@ -602,6 +602,8 @@ xfs_growfs_data(
 	if (!mutex_trylock(&mp->m_growlock))
 		return -EWOULDBLOCK;
 	error = xfs_growfs_data_private(mp, in);
+	if (!error)
+		mp->m_generation++;
 	mutex_unlock(&mp->m_growlock);
 	return error;
 }
fs/xfs/xfs_iops.c +1 -1
index c50311c..6ff84e8 100644
@@ -496,7 +496,7 @@ xfs_setattr_mode(
 	inode->i_mode |= mode & ~S_IFMT;
 }
 
-static void
+void
 xfs_setattr_time(
 	struct xfs_inode	*ip,
 	struct iattr		*iattr)
fs/xfs/xfs_iops.h +1 -0
index 1c34e43..ea7a98e 100644
@@ -32,6 +32,7 @@ extern void xfs_setup_inode(struct xfs_inode *);
  */
 #define XFS_ATTR_NOACL		0x01	/* Don't call posix_acl_chmod */
 
+extern void xfs_setattr_time(struct xfs_inode *ip, struct iattr *iattr);
 extern int xfs_setattr_nonsize(struct xfs_inode *ip, struct iattr *vap,
 			       int flags);
 extern int xfs_setattr_size(struct xfs_inode *ip, struct iattr *vap);
fs/xfs/xfs_mount.h +11 -0
index 22ccf69..12925d5 100644
@@ -175,6 +175,17 @@ typedef struct xfs_mount {
 	struct workqueue_struct	*m_reclaim_workqueue;
 	struct workqueue_struct	*m_log_workqueue;
 	struct workqueue_struct *m_eofblocks_workqueue;
+
+	/*
+	 * Generation of the filesysyem layout.  This is incremented by each
+	 * growfs, and used by the pNFS server to ensure the client updates
+	 * its view of the block device once it gets a layout that might
+	 * reference the newly added blocks.  Does not need to be persistent
+	 * as long as we only allow file system size increments, but if we
+	 * ever support shrinks it would have to be persisted in addition
+	 * to various other kinds of pain inflicted on the pNFS server.
+	 */
+	__uint32_t		m_generation;
 } xfs_mount_t;
 
 /*
fs/xfs/xfs_pnfs.c new file +243 -0

243 lines not on this page · raw message

fs/xfs/xfs_pnfs.h new file +11 -0
new file mode 100644
index 0000000..0d91255
@@ -0,0 +1,11 @@
+#ifndef _XFS_PNFS_H
+#define _XFS_PNFS_H 1
+
+#ifdef CONFIG_NFSD_PNFS
+int xfs_fs_get_uuid(struct super_block *sb, u8 *buf, u32 *len, u64 *offset);
+int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
+		struct iomap *iomap, bool write, u32 *device_generation);
+int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
+		struct iattr *iattr);
+#endif /* CONFIG_NFSD_PNFS */
+#endif /* _XFS_PNFS_H */
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-20-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#45 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 4 lines · Christoph Hellwig

Add operations to export pNFS block layouts from an XFS filesystem. See the previous commit adding the operations for an explanation of them.

Note that I haven't applied this patch, or attempted to compile it yet....

fs/xfs/Makefile xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o · line 124

Quoted 9 lines
fs/xfs/Makefile +1 -0
index d617999..df68285 100644
@@ -121,3 +121,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL)	+= xfs_acl.o
 xfs-$(CONFIG_PROC_FS)		+= xfs_stats.o
 xfs-$(CONFIG_SYSCTL)		+= xfs_sysctl.o
 xfs-$(CONFIG_COMPAT)		+= xfs_ioctl32.o
+xfs-$(CONFIG_NFSD_PNFS)		+= xfs_pnfs.o

... because I'll have to jump through hoops to get it to compile, I think.

fs/xfs/xfs_fsops.c xfs_growfs_data( · line 609

Quoted 13 lines
fs/xfs/xfs_fsops.c +2 -0
index 74c6211..99465ba 100644
@@ -602,6 +602,8 @@ xfs_growfs_data(
 	if (!mutex_trylock(&mp->m_growlock))
 		return -EWOULDBLOCK;
 	error = xfs_growfs_data_private(mp, in);
+	if (!error)
+		mp->m_generation++;
 	mutex_unlock(&mp->m_growlock);
 	return error;
 }

Even on error I think we should bump this. Errors can come from secondary superblock updates after the filesystem has been grown, hence an error is not a reliable indication of whether the layout has changed or not.

Quoted 5 lines
fs/xfs/xfs_fsops.c +5 -0
+int
+xfs_fs_get_uuid(
+	struct super_block	*sb,
+	u8			*buf,
+	u32			*len,

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+	u64			*offset)
+{
+	struct xfs_mount	*mp = XFS_M(sb);
+
+	if (*len < sizeof(uuid_t))
+		return -EINVAL;
+
+	memcpy(buf, &mp->m_sb.sb_uuid, sizeof(uuid_t));
+	*len = sizeof(uuid_t);
+	*offset = offsetof(struct xfs_dsb, sb_uuid);

What purpose does the offset serve here? I can't tell from the usage in the PNFS code. Can we ignore offset - as it seems entirely arbitrary - and still have this work? Either way, comment please.

Quoted 4 lines
fs/xfs/xfs_fsops.c +4 -0
+static void
+xfs_bmbt_to_iomap(
+	struct xfs_inode	*ip,
+	struct iomap		*iomap,

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+	struct xfs_bmbt_irec	*imap)
+{
+	struct xfs_mount	*mp = ip->i_mount;
+
+	if (imap->br_startblock == HOLESTARTBLOCK) {
+		iomap->blkno = -1;
+		iomap->type = IOMAP_HOLE;
+	} else if (imap->br_startblock == DELAYSTARTBLOCK) {
+		iomap->blkno = -1;
+		iomap->type = IOMAP_DELALLOC;

I'd like to see a IOMAP_NULL_BLOCK define here for the -1 value, say:

#define IOMAP_NULL_BLOCK -1LL

Quoted 13 lines
fs/xfs/xfs_fsops.c +13 -0
+int
+xfs_fs_map_blocks(
+	struct inode		*inode,
+	loff_t			offset,
+	u64			length,
+	struct iomap		*iomap,
+	bool			write,
+	u32			*device_generation)
+{
+	struct xfs_inode	*ip = XFS_I(inode);
+	struct xfs_mount	*mp = ip->i_mount;
+	struct xfs_bmbt_irec	imap;
+	xfs_fileoff_t		offset_fsb, end_fsb;

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+	loff_t			limit;
+	int			bmapi_flags = XFS_BMAPI_ENTIRE;
+	int			nimaps = 1;
+	uint			lock_flags;
+	int			error = 0;
+
+	if (XFS_FORCED_SHUTDOWN(mp))
+		return -EIO;
+	if (XFS_IS_REALTIME_INODE(ip))
+		return -ENXIO;

OK, so we are not mapping realtime inodes here. Any specific reason? FWIW, that also means you can use XFS_FSB_TO_DADDR() in the iomap mapping as xfs_fsb_to_db() is only needed if we might be mapping realtime extents...

Quoted 5 lines
fs/xfs/xfs_fsops.c +5 -0
+	for (i = 0; i < nr_maps; i++) {
+		u64 start, length, end;
+
+		start = maps[i].offset;
+		if (start > size)

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+			continue;
+
+		end = start + maps[i].length;
+		if (end > size)
+			end = size;
+
+		length = end - start;
+		if (!length)
+			continue;
+	

^^^^^
Stray whitespace

Quoted 6 lines
fs/xfs/xfs_fsops.c +6 -0
+	tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
+	error = xfs_trans_reserve(tp, &M_RES(mp)->tr_ichange, 0, 0);
+	if (error)
+		goto out_drop_iolock;
+
+	xfs_ilock(ip, XFS_ILOCK_EXCL);

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+	xfs_setattr_time(ip, iattr);
+	if (iattr->ia_valid & ATTR_SIZE) {
+		if (iattr->ia_size > i_size_read(inode)) {
+			i_size_write(inode, iattr->ia_size);
+			ip->i_d.di_size = iattr->ia_size;
+		}
+	}

The concern I have about this is that extending the file size can expose uninitialised blocks beyond the old EOF. That can happen if delayed allocation has previously been done on the file and we haven't trimmed the excess beyond EOF back yet. I know the pnfs server is not aimed at mixed usage, but it still makes me uncomfortable in the case where you have normal NFS and PNFS clients accessing the same files...

fs/xfs/xfs_fsops.c

Quoted 2 lines
fs/xfs/xfs_fsops.c +2 -0
+	xfs_trans_set_sync(tp);
+	error = xfs_trans_commit(tp, 0);

I just had a thought about these sync transctions - could the NFS server handle persistence of the maps via ->commit_metadata?

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#46 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 9 lines · Dave Chinner

fs/xfs/Makefile xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o · line 124

Quoted 6 lines

+++ b/fs/xfs/Makefile

fs/xfs/Makefile +1 -0
@@ -121,3 +121,4 @@ xfs-$(CONFIG_XFS_POSIX_ACL)	+= xfs_acl.o
 xfs-$(CONFIG_PROC_FS)		+= xfs_stats.o
 xfs-$(CONFIG_SYSCTL)		+= xfs_sysctl.o
 xfs-$(CONFIG_COMPAT)		+= xfs_ioctl32.o
+xfs-$(CONFIG_NFSD_PNFS)		+= xfs_pnfs.o

... because I'll have to jump through hoops to get it to compile, I think.

Just get the whole git tree from

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-3
Quoted 10 lines
Quoted 10 lines
fs/xfs/xfs_fsops.c +2 -0
index 74c6211..99465ba 100644
@@ -602,6 +602,8 @@ xfs_growfs_data(
 	if (!mutex_trylock(&mp->m_growlock))
 		return -EWOULDBLOCK;
 	error = xfs_growfs_data_private(mp, in);
+	if (!error)
+		mp->m_generation++;
Quoted 8 lines

fs/xfs/xfs_fsops.c xfs_growfs_data( · line 609

Quoted 3 lines
fs/xfs/xfs_fsops.c +0 -0
 	mutex_unlock(&mp->m_growlock);
 	return error;
 }

Even on error I think we should bump this. Errors can come from secondary superblock updates after the filesystem has been grown, hence an error is not a reliable indication of whether the layout has changed or not.

Ok.

Quoted 12 lines
Quoted 12 lines
fs/xfs/xfs_fsops.c +11 -0
+int
+xfs_fs_get_uuid(
+	struct super_block	*sb,
+	u8			*buf,
+	u32			*len,
+	u64			*offset)
+{
+	struct xfs_mount	*mp = XFS_M(sb);
+
+	if (*len < sizeof(uuid_t))
+		return -EINVAL;
Quoted 8 lines

fs/xfs/xfs_fsops.c

Quoted 4 lines
fs/xfs/xfs_fsops.c +4 -0
+
+	memcpy(buf, &mp->m_sb.sb_uuid, sizeof(uuid_t));
+	*len = sizeof(uuid_t);
+	*offset = offsetof(struct xfs_dsb, sb_uuid);

What purpose does the offset serve here? I can't tell from the usage in the PNFS code. Can we ignore offset - as it seems entirely arbitrary - and still have this work? Either way, comment please.

The get_uuid methods gets content and location of the uuid so that the client can find the disks. The offset simply is part of the wire protocol.

Quoted 4 lines

I'd like to see a IOMAP_NULL_BLOCK define here for the -1 value, say:

#define IOMAP_NULL_BLOCK -1LL

Ok.

Quoted 1 line

OK, so we are not mapping realtime inodes here. Any specific reason?

The realtime device doesn't have a way for the client to find it, as it doesn't have its own superblockuuids.

Quoted 3 lines

FWIW, that also means you can use XFS_FSB_TO_DADDR() in the iomap mapping as xfs_fsb_to_db() is only needed if we might be mapping realtime extents...

ok.

Quoted 17 lines
Quoted 6 lines
fs/xfs/xfs_fsops.c +6 -0
+	tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
+	error = xfs_trans_reserve(tp, &M_RES(mp)->tr_ichange, 0, 0);
+	if (error)
+		goto out_drop_iolock;
+
+	xfs_ilock(ip, XFS_ILOCK_EXCL);

fs/xfs/xfs_fsops.c

Quoted 10 lines
fs/xfs/xfs_fsops.c +10 -0
+	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+	xfs_setattr_time(ip, iattr);
+	if (iattr->ia_valid & ATTR_SIZE) {
+		if (iattr->ia_size > i_size_read(inode)) {
+			i_size_write(inode, iattr->ia_size);
+			ip->i_d.di_size = iattr->ia_size;
+		}
+	}
Quoted 7 lines

The concern I have about this is that extending the file size can expose uninitialised blocks beyond the old EOF. That can happen if delayed allocation has previously been done on the file and we haven't trimmed the excess beyond EOF back yet. I know the pnfs server is not aimed at mixed usage, but it still makes me uncomfortable in the case where you have normal NFS and PNFS clients accessing the same files...

The protocol only allows to commit to a size that we previously returned a layout for, which means we already have allocated space for it at same time. For robustness reasons a sanity check might make sense, though.

Quoted 5 lines

fs/xfs/xfs_fsops.c

Quoted 2 lines
fs/xfs/xfs_fsops.c +2 -0
+	xfs_trans_set_sync(tp);
+	error = xfs_trans_commit(tp, 0);

I just had a thought about these sync transctions - could the NFS server handle persistence of the maps via ->commit_metadata?

It probably could, but that wouldn't buy us anything over just committing the transaction synchronously.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#48 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

I've updated the patch and pushed out a new pnfsd-for-3.20-4 branch.

The changes relative to the old one are below:

fs/xfs/xfs_fsops.c +6 -2
index 99465ba..48561a0 100644
@@ -602,8 +602,12 @@ xfs_growfs_data(
 	if (!mutex_trylock(&mp->m_growlock))
 		return -EWOULDBLOCK;
 	error = xfs_growfs_data_private(mp, in);
-	if (!error)
-		mp->m_generation++;
+	/*
+	 * Increment the generation unconditionally, the error could be from
+	 * updating the secondary superblocks, in which case the new size
+	 * is live already.
+	 */
+	mp->m_generation++;
 	mutex_unlock(&mp->m_growlock);
 	return error;
 }
fs/xfs/xfs_pnfs.c +58 -9

132 lines not on this page · raw message

include/linux/exportfs.h +2 -0
index ff46bf7..fa05e04 100644
@@ -187,6 +187,8 @@ struct fid {
 #define IOMAP_MAPPED	0x03	/* blocks allocated @blkno */
 #define IOMAP_UNWRITTEN	0x04	/* blocks allocated @blkno in unwritten state */
 
+#define IOMAP_NULL_BLOCK -1LL	/* blkno is not valid */
+
 struct iomap {
 	sector_t	blkno;	/* first sector of mapping */
 	loff_t		offset;	/* file offset of mapping, bytes */

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Acked-by
#50 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 3 lines · Christoph Hellwig

I've updated the patch and pushed out a new pnfsd-for-3.20-4 branch.

The changes relative to the old one are below:

Hi Christoph, with these changes I think this is fine to be merged with the experimental tag attached to it

I'm expecting the merge window to open on Monday so it's kinda late to be adding new stuff to the XFS tree and co-ordinating it with the NFS tree merge - how were you planning to get this to merged?

I've already merged all but the two pNFSD support patches, so there's some duplicate commits in your pnfsd-for-3.20-4 branch. i.e. these commits in your tree:

b8d5187 xfs: factor out a xfs_update_prealloc_flags() helper 6d5ca2a xfs: update the superblock using a synchronous transaction in growfs e3ea93e xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

are already merged into the xfs for-next branch as:

8add71c xfs: factor out a xfs_update_prealloc_flags() helper f8079b8 xfs: growfs should use synchronous transactions d32057f xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

A straight merge from your tree ends up with both sets of commits in the history. So a rebase on your side, or me pulling them into the XFS tree is probably required to keep the history clean.

I didn't really want to add any more to the XFS tree this close to the merge window opening, but I've already got a regression fix that needs to be added, so perhaps I'll delay sending Linus a pull request for a week and just merge all of these XFS changes directly.

What do you think?

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#51 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 32 lines · Dave Chinner
Quoted 3 lines · Christoph Hellwig

I've updated the patch and pushed out a new pnfsd-for-3.20-4 branch.

The changes relative to the old one are below:

Hi Christoph, with these changes I think this is fine to be merged with the experimental tag attached to it

I'm expecting the merge window to open on Monday so it's kinda late to be adding new stuff to the XFS tree and co-ordinating it with the NFS tree merge - how were you planning to get this to merged?

I've already merged all but the two pNFSD support patches, so there's some duplicate commits in your pnfsd-for-3.20-4 branch. i.e. these commits in your tree:

b8d5187 xfs: factor out a xfs_update_prealloc_flags() helper 6d5ca2a xfs: update the superblock using a synchronous transaction in growfs e3ea93e xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

are already merged into the xfs for-next branch as:

8add71c xfs: factor out a xfs_update_prealloc_flags() helper f8079b8 xfs: growfs should use synchronous transactions d32057f xfs: pass a 64-bit count argument to xfs_iomap_write_unwritten

A straight merge from your tree ends up with both sets of commits in the history. So a rebase on your side, or me pulling them into the XFS tree is probably required to keep the history clean.

Quoted 6 lines

I didn't really want to add any more to the XFS tree this close to the merge window opening, but I've already got a regression fix that needs to be added, so perhaps I'll delay sending Linus a pull request for a week and just merge all of these XFS changes directly.

What do you think?

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

--b.

author reviewer
#52 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#54 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 17 lines · On Sun, 8 Feb 2015 14:34:35 +0100 Christoph Hellwig…
Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Quoted 7 lines

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

FWIW, I plan to send a pull request for the locking changes as soon as the merge window opens. Hopefully that won't be an issue for long...

Signature · 5 lines
Jeff Layton <jeff.layton-7I+n7zu2hftEKMMhf/[email protected]>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20150208090942.51e99687-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org

#56 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 24 lines · Jeff Layton
Quoted 21 lines · On Sun, 8 Feb 2015 14:34:35 +0100 Christoph Hellwig…
Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Quoted 6 lines
Quoted 3 lines

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

FWIW, I plan to send a pull request for the locking changes as soon as the merge window opens. Hopefully that won't be an issue for long...

This includes Christoph's branch (all but the final xfs commits):

	git://linux-nfs.org/~bfields/linux.git for-3.20

That's what I intend to submit. Hope that's OK. Then it's up to Dave whether he wants to pull that in and include the xfs patches.

Worst case we end up with not-yet-usable pnfs code in 3.20, which wouldn't be ideal but shouldn't cause any serious problem either.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#58 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 32 lines · J. Bruce Fields
Quoted 24 lines · Jeff Layton
Quoted 21 lines · On Sun, 8 Feb 2015 14:34:35 +0100 Christoph Hellwig…
Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Quoted 6 lines
Quoted 3 lines

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

FWIW, I plan to send a pull request for the locking changes as soon as the merge window opens. Hopefully that won't be an issue for long...

Quoted 6 lines

This includes Christoph's branch (all but the final xfs commits):

	git://linux-nfs.org/~bfields/linux.git for-3.20

That's what I intend to submit. Hope that's OK. Then it's up to Dave whether he wants to pull that in and include the xfs patches.

I'm about to send a pull request to Linus for the current XFS tree. Once that is merged, I'll pull in the remaining xfs-pnfs patches and send another pull request to Linus after the NFS tree is merged.

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#59 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 35 lines · Dave Chinner
Quoted 34 lines · J. Bruce Fields
Quoted 24 lines · Jeff Layton
Quoted 21 lines · On Sun, 8 Feb 2015 14:34:35 +0100 Christoph Hellwig…
Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Quoted 6 lines
Quoted 3 lines

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

FWIW, I plan to send a pull request for the locking changes as soon as the merge window opens. Hopefully that won't be an issue for long...

This includes Christoph's branch (all but the final xfs commits):

Quoted 8 lines
Quoted 4 lines
	git://linux-nfs.org/~bfields/linux.git for-3.20

That's what I intend to submit. Hope that's OK. Then it's up to Dave whether he wants to pull that in and include the xfs patches.

I'm about to send a pull request to Linus for the current XFS tree. Once that is merged, I'll pull in the remaining xfs-pnfs patches and send another pull request to Linus after the NFS tree is merged.

Sounds good, thanks. The nfsd tree's merged now so it should be good to go if you haven't found any show-stoppers.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#60 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 41 lines · J. Bruce Fields
Quoted 40 lines · Dave Chinner
Quoted 38 lines · J. Bruce Fields
Quoted 24 lines · Jeff Layton
Quoted 21 lines · On Sun, 8 Feb 2015 14:34:35 +0100 Christoph Hellwig <[email protected]> wrote:
Quoted 9 lines · J. Bruce Fields

You'd basically just be pulling my tree (Christoph's is just my nfsd tree with his patches on top, and I've been testing with exactly that locally, just putting off pushing it out till we decide this.)

So anyway, fine with me if you want to just pull that into the xfs tree. Mine's ready whenever, so if I send my pull pretty soon after the merge window and you send it a little later then we still keep the property that Linus's merge still has a diffstat only in our respective areas.

Quoted 6 lines

(OK, it's a little more complicated because I've got the same arrangement with jlayton, so the order is jlayton's lock pull, then my nfsd pull, then your xfs pull. Is this getting too complicated? jlayton and I are both ready to so and I think it'd work.)

I'm also fine with duplicating those few patches, or whatever.

Maybe the better idea is to pull the xfs tree in the nfsd tree, but that would require Dave sending an early pull request so that the nfsd pull doesn't get delayed.

Quoted 6 lines
Quoted 3 lines

Or we just defer the pnfsd merge. While I tried to get it in in time for 3.20 all the delays during review mean we're really late no and should punt it to 3.21.

FWIW, I plan to send a pull request for the locking changes as soon as the merge window opens. Hopefully that won't be an issue for long...

This includes Christoph's branch (all but the final xfs commits):

	git://linux-nfs.org/~bfields/linux.git for-3.20

That's what I intend to submit. Hope that's OK. Then it's up to Dave whether he wants to pull that in and include the xfs patches.

Quoted 6 lines
Quoted 3 lines

I'm about to send a pull request to Linus for the current XFS tree. Once that is merged, I'll pull in the remaining xfs-pnfs patches and send another pull request to Linus after the NFS tree is merged.

Sounds good, thanks. The nfsd tree's merged now so it should be good to go if you haven't found any show-stoppers.

Thanks Bruce. I might have to build a merged tree because one of the changes from the review modified a header file introduced in the NFS tree.

I'll see how it goes, and see if I can avoid doing something that will make Linus yell at me :P

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#61 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Hi Dave,

Quoted 7 lines · On Fri, 13 Feb 2015 12:54:22 +1100 Dave Chinner <[email protected]> wrote:

Thanks Bruce. I might have to build a merged tree because one of the changes from the review modified a header file introduced in the NFS tree.

I'll see how it goes, and see if I can avoid doing something that will make Linus yell at me :P

If its a syntactic conflict with an obvious resolution, just leave it (or maybe mention it in the pull request). It gives him something to do so he feels useful. :-)

/me notes that I did not see a conflict while merging the xfs tree in linux-next today ...

Signature · 2 lines
Cheers,
Stephen Rothwell                    [email protected]
#62 lore raw

Re: [PATCH 19/20] xfs: implement pNFS export operations

Quoted 8 lines · Stephen Rothwell

Hi Dave,

Quoted 5 lines · On Fri, 13 Feb 2015 12:54:22 +1100 Dave Chinner <[email protected]> wrote:

Thanks Bruce. I might have to build a merged tree because one of the changes from the review modified a header file introduced in the NFS tree.

Quoted 6 lines
Quoted 2 lines

I'll see how it goes, and see if I can avoid doing something that will make Linus yell at me :P

If its a syntactic conflict with an obvious resolution, just leave it (or maybe mention it in the pull request). It gives him something to do so he feels useful. :-)

Heh.

It's not actually a conflict, though. The issue is that some of the review comments on the XFS side resulted in changes files introduced from the NFS tree. So the XFS side can't be merged without the NFS changes being present....

Quoted 2 lines

/me notes that I did not see a conflict while merging the xfs tree in linux-next today ...

Because I haven't pushed them yet ;)

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#63 lore raw

PATCH 20/20 xfs: recall pNFS layouts on conflicting access

In reply to a message not in the archive

Recall all outstanding pNFS layouts and truncates, writes and similar extent list modifying operations.

---
 fs/xfs/xfs_file.c  | 14 ++++++++++++--
 fs/xfs/xfs_ioctl.c |  9 +++++++--
 fs/xfs/xfs_iops.c  | 11 ++++++++---
 fs/xfs/xfs_pnfs.c  | 31 +++++++++++++++++++++++++++++++
 fs/xfs/xfs_pnfs.h  |  7 +++++++
 5 files changed, 65 insertions(+), 7 deletions(-)
fs/xfs/xfs_file.c +12 -2
index 712d312..56dcfce 100644
@@ -36,6 +36,7 @@
 #include "xfs_trace.h"
 #include "xfs_log.h"
 #include "xfs_icache.h"
+#include "xfs_pnfs.h"
 
 #include <linux/aio.h>
 #include <linux/dcache.h>
@@ -554,6 +555,10 @@ restart:
 	if (error)
 		return error;
 
+	error = xfs_break_layouts(inode, iolock);
+	if (error)
+		return error;
+
 	/*
 	 * If the offset is beyond the size of the file, we need to zero any
 	 * blocks that fall between the existing EOF and the start of this
@@ -822,6 +827,7 @@ xfs_file_fallocate(
 	struct xfs_inode	*ip = XFS_I(inode);
 	long			error;
 	enum xfs_prealloc_flags	flags = 0;
+	uint			iolock = XFS_IOLOCK_EXCL;
 	loff_t			new_size = 0;
 
 	if (!S_ISREG(inode->i_mode))
@@ -830,7 +836,11 @@ xfs_file_fallocate(
 		     FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_ZERO_RANGE))
 		return -EOPNOTSUPP;
 
-	xfs_ilock(ip, XFS_IOLOCK_EXCL);
+	xfs_ilock(ip, iolock);
+	error = xfs_break_layouts(inode, &iolock);
+	if (error)
+		goto out_unlock;
+
 	if (mode & FALLOC_FL_PUNCH_HOLE) {
 		error = xfs_free_file_space(ip, offset, len);
 		if (error)
@@ -894,7 +904,7 @@ xfs_file_fallocate(
 	}
 
 out_unlock:
-	xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+	xfs_iunlock(ip, iolock);
 	return error;
 }
 
fs/xfs/xfs_ioctl.c +7 -2
index d58bcd2..0b64310 100644
@@ -39,6 +39,7 @@
 #include "xfs_icache.h"
 #include "xfs_symlink.h"
 #include "xfs_trans.h"
+#include "xfs_pnfs.h"
 
 #include <linux/capability.h>
 #include <linux/dcache.h>
@@ -608,6 +609,7 @@ xfs_ioc_space(
 {
 	struct iattr		iattr;
 	enum xfs_prealloc_flags	flags = 0;
+	uint			iolock = XFS_IOLOCK_EXCL;
 	int			error;
 
 	/*
@@ -636,7 +638,10 @@ xfs_ioc_space(
 	if (error)
 		return error;
 
-	xfs_ilock(ip, XFS_IOLOCK_EXCL);
+	xfs_ilock(ip, iolock);
+	error = xfs_break_layouts(inode, &iolock);
+	if (error)
+		goto out_unlock;
 
 	switch (bf->l_whence) {
 	case 0: /*SEEK_SET*/
@@ -725,7 +730,7 @@ xfs_ioc_space(
 	error = xfs_update_prealloc_flags(ip, flags);
 
 out_unlock:
-	xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+	xfs_iunlock(ip, iolock);
 	mnt_drop_write_file(filp);
 	return error;
 }
fs/xfs/xfs_iops.c +8 -3
index 6ff84e8..b1e849a 100644
@@ -37,6 +37,7 @@
 #include "xfs_da_btree.h"
 #include "xfs_dir2.h"
 #include "xfs_trans_space.h"
+#include "xfs_pnfs.h"
 
 #include <linux/capability.h>
 #include <linux/xattr.h>
@@ -970,9 +971,13 @@ xfs_vn_setattr(
 	int			error;
 
 	if (iattr->ia_valid & ATTR_SIZE) {
-		xfs_ilock(ip, XFS_IOLOCK_EXCL);
-		error = xfs_setattr_size(ip, iattr);
-		xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+		uint		iolock = XFS_IOLOCK_EXCL;
+
+		xfs_ilock(ip, iolock);
+		error = xfs_break_layouts(dentry->d_inode, &iolock);
+		if (!error)
+			error = xfs_setattr_size(ip, iattr);
+		xfs_iunlock(ip, iolock);
 	} else {
 		error = xfs_setattr_nonsize(ip, iattr, 0);
 	}
fs/xfs/xfs_pnfs.c +31 -0
index 5d25f5d..f2bbc2f 100644
@@ -17,6 +17,37 @@
 #include "xfs_shared.h"
 #include "xfs_pnfs.h"
 
+/*
+ * Ensure that we do not have any outstanding pNFS layouts that can be
+ * used by clients to directly read from or write to this inode.
+ * This must be called before every operation that can remove blocks
+ * from the extent map.  Additionally we call it during the write
+ * operation, where aren't concerned about exposing unallocated blocks
+ * but just want to provide basic synchronization between a local
+ * writer and pNFS clients.  mmap writes would also benefit from
+ * this sort of synchronization, but due to the tricky locking
+ * rules in the page fault path we don't bother.
+ */
+int
+xfs_break_layouts(
+	struct inode		*inode,
+	uint			*iolock)
+{
+	struct xfs_inode	*ip = XFS_I(inode);
+	int			error;
+
+	ASSERT(xfs_isilocked(ip, XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL));
+
+	while ((error = break_layout(inode, false) == -EWOULDBLOCK)) {
+		xfs_iunlock(ip, *iolock);
+		error = break_layout(inode, true);
+		*iolock = XFS_IOLOCK_EXCL;
+		xfs_ilock(ip, *iolock);
+	}
+
+	return error;
+}
+
 int
 xfs_fs_get_uuid(
 	struct super_block	*sb,
fs/xfs/xfs_pnfs.h +7 -0
index 0d91255..b7fbfce 100644
@@ -7,5 +7,12 @@ int xfs_fs_map_blocks(struct inode *inode, loff_t offset, u64 length,
 		struct iomap *iomap, bool write, u32 *device_generation);
 int xfs_fs_commit_blocks(struct inode *inode, struct iomap *maps, int nr_maps,
 		struct iattr *iattr);
+
+int xfs_break_layouts(struct inode *inode, uint *iolock);
+#else
+static inline int xfs_break_layouts(struct inode *inode, uint *iolock)
+{
+	return 0;
+}
 #endif /* CONFIG_NFSD_PNFS */
 #endif /* _XFS_PNFS_H */
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-21-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#65 lore raw

Re: [PATCH 20/20] xfs: recall pNFS layouts on conflicting access

Quoted 4 lines · Christoph Hellwig

Recall all outstanding pNFS layouts and truncates, writes and similar extent list modifying operations.

This looks fine, assuming break_layout() doesn't require any other VFS inode locks to be held for serialisation.

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#66 lore raw
In reply to a message not in the archive
Quoted 16 lines · Christoph Hellwig

This series adds support for the pNFS operations in NFS v4.1, as well as a block layout driver that can export block based filesystems that implement a few additional export operations. Support for XFS is provided in this series, but other filesystems could be added easily.

The core pNFS code of course owns its heritage to the existing Linux pNFS server prototype, but except for a few bits and pieces in the XDR path nothing is left from it.

The design of this new pNFS server is fairly different from the old one - while the old one implemented very little semantics in nfsd and left almost everything to filesystems my implementation implements as much as possible in common nfsd code, then dispatches to a layout driver that still is part of nfsd and only then calls into the filesystem, thus keeping it free from intimate pNFS knowledge.

Quoted 8 lines

This version of the code has been rebased to the locks-for-3.20 tree which adds a new lock context structure to the inode. For now we still (ab)use the lease list like in the last version. Adding a pNFS-specific list would duplicate a lot of code without much benefit. But during this research I came up with way to associate a nfs4_file with a struct file at open time which should allow to greatly simplify the pNFS and delegation code. So stay tuned for some patches in this area!

Good, that could definitely use simplification.--b.

Quoted 14 lines

For now this also doesn't take errata 3901 for rfc 5661 into account yet and sticks to the verified errata. The changes in 3901 seem useful in the longer run one verified and will be implemented eventually. Note that the only existing pNFS block client (Linux) would not benefit from the longer layout lifetimes anyway.

More details are document in the individual patch descriptions and code comments.

This code is also available from:

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-2
Quoted 8 lines

Changes since V1:
- rebased to the locks-for-3.20 tree
- fixed a memory leak in the nfsd4_encode_getdeviceinfo error path
- removed the always one lg_roc field
- added a nopnfs export option
- use a synchronous transaction in layoutget to simplify recovery
- various XFS fixes pointed out by Dave
- various documentation typo fixes

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#67 lore raw
In reply to a message not in the archive
Quoted 16 lines · Christoph Hellwig

This series adds support for the pNFS operations in NFS v4.1, as well as a block layout driver that can export block based filesystems that implement a few additional export operations. Support for XFS is provided in this series, but other filesystems could be added easily.

The core pNFS code of course owns its heritage to the existing Linux pNFS server prototype, but except for a few bits and pieces in the XDR path nothing is left from it.

The design of this new pNFS server is fairly different from the old one - while the old one implemented very little semantics in nfsd and left almost everything to filesystems my implementation implements as much as possible in common nfsd code, then dispatches to a layout driver that still is part of nfsd and only then calls into the filesystem, thus keeping it free from intimate pNFS knowledge.

Quoted 2 lines

This version of the code has been rebased to the locks-for-3.20 tree which adds a new lock context structure to the inode.

By the way, Jeff, do you consider the lock changes are ready? Would you mind if I pulled or cherry-picked them into the nfsd tree just to simplify merging this stuff for 3.20? (Assuming it's going to be ready.)

--b.

Quoted 21 lines

For now
we still (ab)use the lease list like in the last version. Adding a pNFS-specific list would duplicate a lot of code without much benefit. But during this research I came up with way to associate a nfs4_file with a struct file at open time which should allow to greatly simplify the pNFS and delegation code. So stay tuned for some patches in this area!

For now this also doesn't take errata 3901 for rfc 5661 into account yet and sticks to the verified errata. The changes in 3901 seem useful in the longer run one verified and will be implemented eventually. Note that the only existing pNFS block client (Linux) would not benefit from the longer layout lifetimes anyway.

More details are document in the individual patch descriptions and code comments.

This code is also available from:

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-2
Quoted 8 lines

Changes since V1:
- rebased to the locks-for-3.20 tree
- fixed a memory leak in the nfsd4_encode_getdeviceinfo error path
- removed the always one lg_roc field
- added a nopnfs export option
- use a synchronous transaction in layoutget to simplify recovery
- various XFS fixes pointed out by Dave
- various documentation typo fixes

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author reviewer
#68 lore raw
Quoted 4 lines · J. Bruce Fields

By the way, Jeff, do you consider the lock changes are ready? Would you mind if I pulled or cherry-picked them into the nfsd tree just to simplify merging this stuff for 3.20? (Assuming it's going to be ready.)

I did the rebase because he mentioned he wanted to get it into 3.20.

Note that the conflicts were fairly trivial, so it's something we could just ignore and let Linus handle the merge in the worst case.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#70 lore raw
Quoted 20 lines · On Thu, 22 Jan 2015 15:06:18 -0500 "J. Bruce Fields"…
Quoted 16 lines · Christoph Hellwig

This series adds support for the pNFS operations in NFS v4.1, as well as a block layout driver that can export block based filesystems that implement a few additional export operations. Support for XFS is provided in this series, but other filesystems could be added easily.

The core pNFS code of course owns its heritage to the existing Linux pNFS server prototype, but except for a few bits and pieces in the XDR path nothing is left from it.

The design of this new pNFS server is fairly different from the old one - while the old one implemented very little semantics in nfsd and left almost everything to filesystems my implementation implements as much as possible in common nfsd code, then dispatches to a layout driver that still is part of nfsd and only then calls into the filesystem, thus keeping it free from intimate pNFS knowledge.

Quoted 2 lines

This version of the code has been rebased to the locks-for-3.20 tree which adds a new lock context structure to the inode.

Quoted 7 lines

By the way, Jeff, do you consider the lock changes are ready? Would you mind if I pulled or cherry-picked them into the nfsd tree just to simplify merging this stuff for 3.20? (Assuming it's going to be ready.)

--b.

Sure, that'd be fine.

The latest version of that set seems to be OK (tip commit == 8116bf4cb62d33) and has been sitting in linux-next for a while now. I'll let you know if I get any reports of problems in that area between now and the merge window though.

Thanks,
Jeff

Quoted 29 lines
Quoted 29 lines

For now
we still (ab)use the lease list like in the last version. Adding a pNFS-specific list would duplicate a lot of code without much benefit. But during this research I came up with way to associate a nfs4_file with a struct file at open time which should allow to greatly simplify the pNFS and delegation code. So stay tuned for some patches in this area!

For now this also doesn't take errata 3901 for rfc 5661 into account yet and sticks to the verified errata. The changes in 3901 seem useful in the longer run one verified and will be implemented eventually. Note that the only existing pNFS block client (Linux) would not benefit from the longer layout lifetimes anyway.

More details are document in the individual patch descriptions and code comments.

This code is also available from:

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-2

Changes since V1:
- rebased to the locks-for-3.20 tree
- fixed a memory leak in the nfsd4_encode_getdeviceinfo error path
- removed the always one lg_roc field
- added a nopnfs export option
- use a synchronous transaction in layoutget to simplify recovery
- various XFS fixes pointed out by Dave
- various documentation typo fixes

Signature · 5 lines
Jeff Layton <jlayton-7I+n7zu2hftEKMMhf/[email protected]>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author reviewer
#71 lore raw

PATCH 4/20 nfsd: factor out a helper to decode nfstime4 values

In reply to Christoph Hellwig
---
 fs/nfsd/nfs4xdr.c | 43 ++++++++++++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 17 deletions(-)
fs/nfsd/nfs4xdr.c +26 -17
index 15f7b73..884ffa3 100644
@@ -234,6 +234,26 @@ static char *savemem(struct nfsd4_compoundargs *argp, __be32 *p, int nbytes)
 	return ret;
 }
 
+/*
+ * We require the high 32 bits of 'seconds' to be 0, and
+ * we ignore all 32 bits of 'nseconds'.
+ */
+static __be32
+nfsd4_decode_time(struct nfsd4_compoundargs *argp, struct timespec *tv)
+{
+	DECODE_HEAD;
+	u64 sec;
+
+	READ_BUF(12);
+	p = xdr_decode_hyper(p, &sec);
+	tv->tv_sec = sec;
+	tv->tv_nsec = be32_to_cpup(p++);
+	if (tv->tv_nsec >= (u32)1000000000)
+		return nfserr_inval;
+
+	DECODE_TAIL;
+}
+
 static __be32
 nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 {
@@ -267,7 +287,6 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 {
 	int expected_len, len = 0;
 	u32 dummy32;
-	u64 sec;
 	char *buf;
 
 	DECODE_HEAD;
@@ -358,15 +377,10 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 		dummy32 = be32_to_cpup(p++);
 		switch (dummy32) {
 		case NFS4_SET_TO_CLIENT_TIME:
-			/* We require the high 32 bits of 'seconds' to be 0, and we ignore
-			   all 32 bits of 'nseconds'. */
-			READ_BUF(12);
 			len += 12;
-			p = xdr_decode_hyper(p, &sec);
-			iattr->ia_atime.tv_sec = (time_t)sec;
-			iattr->ia_atime.tv_nsec = be32_to_cpup(p++);
-			if (iattr->ia_atime.tv_nsec >= (u32)1000000000)
-				return nfserr_inval;
+			status = nfsd4_decode_time(argp, &iattr->ia_atime);
+			if (status)
+				return status;
 			iattr->ia_valid |= (ATTR_ATIME | ATTR_ATIME_SET);
 			break;
 		case NFS4_SET_TO_SERVER_TIME:
@@ -382,15 +396,10 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 		dummy32 = be32_to_cpup(p++);
 		switch (dummy32) {
 		case NFS4_SET_TO_CLIENT_TIME:
-			/* We require the high 32 bits of 'seconds' to be 0, and we ignore
-			   all 32 bits of 'nseconds'. */
-			READ_BUF(12);
 			len += 12;
-			p = xdr_decode_hyper(p, &sec);
-			iattr->ia_mtime.tv_sec = sec;
-			iattr->ia_mtime.tv_nsec = be32_to_cpup(p++);
-			if (iattr->ia_mtime.tv_nsec >= (u32)1000000000)
-				return nfserr_inval;
+			status = nfsd4_decode_time(argp, &iattr->ia_mtime);
+			if (status)
+				return status;
 			iattr->ia_valid |= (ATTR_MTIME | ATTR_MTIME_SET);
 			break;
 		case NFS4_SET_TO_SERVER_TIME:
Signature · 1 line
1.9.1

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-5-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#73 lore raw

Re: [PATCH 04/20] nfsd: factor out a helper to decode nfstime4 values

I'll go ahead and take this one.--b.

Quoted 84 lines · Christoph Hellwig
---
 fs/nfsd/nfs4xdr.c | 43 ++++++++++++++++++++++++++-----------------
 1 file changed, 26 insertions(+), 17 deletions(-)
fs/nfsd/nfs4xdr.c +26 -17
index 15f7b73..884ffa3 100644
@@ -234,6 +234,26 @@ static char *savemem(struct nfsd4_compoundargs *argp, __be32 *p, int nbytes)
 	return ret;
 }
 
+/*
+ * We require the high 32 bits of 'seconds' to be 0, and
+ * we ignore all 32 bits of 'nseconds'.
+ */
+static __be32
+nfsd4_decode_time(struct nfsd4_compoundargs *argp, struct timespec *tv)
+{
+	DECODE_HEAD;
+	u64 sec;
+
+	READ_BUF(12);
+	p = xdr_decode_hyper(p, &sec);
+	tv->tv_sec = sec;
+	tv->tv_nsec = be32_to_cpup(p++);
+	if (tv->tv_nsec >= (u32)1000000000)
+		return nfserr_inval;
+
+	DECODE_TAIL;
+}
+
 static __be32
 nfsd4_decode_bitmap(struct nfsd4_compoundargs *argp, u32 *bmval)
 {
@@ -267,7 +287,6 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 {
 	int expected_len, len = 0;
 	u32 dummy32;
-	u64 sec;
 	char *buf;
 
 	DECODE_HEAD;
@@ -358,15 +377,10 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 		dummy32 = be32_to_cpup(p++);
 		switch (dummy32) {
 		case NFS4_SET_TO_CLIENT_TIME:
-			/* We require the high 32 bits of 'seconds' to be 0, and we ignore
-			   all 32 bits of 'nseconds'. */
-			READ_BUF(12);
 			len += 12;
-			p = xdr_decode_hyper(p, &sec);
-			iattr->ia_atime.tv_sec = (time_t)sec;
-			iattr->ia_atime.tv_nsec = be32_to_cpup(p++);
-			if (iattr->ia_atime.tv_nsec >= (u32)1000000000)
-				return nfserr_inval;
+			status = nfsd4_decode_time(argp, &iattr->ia_atime);
+			if (status)
+				return status;
 			iattr->ia_valid |= (ATTR_ATIME | ATTR_ATIME_SET);
 			break;
 		case NFS4_SET_TO_SERVER_TIME:
@@ -382,15 +396,10 @@ nfsd4_decode_fattr(struct nfsd4_compoundargs *argp, u32 *bmval,
 		dummy32 = be32_to_cpup(p++);
 		switch (dummy32) {
 		case NFS4_SET_TO_CLIENT_TIME:
-			/* We require the high 32 bits of 'seconds' to be 0, and we ignore
-			   all 32 bits of 'nseconds'. */
-			READ_BUF(12);
 			len += 12;
-			p = xdr_decode_hyper(p, &sec);
-			iattr->ia_mtime.tv_sec = sec;
-			iattr->ia_mtime.tv_nsec = be32_to_cpup(p++);
-			if (iattr->ia_mtime.tv_nsec >= (u32)1000000000)
-				return nfserr_inval;
+			status = nfsd4_decode_time(argp, &iattr->ia_mtime);
+			if (status)
+				return status;
 			iattr->ia_valid |= (ATTR_MTIME | ATTR_MTIME_SET);
 			break;
 		case NFS4_SET_TO_SERVER_TIME:
Signature · 1 line
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author reviewer
#74 lore raw

PATCH 5/20 nfsd: move nfsd_fh_match to nfsfh.h

In reply to Christoph Hellwig

The pnfs code will need it too. Also remove the nfsd_ prefix to match the other filehandle helpers in that file.

---
 fs/nfsd/nfs4state.c | 12 ++----------
 fs/nfsd/nfsfh.h     |  9 +++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)
fs/nfsd/nfs4state.c +2 -10
index 30ff0d4..b4f86f8 100644
@@ -408,14 +408,6 @@ static unsigned int file_hashval(struct knfsd_fh *fh)
 	return nfsd_fh_hashval(fh) & (FILE_HASH_SIZE - 1);
 }
 
-static bool nfsd_fh_match(struct knfsd_fh *fh1, struct knfsd_fh *fh2)
-{
-	return fh1->fh_size == fh2->fh_size &&
-		!memcmp(fh1->fh_base.fh_pad,
-				fh2->fh_base.fh_pad,
-				fh1->fh_size);
-}
-
 static struct hlist_head file_hashtbl[FILE_HASH_SIZE];
 
 static void
@@ -3300,7 +3292,7 @@ find_file_locked(struct knfsd_fh *fh, unsigned int hashval)
 	struct nfs4_file *fp;
 
 	hlist_for_each_entry_rcu(fp, &file_hashtbl[hashval], fi_hash) {
-		if (nfsd_fh_match(&fp->fi_fhandle, fh)) {
+		if (fh_match(&fp->fi_fhandle, fh)) {
 			if (atomic_inc_not_zero(&fp->fi_ref))
 				return fp;
 		}
@@ -4295,7 +4287,7 @@ laundromat_main(struct work_struct *laundry)
 
 static inline __be32 nfs4_check_fh(struct svc_fh *fhp, struct nfs4_ol_stateid *stp)
 {
-	if (!nfsd_fh_match(&fhp->fh_handle, &stp->st_stid.sc_file->fi_fhandle))
+	if (!fh_match(&fhp->fh_handle, &stp->st_stid.sc_file->fi_fhandle))
 		return nfserr_bad_stateid;
 	return nfs_ok;
 }
fs/nfsd/nfsfh.h +9 -0
index 08236d7..e24d954 100644
@@ -187,6 +187,15 @@ fh_init(struct svc_fh *fhp, int maxsize)
 	return fhp;
 }
 
+static inline bool fh_match(struct knfsd_fh *fh1, struct knfsd_fh *fh2)
+{
+	if (fh1->fh_size != fh2->fh_size)
+		return false;
+	if (memcmp(fh1->fh_base.fh_pad, fh2->fh_base.fh_pad, fh1->fh_size) != 0)
+		return false;
+	return true;
+}
+
 #ifdef CONFIG_NFSD_V3
 /*
  * The wcc data stored in current_fh should be cleared
Signature · 6 lines
1.9.1

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
author reviewer
#75 lore raw

PATCH 9/20 nfsd: make find_any_file available outside nfs4state.c

In reply to Christoph Hellwig
---
 fs/nfsd/nfs4state.c | 2 +-
 fs/nfsd/state.h     | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
fs/nfsd/nfs4state.c +1 -1
index c31b8f8..444d43c 100644
@@ -352,7 +352,7 @@ find_readable_file(struct nfs4_file *f)
 	return ret;
 }
 
-static struct file *
+struct file *
 find_any_file(struct nfs4_file *f)
 {
 	struct file *ret;
fs/nfsd/state.h +1 -0
index cd61e7d..18b5ab8 100644
@@ -579,6 +579,7 @@ static inline void get_nfs4_file(struct nfs4_file *fi)
 {
 	atomic_inc(&fi->fi_ref);
 }
+struct file *find_any_file(struct nfs4_file *f);
 
 /* grace period management */
 void nfsd4_end_grace(struct nfsd_net *nn);
Signature · 1 line
1.9.1
author reviewer
#76 lore raw

PATCH 12/20 nfsd: update documentation for pNFS support

In reply to Christoph Hellwig
---
 Documentation/filesystems/nfs/nfs41-server.txt | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)
Documentation/filesystems/nfs/nfs41-server.txt +8 -15
index c49cd7e..682a59f 100644
@@ -24,11 +24,6 @@ focuses on the mandatory-to-implement NFSv4.1 Sessions, providing
 "exactly once" semantics and better control and throttling of the
 resources allocated for each client.
 
-Other NFSv4.1 features, Parallel NFS operations in particular,
-are still under development out of tree.
-See http://wiki.linux-nfs.org/wiki/index.php/PNFS_prototype_design
-for more information.
-
 The table below, taken from the NFSv4.1 document, lists
 the operations that are mandatory to implement (REQ), optional
 (OPT), and NFSv4.0 operations that are required not to implement (MNI)
@@ -43,9 +38,7 @@ The OPTIONAL features identified and their abbreviations are as follows:
 The following abbreviations indicate the linux server implementation status.
 	I	Implemented NFSv4.1 operations.
 	NS	Not Supported.
-	NS*	unimplemented optional feature.
-	P	pNFS features implemented out of tree.
-	PNS	pNFS features that are not supported yet (out of tree).
+	NS*	Unimplemented optional feature.
 
 Operations
 
@@ -70,13 +63,13 @@ I  | DESTROY_SESSION      | REQ        |              | Section 18.37  |
 I  | EXCHANGE_ID          | REQ        |              | Section 18.35  |
 I  | FREE_STATEID         | REQ        |              | Section 18.38  |
    | GETATTR              | REQ        |              | Section 18.7   |
-P  | GETDEVICEINFO        | OPT        | pNFS (REQ)   | Section 18.40  |
-P  | GETDEVICELIST        | OPT        | pNFS (OPT)   | Section 18.41  |
+I  | GETDEVICEINFO        | OPT        | pNFS (REQ)   | Section 18.40  |
+NS*| GETDEVICELIST        | OPT        | pNFS (OPT)   | Section 18.41  |
    | GETFH                | REQ        |              | Section 18.8   |
 NS*| GET_DIR_DELEGATION   | OPT        | DDELG (REQ)  | Section 18.39  |
-P  | LAYOUTCOMMIT         | OPT        | pNFS (REQ)   | Section 18.42  |
-P  | LAYOUTGET            | OPT        | pNFS (REQ)   | Section 18.43  |
-P  | LAYOUTRETURN         | OPT        | pNFS (REQ)   | Section 18.44  |
+I  | LAYOUTCOMMIT         | OPT        | pNFS (REQ)   | Section 18.42  |
+I  | LAYOUTGET            | OPT        | pNFS (REQ)   | Section 18.43  |
+I  | LAYOUTRETURN         | OPT        | pNFS (REQ)   | Section 18.44  |
    | LINK                 | OPT        |              | Section 18.9   |
    | LOCK                 | REQ        |              | Section 18.10  |
    | LOCKT                | REQ        |              | Section 18.11  |
@@ -122,9 +115,9 @@ Callback Operations
    |                         | MNI       | or OPT)     |               |
    +-------------------------+-----------+-------------+---------------+
    | CB_GETATTR              | OPT       | FDELG (REQ) | Section 20.1  |
-P  | CB_LAYOUTRECALL         | OPT       | pNFS (REQ)  | Section 20.3  |
+I  | CB_LAYOUTRECALL         | OPT       | pNFS (REQ)  | Section 20.3  |
 NS*| CB_NOTIFY               | OPT       | DDELG (REQ) | Section 20.4  |
-P  | CB_NOTIFY_DEVICEID      | OPT       | pNFS (OPT)  | Section 20.12 |
+NS*| CB_NOTIFY_DEVICEID      | OPT       | pNFS (OPT)  | Section 20.12 |
 NS*| CB_NOTIFY_LOCK          | OPT       |             | Section 20.11 |
 NS*| CB_PUSH_DELEG           | OPT       | FDELG (OPT) | Section 20.5  |
    | CB_RECALL               | OPT       | FDELG,      | Section 20.2  |
Signature · 1 line
1.9.1
author reviewer
#77 lore raw

PATCH 13/20 nfsd: add trace events

In reply to Christoph Hellwig

For now just a few simple events to trace the layout stateid lifetime, but these already were enough to find several bugs in the Linux client layout stateid handling.

---
 fs/nfsd/Makefile      |  7 ++++++-
 fs/nfsd/nfs4layouts.c | 16 ++++++++++++++-
 fs/nfsd/nfs4proc.c    |  6 +++++-
 fs/nfsd/trace.c       |  5 +++++
 fs/nfsd/trace.h       | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 85 insertions(+), 3 deletions(-)
 create mode 100644 fs/nfsd/trace.c
 create mode 100644 fs/nfsd/trace.h
fs/nfsd/Makefile +6 -1
index 5806270..6cba933 100644
@@ -2,9 +2,14 @@
 # Makefile for the Linux nfs server
 #
 
+ccflags-y += -I$(src)			# needed for trace events
+
 obj-$(CONFIG_NFSD)	+= nfsd.o
 
-nfsd-y 			:= nfssvc.o nfsctl.o nfsproc.o nfsfh.o vfs.o \
+# this one should be compiled first, as the tracing macros can easily blow up
+nfsd-y			+= trace.o
+
+nfsd-y 			+= nfssvc.o nfsctl.o nfsproc.o nfsfh.o vfs.o \
 			   export.o auth.o lockd.o nfscache.o nfsxdr.o stats.o
 nfsd-$(CONFIG_NFSD_FAULT_INJECTION) += fault_inject.o
 nfsd-$(CONFIG_NFSD_V2_ACL) += nfs2acl.o
fs/nfsd/nfs4layouts.c +15 -1
index 04a358a..1650075 100644
@@ -9,6 +9,7 @@
 
 #include "pnfs.h"
 #include "netns.h"
+#include "trace.h"
 
 #define NFSDDBG_FACILITY                NFSDDBG_PNFS
 
@@ -126,6 +127,8 @@ nfsd4_free_layout_stateid(struct nfs4_stid *stid)
 	struct nfs4_client *clp = ls->ls_stid.sc_client;
 	struct nfs4_file *fp = ls->ls_stid.sc_file;
 
+	trace_layoutstate_free(&ls->ls_stid.sc_stateid);
+
 	spin_lock(&clp->cl_lock);
 	list_del_init(&ls->ls_perclnt);
 	spin_unlock(&clp->cl_lock);
@@ -216,6 +219,7 @@ nfsd4_alloc_layout_stateid(struct nfsd4_compound_state *cstate,
 	list_add(&ls->ls_perfile, &fp->fi_lo_states);
 	spin_unlock(&fp->fi_lock);
 
+	trace_layoutstate_alloc(&ls->ls_stid.sc_stateid);
 	return ls;
 }
 
@@ -281,6 +285,8 @@ nfsd4_recall_file_layout(struct nfs4_layout_stateid *ls)
 	if (list_empty(&ls->ls_layouts))
 		goto out_unlock;
 
+	trace_layout_recall(&ls->ls_stid.sc_stateid);
+
 	atomic_inc(&ls->ls_stid.sc_count);
 	update_stateid(&ls->ls_stid.sc_stateid);
 	memcpy(&ls->ls_recall_sid, &ls->ls_stid.sc_stateid, sizeof(stateid_t));
@@ -455,8 +461,10 @@ nfsd4_return_file_layouts(struct svc_rqst *rqstp,
 	nfserr = nfsd4_preprocess_layout_stateid(rqstp, cstate, &lrp->lr_sid,
 						false, lrp->lr_layout_type,
 						&ls);
-	if (nfserr)
+	if (nfserr) {
+		trace_layout_return_lookup_fail(&lrp->lr_sid);
 		return nfserr;
+	}
 
 	spin_lock(&ls->ls_lock);
 	list_for_each_entry_safe(lp, n, &ls->ls_layouts, lo_perstate) {
@@ -473,6 +481,7 @@ nfsd4_return_file_layouts(struct svc_rqst *rqstp,
 		}
 		lrp->lrs_present = 1;
 	} else {
+		trace_layoutstate_unhash(&ls->ls_stid.sc_stateid);
 		nfs4_unhash_stid(&ls->ls_stid);
 		lrp->lrs_present = 0;
 	}
@@ -571,6 +580,8 @@ nfsd4_cb_layout_fail(struct nfs4_layout_stateid *ls)
 
 	rpc_ntop((struct sockaddr *)&clp->cl_addr, addr_str, sizeof(addr_str));
 
+	nfsd4_cb_layout_fail(ls);
+
 	printk(KERN_WARNING
 		"nfsd: client %s failed to respond to layout recall. "
 		"  Fencing..\n", addr_str);
@@ -598,6 +609,7 @@ nfsd4_cb_layout_done(struct nfsd4_callback *cb, struct rpc_task *task)
 	case 0:
 		return 1;
 	case -NFS4ERR_NOMATCHING_LAYOUT:
+		trace_layout_recall_done(&ls->ls_stid.sc_stateid);
 		task->tk_status = 0;
 		return 1;
 	case -NFS4ERR_DELAY:
@@ -625,6 +637,8 @@ nfsd4_cb_layout_release(struct nfsd4_callback *cb)
 		container_of(cb, struct nfs4_layout_stateid, ls_recall);
 	LIST_HEAD(reaplist);
 
+	trace_layout_recall_release(&ls->ls_stid.sc_stateid);
+
 	nfsd4_return_all_layouts(ls, &reaplist);
 	nfsd4_free_layouts(&reaplist);
 	nfs4_put_stid(&ls->ls_stid);
fs/nfsd/nfs4proc.c +5 -1
index c051d5b..28e3927 100644
@@ -44,6 +44,7 @@
 #include "netns.h"
 #include "acl.h"
 #include "pnfs.h"
+#include "trace.h"
 
 #ifdef CONFIG_NFSD_V4_SECURITY_LABEL
 #include <linux/security.h>
@@ -1298,8 +1299,10 @@ nfsd4_layoutget(struct svc_rqst *rqstp,
 
 	nfserr = nfsd4_preprocess_layout_stateid(rqstp, cstate, &lgp->lg_sid,
 						true, lgp->lg_layout_type, &ls);
-	if (nfserr)
+	if (nfserr) {
+		trace_layout_get_lookup_fail(&lgp->lg_sid);
 		goto out;
+	}
 
 	nfserr = nfserr_recallconflict;
 	if (atomic_read(&ls->ls_stid.sc_file->fi_lo_recalls))
@@ -1359,6 +1362,7 @@ nfsd4_layoutcommit(struct svc_rqst *rqstp,
 						false, lcp->lc_layout_type,
 						&ls);
 	if (nfserr) {
+		trace_layout_commit_lookup_fail(&lcp->lc_sid);
 		/* fixup error code as per RFC5661 */
 		if (nfserr == nfserr_bad_stateid)
 			nfserr = nfserr_badlayout;
fs/nfsd/trace.c new file +5 -0
new file mode 100644
index 0000000..82f8907
@@ -0,0 +1,5 @@
+
+#include "state.h"
+
+#define CREATE_TRACE_POINTS
+#include "trace.h"
fs/nfsd/trace.h new file +54 -0

54 lines not on this page · raw message

Signature · 1 line
1.9.1
author reviewer
#78 lore raw

PATCH 15/20 nfsd: pNFS block layout driver

In reply to Christoph Hellwig

Add a small shim between core nfsd and filesystems to translate the somewhat cumbersome pNFS data structures and semantics to something more palatable for Linux filesystems.

Thanks to Rick McNeal for the old prototype pNFS blocklayout server code, which gave a lot of inspiration to this version even if no code is left from it.

---
 .../filesystems/nfs/pnfs-block-server.txt          |  37 ++++
 fs/nfsd/Makefile                                   |   2 +-
 fs/nfsd/blocklayout.c                              | 190 +++++++++++++++++++++
 fs/nfsd/blocklayoutxdr.c                           | 157 +++++++++++++++++
 fs/nfsd/blocklayoutxdr.h                           |  62 +++++++
 fs/nfsd/nfs4layouts.c                              |   8 +
 fs/nfsd/pnfs.h                                     |   1 +
 7 files changed, 456 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/filesystems/nfs/pnfs-block-server.txt
 create mode 100644 fs/nfsd/blocklayout.c
 create mode 100644 fs/nfsd/blocklayoutxdr.c
 create mode 100644 fs/nfsd/blocklayoutxdr.h
Documentation/filesystems/nfs/pnfs-block-server.txt new file +37 -0
new file mode 100644
index 0000000..2143673
@@ -0,0 +1,37 @@
+pNFS block layout server user guide
+
+The Linux NFS server now supports the pNFS block layout extension.  In this
+case the NFS server acts as Metadata Server (MDS) for pNFS, which in addition
+to handling all the metadata access to the NFS export also hands out layouts
+to the clients to directly access the underlying block devices that are
+shared with the client.
+
+To use pNFS block layouts with with the Linux NFS server the exported file
+system needs to support the pNFS block layouts (currently just XFS), and the
+file system must sit on shared storage (typically iSCSI) that is accessible
+to the clients in addition to the MDS.  As of now the file system needs to
+sit directly on the exported volume, striping or concatenation of
+volumes on the MDS and clients is not supported yet.
+
+On the server, pNFS block volume support is automatically if the file system
+support it.  On the client make sure the kernel has the CONFIG_PNFS_BLOCK
+option enabled, the blkmapd daemon from nfs-utils is running, and the
+file system is mounted using the NFSv4.1 protocol version (mount -o vers=4.1).
+
+If the nfsd server needs to fence a non-responding client it calls
+/sbin/nfsd-recall-failed with the first argument set to the IP address of
+the client, and the second argument set to the device node without the /dev
+prefix for the file system to be fenced. Below is an example file that shows
+how to translate the device into a serial number from SCSI EVPD 0x80:
+
+cat > /sbin/nfsd-recall-failed << EOF
+#!/bin/sh
+
+CLIENT="$1"
+DEV="/dev/$2"
+EVPD=`sg_inq --page=0x80 ${DEV} | \
+	grep "Unit serial number:" | \
+	awk -F ': ' '{print $2}'`
+
+echo "fencing client ${CLIENT} serial ${EVPD}" >> /var/log/pnfsd-fence.log
+EOF
fs/nfsd/Makefile +1 -1
index 6cba933..9a6028e 100644
@@ -17,4 +17,4 @@ nfsd-$(CONFIG_NFSD_V3)	+= nfs3proc.o nfs3xdr.o
 nfsd-$(CONFIG_NFSD_V3_ACL) += nfs3acl.o
 nfsd-$(CONFIG_NFSD_V4)	+= nfs4proc.o nfs4xdr.o nfs4state.o nfs4idmap.o \
 			   nfs4acl.o nfs4callback.o nfs4recover.o
-nfsd-$(CONFIG_NFSD_PNFS) += nfs4layouts.o
+nfsd-$(CONFIG_NFSD_PNFS) += nfs4layouts.o blocklayout.o blocklayoutxdr.o
fs/nfsd/blocklayout.c new file +190 -0

190 lines not on this page · raw message

fs/nfsd/blocklayoutxdr.c new file +157 -0

157 lines not on this page · raw message

fs/nfsd/blocklayoutxdr.h new file +62 -0

62 lines not on this page · raw message

fs/nfsd/nfs4layouts.c +8 -0
index 1650075..7152250 100644
@@ -26,6 +26,7 @@ static struct nfsd4_callback_ops nfsd4_cb_layout_ops;
 static const struct lock_manager_operations nfsd4_layouts_lm_ops;
 
 const struct nfsd4_layout_ops *nfsd4_layout_ops[LAYOUT_TYPE_MAX] =  {
+	[LAYOUT_BLOCK_VOLUME]	= &bl_layout_ops,
 };
 
 /* pNFS device ID to export fsid mapping */
@@ -116,8 +117,15 @@ nfsd4_set_deviceid(struct nfsd4_deviceid *id, const struct svc_fh *fhp,
 
 void nfsd4_setup_layout_type(struct svc_export *exp)
 {
+	struct super_block *sb = exp->ex_path.mnt->mnt_sb;
+
 	if (exp->ex_flags & NFSEXP_NOPNFS)
 		return;
+
+	if (sb->s_export_op->get_uuid &&
+	    sb->s_export_op->map_blocks &&
+	    sb->s_export_op->commit_blocks)
+		exp->ex_layout_type = LAYOUT_BLOCK_VOLUME;
 }
 
 static void
fs/nfsd/pnfs.h +1 -0
index a9616a4..fedb4d6 100644
@@ -34,6 +34,7 @@ struct nfsd4_layout_ops {
 };
 
 extern const struct nfsd4_layout_ops *nfsd4_layout_ops[];
+extern const struct nfsd4_layout_ops bl_layout_ops;
 
 __be32 nfsd4_preprocess_layout_stateid(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *cstate, stateid_t *stateid,
Signature · 1 line
1.9.1
author reviewer
#79 lore raw

PATCH 18/20 xfs: factor out a xfs_update_prealloc_flags() helper

In reply to Christoph Hellwig

This logic is duplicated in xfs_file_fallocate and xfs_ioc_space, and we'll need another copy of it for pNFS block support.

---
 fs/xfs/xfs_file.c  | 64 +++++++++++++++++++++++++++++++++++-------------------
 fs/xfs/xfs_inode.h |  9 ++++++++
 fs/xfs/xfs_ioctl.c | 50 ++++++++++--------------------------------
 3 files changed, 63 insertions(+), 60 deletions(-)
fs/xfs/xfs_file.c +42 -22
index 13e974e..712d312 100644
@@ -127,6 +127,42 @@ xfs_iozero(
 	return (-status);
 }
 
+int
+xfs_update_prealloc_flags(
+	struct xfs_inode	*ip,
+	enum xfs_prealloc_flags	flags)
+{
+	struct xfs_trans	*tp;
+	int			error;
+
+	tp = xfs_trans_alloc(ip->i_mount, XFS_TRANS_WRITEID);
+	error = xfs_trans_reserve(tp, &M_RES(ip->i_mount)->tr_writeid, 0, 0);
+	if (error) {
+		xfs_trans_cancel(tp, 0);
+		return error;
+	}
+
+	xfs_ilock(ip, XFS_ILOCK_EXCL);
+	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
+
+	if (!(flags & XFS_PREALLOC_INVISIBLE)) {
+		ip->i_d.di_mode &= ~S_ISUID;
+		if (ip->i_d.di_mode & S_IXGRP)
+			ip->i_d.di_mode &= ~S_ISGID;
+		xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
+	}
+
+	if (flags & XFS_PREALLOC_SET)
+		ip->i_d.di_flags |= XFS_DIFLAG_PREALLOC;
+	if (flags & XFS_PREALLOC_CLEAR)
+		ip->i_d.di_flags &= ~XFS_DIFLAG_PREALLOC;
+
+	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+	if (flags & XFS_PREALLOC_SYNC)
+		xfs_trans_set_sync(tp);
+	return xfs_trans_commit(tp, 0);
+}
+
 /*
  * Fsync operations on directories are much simpler than on regular files,
  * as there is no file data to flush, and thus also no need for explicit
@@ -784,8 +820,8 @@ xfs_file_fallocate(
 {
 	struct inode		*inode = file_inode(file);
 	struct xfs_inode	*ip = XFS_I(inode);
-	struct xfs_trans	*tp;
 	long			error;
+	enum xfs_prealloc_flags	flags = 0;
 	loff_t			new_size = 0;
 
 	if (!S_ISREG(inode->i_mode))
@@ -822,6 +858,8 @@ xfs_file_fallocate(
 		if (error)
 			goto out_unlock;
 	} else {
+		flags |= XFS_PREALLOC_SET;
+
 		if (!(mode & FALLOC_FL_KEEP_SIZE) &&
 		    offset + len > i_size_read(inode)) {
 			new_size = offset + len;
@@ -839,28 +877,10 @@ xfs_file_fallocate(
 			goto out_unlock;
 	}
 
-	tp = xfs_trans_alloc(ip->i_mount, XFS_TRANS_WRITEID);
-	error = xfs_trans_reserve(tp, &M_RES(ip->i_mount)->tr_writeid, 0, 0);
-	if (error) {
-		xfs_trans_cancel(tp, 0);
-		goto out_unlock;
-	}
-
-	xfs_ilock(ip, XFS_ILOCK_EXCL);
-	xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
-	ip->i_d.di_mode &= ~S_ISUID;
-	if (ip->i_d.di_mode & S_IXGRP)
-		ip->i_d.di_mode &= ~S_ISGID;
-
-	if (!(mode & (FALLOC_FL_PUNCH_HOLE | FALLOC_FL_COLLAPSE_RANGE)))
-		ip->i_d.di_flags |= XFS_DIFLAG_PREALLOC;
-
-	xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
-	xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
-
 	if (file->f_flags & O_DSYNC)
-		xfs_trans_set_sync(tp);
-	error = xfs_trans_commit(tp, 0);
+		flags |= XFS_PREALLOC_SYNC;
+
+	error = xfs_update_prealloc_flags(ip, flags);
 	if (error)
 		goto out_unlock;
 
fs/xfs/xfs_inode.h +9 -0
index 4ed2ba9..bc220bc 100644
@@ -377,6 +377,15 @@ int		xfs_droplink(struct xfs_trans *, struct xfs_inode *);
 int		xfs_bumplink(struct xfs_trans *, struct xfs_inode *);
 
 /* from xfs_file.c */
+enum xfs_prealloc_flags {
+	XFS_PREALLOC_SET	= (1 << 1),
+	XFS_PREALLOC_CLEAR	= (1 << 2),
+	XFS_PREALLOC_SYNC	= (1 << 3),
+	XFS_PREALLOC_INVISIBLE	= (1 << 4),
+};
+
+int		xfs_update_prealloc_flags(struct xfs_inode *,
+			enum xfs_prealloc_flags);
 int		xfs_zero_eof(struct xfs_inode *, xfs_off_t, xfs_fsize_t);
 int		xfs_iozero(struct xfs_inode *, loff_t, size_t);
 
fs/xfs/xfs_ioctl.c +12 -38

100 lines not on this page · raw message

Signature · 1 line
1.9.1

A message that is not in the archive (sent off-list or not yet received): 1421925006-24231-19-git-send-email-hch-jcswGhMUV9g@public.gmane.org

#81 lore raw

Re: [PATCH 18/20] xfs: factor out a xfs_update_prealloc_flags() helper

Quoted 4 lines · Christoph Hellwig

This logic is duplicated in xfs_file_fallocate and xfs_ioc_space, and we'll need another copy of it for pNFS block support.

This stands alone, so I'm pulling it into the XFS tree without waiting for the rest of the patch set.

Cheers,

Dave.

Signature · 6 lines
Dave Chinner
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#82 lore raw
In reply to Christoph Hellwig

Hi Christoph-

Very nice, and welcome, work.

Quoted 4 lines · Christoph Hellwig

This series adds support for the pNFS operations in NFS v4.1, as well as a block layout driver that can export block based filesystems that implement a few additional export operations. Support for XFS is provided in this series, but other filesystems could be added easily.

I’m still not clear on what layout types are supported. Clearly the block layout type is supported. But you mention XFS here. Does that mean file layouts are also supported now? If so, can you briefly describe the architecture and any current limitations?

Did I miss documentation for how administrators construct layouts?

Quoted 42 lines

The core pNFS code of course owns its heritage to the existing Linux pNFS server prototype, but except for a few bits and pieces in the XDR path nothing is left from it.

The design of this new pNFS server is fairly different from the old one - while the old one implemented very little semantics in nfsd and left almost everything to filesystems my implementation implements as much as possible in common nfsd code, then dispatches to a layout driver that still is part of nfsd and only then calls into the filesystem, thus keeping it free from intimate pNFS knowledge.

This version of the code has been rebased to the locks-for-3.20 tree which adds a new lock context structure to the inode. For now we still (ab)use the lease list like in the last version. Adding a pNFS-specific list would duplicate a lot of code without much benefit. But during this research I came up with way to associate a nfs4_file with a struct file at open time which should allow to greatly simplify the pNFS and delegation code. So stay tuned for some patches in this area!

For now this also doesn't take errata 3901 for rfc 5661 into account yet and sticks to the verified errata. The changes in 3901 seem useful in the longer run one verified and will be implemented eventually. Note that the only existing pNFS block client (Linux) would not benefit from the longer layout lifetimes anyway.

More details are document in the individual patch descriptions and code comments.

This code is also available from:

	git://git.infradead.org/users/hch/pnfs.git pnfsd-for-3.20-2

Changes since V1:
- rebased to the locks-for-3.20 tree
- fixed a memory leak in the nfsd4_encode_getdeviceinfo error path
- removed the always one lg_roc field
- added a nopnfs export option
- use a synchronous transaction in layoutget to simplify recovery
- various XFS fixes pointed out by Dave
- various documentation typo fixes

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

author reviewer
#83 lore raw
Quoted 6 lines · Chuck Lever

I’m still not clear on what layout types are supported. Clearly the block layout type is supported. But you mention XFS here. Does that mean file layouts are also supported now? If so, can you briefly describe the architecture and any current limitations?

Did I miss documentation for how administrators construct layouts?

Only block layouts are supporte at the moment. They are automatically
enabled when an XFS filesystem is exported. There is no requirement
for the administrator to construct in layout, the only requirement for working block layouts is shared storage that is accessible to the NFS Server / MDS and the clients that wish to use pNFS‥

If you are interested in file layout support porting over the existing file layout drivers from the old pNFS server tree should only be a moderate effort, and at least the lexp code would be grately simplified by the common recall infrastructure in this series. --
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html