Skip to contentSkip to conversation

NFS/NFSD support for async and inter COPY

PATCH Superseded Olga Kornievskaia 6 participants 81 messages Jul 11, 2017 – Sep 16, 2017

Subsystems (4)

v3 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover NFS/NFSD support for async and inter COPY

This patch series provides support for NFSv4.2 COPY featuring support for asynchronous copy and inter SSC copy.

In this iteration including both client and server side in one series because server side does depend on client side patches.

Client side:
In case, of the "inter" SSC copy files reside on different servers and thus under different superblocks and require that VFS removes the restriction that src and dst files must be on the same superblock.

NFS's copy_file_range() determines if the copy is "intra" or "inter" and for "inter" it sends the COPY_NOTIFY to the source server. Then, it would send of an asynchronous COPY to the server (destination in case of "inter"). If server errs with ERR_OFFLOAD_NOREQS the copy will be re-sent as a synchronous COPY. If application cancels an in-flight COPY, OFFLOAD_CANCEL is sent to the source server.

If server replies to the COPY with the copy stateid, client will go wait on the CB_OFFLOAD. To fend off the race between CB_OFFLOAD and COPY reply, we check the list of pending callbacks before going to wait. Client adds the copy to the global list of copy stateids for the callback to look thru and signal the waiting copy.

If application cancels async COPY after reply is received, wait will be interrupted and client will send OFFLOAD_CANCEL to the source and destination servers (sending it as an async RPC in the context of the nfsiod_workqueue).

When the client receives reply from the CB_OFFLOAD with some bytes and committed how is UNSTABLE, then COMMIT is sent to the server. The results are propagated to the VFS and application. Assuming that application will deal with a partial result and continue from the new offset if needed.

Handling reboot of the destination server when client is waiting on the CB_OFFLOAD happens when SEQUENCE discovers that destination server rebooted. The open state initially is marked to be NFS_CLNT_DST_SSC_COPY_STATE during the COPY. Then during the recovery if state is marked as such, then look thru the list of copies for the server and see if any are associated with this recovering open, if so mark the copy rebooted and wake up the waiting copy. Upon wake up the waiting copy, will restart the copy from scratch.

If the source server is rebooted, the destination server will also know about it and it will return the partial result via CB_OFFLOAD, then the result will be propagated back to the application which will initiate the new copy and new COPY_NOTIFY will be sent.

If CB_OFFLOAD returned an error and non negative value of partial copy and error is not ENOSPC, then ignore the error and send the commit and return partial result to the client to start the next copy.

On the destination server, it's now acting as a client and needs to do a special "open" and "close". Since destination server doesn't do an open on the wire, we "fake" create the needed data structures and that's done in the new function nfs42_ssc_open(). To clean up this open but not trigger the CLOSE on the wire, we have a new function nfs42_ssc_close() that accomplishes that.

Server side:
This is server-side support for NFSv4.2 inter and async COPY which is on top of existing intra sync COPY. It also depends on the NFS client piece for NFSv4.2 to do client side of the destination server piece in the inter SSC.

NFSD determines if COPY is intra or inter and if sync or async. For inter, NSFD uses NFSv4.1 protocol and creates an internal mount point (superblock).

To do asynchronous copies, NFSD creates a single threaded workqueue and does not tie up an NFSD thread to complete the copy. Upon receiving the COPY, it generates a unique copy stateid (stores a global list for keeping track of state for OFFLOAD_STATUS to be queried by), queues up a workqueue for the copy, and replies back to the client. nfsd4_copy arguments that are allocated on the stack are copied for the work item.

In the async copy handler, it calls into VFS copy_file_range() and loops until it completes the requested copy size. If error is encountered it's saved but also we save the amount of data copied so far. Once done, the results are queued for the callback workqueue and sent via CB_OFFLOAD. Also currently, choosing to clean up the copy state information stored in the global list when cope is done and not doing it when callback's release function (it could be done there alternatively if needed it?).

On the source server, upon receiving a COPY_NOTIFY, it generate a unique stateid that's kept in the global list. Upon receiving a READ with a stateid, the code checks the normal list of open stateid and now additionally, it'll check the copy state list as well before deciding to either fail with BAD_STATEID or find one that matches. The stored stateid is only valid to be used for the first time with a choosen lease period (90s currently). When the source server received an OFFLOAD_CANCEL, it will remove the stateid from the global list. Otherwise, the copy stateid is removed upon the removal of its "parent" stateid (open/lock/delegation stateid).

v3
Add support to delay the unmount of the source server on the destination server for 90s to allow for consecutive COPY operations to reuse the session/clientid.

Andy Adamson (9):
NFS inter ssc open
NFS add COPY_NOTIFY operation
NFSD add ca_source_server<> to COPY
NFSD generalize nfsd4_compound_state flag names
NFSD add COPY_NOTIFY operation
NFSD: allow inter server COPY to have a STALE source server fh NFSD add nfs4 inter ssc to nfsd4_copy
NFSD return nfs4_stid in nfs4_preprocess_stateid_op
NFSD Unique stateid_t for inter server to server COPY authentication

Anna Schumaker (1):
fs: Don't copy beyond the end of the file

Olga Kornievskaia (32):
VFS permit cross device vfs_copy_file_range
NFS CB_OFFLOAD xdr
NFS OFFLOAD_STATUS xdr
NFS OFFLOAD_STATUS op
NFS OFFLOAD_CANCEL xdr
NFS COPY xdr handle async reply
NFS add support for asynchronous COPY
NFS handle COPY reply CB_OFFLOAD call race
NFS export nfs4_async_handle_error
NFS test for intra vs inter COPY
NFS send OFFLOAD_CANCEL when COPY killed
NFS handle COPY ERR_OFFLOAD_NO_REQS
NFS if we got partial copy ignore errors
NFS recover from destination server reboot for copies NFS NFSD defining nl4_servers structure needed by both NFS also send OFFLOAD_CANCEL to source server
NFS skip recovery of copy open on dest server
NFS also send OFFLOAD_CANCEL to source server
NFS add a simple sync nfs4_proc_commit after async COPY NFSD CB_OFFLOAD xdr
NFSD OFFLOAD_STATUS xdr
NFSD OFFLOAD_CANCEL xdr
NFSD xdr callback stateid in async COPY reply NFSD first draft of async copy
NFSD handle OFFLOAD_CANCEL op
NFSD stop queued async copies on client shutdown NFSD create new stateid for async copy
NFSD define EBADF in nfserrno
NFSD support OFFLOAD_STATUS
NFSD remove copy stateid when vfs_copy_file_range completes NFSD delay the umount after COPY

 Documentation/filesystems/vfs.txt |   6 +
 fs/nfs/callback.h                 |  13 +
 fs/nfs/callback_proc.c            |  52 +++
 fs/nfs/callback_xdr.c             |  80 ++++-
 fs/nfs/client.c                   |   1 +
 fs/nfs/internal.h                 |  10 +
 fs/nfs/nfs42.h                    |  10 +-
 fs/nfs/nfs42proc.c                | 380 +++++++++++++++++++-
 fs/nfs/nfs42xdr.c                 | 388 +++++++++++++++++++-
 fs/nfs/nfs4_fs.h                  |  18 +-
 fs/nfs/nfs4client.c               |  15 +
 fs/nfs/nfs4file.c                 | 146 +++++++-
 fs/nfs/nfs4proc.c                 |  44 ++-
 fs/nfs/nfs4state.c                |  31 +-
 fs/nfs/nfs4xdr.c                  |   3 +
 fs/nfsd/Kconfig                   |  10 +
 fs/nfsd/netns.h                   |   8 +
 fs/nfsd/nfs4callback.c            |  95 +++++
 fs/nfsd/nfs4proc.c                | 727 ++++++++++++++++++++++++++++++++++++--
 fs/nfsd/nfs4state.c               | 149 +++++++-
 fs/nfsd/nfs4xdr.c                 | 266 +++++++++++++-
 fs/nfsd/nfsctl.c                  |   2 +
 fs/nfsd/nfsd.h                    |   2 +
 fs/nfsd/nfsproc.c                 |   1 +
 fs/nfsd/state.h                   |  34 +-
 fs/nfsd/xdr4.h                    |  61 +++-
 fs/nfsd/xdr4cb.h                  |  10 +
 fs/read_write.c                   |  16 +-
 include/linux/nfs4.h              |  37 ++
 include/linux/nfs_fs.h            |  11 +
 include/linux/nfs_fs_sb.h         |   5 +
 include/linux/nfs_xdr.h           |  33 ++
 32 files changed, 2563 insertions(+), 101 deletions(-)
Signature · 1 line
1.8.3.1
#1 lore raw

RFC v3 1/42 fs: Don't copy beyond the end of the file

From: Anna Schumaker <[email protected]>

---
 fs/read_write.c | 3 +++
 1 file changed, 3 insertions(+)
fs/read_write.c +3 -0
index 19d4d88..cc2618b 100644
@@ -1584,6 +1584,9 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,
 	if (unlikely(ret))
 		return ret;
 
+	if (pos_in >= i_size_read(inode_in))
+		return -EINVAL;
+
 	if (!(file_in->f_mode & FMODE_READ) ||
 	    !(file_out->f_mode & FMODE_WRITE) ||
 	    (file_out->f_flags & O_APPEND))
Signature · 1 line
1.8.3.1
#2 lore raw

Re: [RFC v3 01/42] fs: Don't copy beyond the end of the file

Hi Olga,

Quoted 3 lines · Olga Kornievskaia

From: Anna Schumaker <[email protected]>

Would it make sense to submit this patch to the VFS in the same series as the cross mountpoint patch, that way they both go in at once? This one should really be able to go in now, rather than waiting for the rest of the series.

Thanks,
Anna

Quoted 19 lines
---
 fs/read_write.c | 3 +++
 1 file changed, 3 insertions(+)
fs/read_write.c +3 -0
index 19d4d88..cc2618b 100644
@@ -1584,6 +1584,9 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,
 	if (unlikely(ret))
 		return ret;
 
+	if (pos_in >= i_size_read(inode_in))
+		return -EINVAL;
+
 	if (!(file_in->f_mode & FMODE_READ) ||
 	    !(file_out->f_mode & FMODE_WRITE) ||
 	    (file_out->f_flags & O_APPEND))
#3 lore raw

RFC v3 2/42 VFS permit cross device vfs_copy_file_range

In reply to Olga Kornievskaia

Allow copy_file_range to copy between different superblocks but only of the same file system types.

This feature is needed by NFSv4.2 to perform file copy operation on the same server or file copy between different NFSv4.2 servers.

If a file system's fileoperations copy_file_range operation prohibits cross-device copies, fall back to do_splice_direct. This would be needed for the NFS (destination) server side implementation of the file copy and currently for CIFS.

Besides NFS, there is only 1 implementor of the copy_file_range FS operation -- CIFS. CIFS assumes incoming file descriptors are both CIFS but it will check if they are coming from different servers and return error code to fall back to do_splice_direct.

NFS will allow for copies between different NFS servers.

Adding to the vfs.txt documentation to explicitly warn about allowing for different superblocks of the same file type to be passed into the copy_file_range for the future users of copy_file_range method.

---
 Documentation/filesystems/vfs.txt |  6 ++++++
 fs/read_write.c                   | 13 ++++++-------
 2 files changed, 12 insertions(+), 7 deletions(-)
Documentation/filesystems/vfs.txt +6 -0
index f42b906..fd6b105 100644
@@ -845,6 +845,8 @@ struct file_operations {
 #ifndef CONFIG_MMU
 	unsigned (*mmap_capabilities)(struct file *);
 #endif
+	ssize_t (*copy_file_range)(struct file *, loff_t, struct file *,
+				   loff_t, size_t, unsigned int);
 };
 
 Again, all methods are called without any locks being held, unless
@@ -913,6 +915,10 @@ otherwise noted.
 
   fallocate: called by the VFS to preallocate blocks or punch a hole.
 
+  copy_file_range: called by copy_file_range(2) system call. This method
+		   works on two file descriptors that might reside on
+		   different superblocks of the same type of file system.
+
 Note that the file operations are implemented by the specific
 filesystem in which the inode resides. When opening a device node
 (character or block special) most filesystems will call special
fs/read_write.c +6 -7
index cc2618b..406d1c6 100644
@@ -1592,10 +1592,6 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,
 	    (file_out->f_flags & O_APPEND))
 		return -EBADF;
 
-	/* this could be relaxed once a method supports cross-fs copies */
-	if (inode_in->i_sb != inode_out->i_sb)
-		return -EXDEV;
-
 	if (len == 0)
 		return 0;
 
@@ -1605,7 +1601,8 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,
 	 * Try cloning first, this is supported by more file systems, and
 	 * more efficient if both clone and copy are supported (e.g. NFS).
 	 */
-	if (file_in->f_op->clone_file_range) {
+	if (inode_in->i_sb == inode_out->i_sb &&
+			file_in->f_op->clone_file_range) {
 		ret = file_in->f_op->clone_file_range(file_in, pos_in,
 				file_out, pos_out, len);
 		if (ret == 0) {
@@ -1614,10 +1611,12 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,
 		}
 	}
 
-	if (file_out->f_op->copy_file_range) {
+	if (file_out->f_op->copy_file_range &&
+			(file_in->f_op->copy_file_range ==
+				file_out->f_op->copy_file_range)) {
 		ret = file_out->f_op->copy_file_range(file_in, pos_in, file_out,
 						      pos_out, len, flags);
-		if (ret != -EOPNOTSUPP)
+		if (ret != -EOPNOTSUPP && ret != -EXDEV)
 			goto done;
 	}
 
Signature · 1 line
1.8.3.1
#4 lore raw

RFC v3 3/42 NFS CB_OFFLOAD xdr

In reply to Olga Kornievskaia
---
 fs/nfs/callback.h      | 13 ++++++++
 fs/nfs/callback_proc.c |  7 +++++
 fs/nfs/callback_xdr.c  | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 1 deletion(-)
fs/nfs/callback.h +13 -0
index c701c30..e4ab65d 100644
@@ -189,6 +189,19 @@ extern __be32 nfs4_callback_notify_lock(struct cb_notify_lock_args *args,
 					 void *dummy,
 					 struct cb_process_state *cps);
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+struct cb_offloadargs {
+	struct nfs_fh		coa_fh;
+	nfs4_stateid		coa_stateid;
+	uint32_t		error;
+	uint64_t		wr_count;
+	struct nfs_writeverf	wr_writeverf;
+};
+
+extern __be32 nfs4_callback_offload(
+	struct cb_offloadargs *args,
+	void *dummy, struct cb_process_state *cps);
+#endif /* CONFIG_NFS_V4_2 */
 extern int check_gss_callback_principal(struct nfs_client *, struct svc_rqst *);
 extern __be32 nfs4_callback_getattr(struct cb_getattrargs *args,
 				    struct cb_getattrres *res,
fs/nfs/callback_proc.c +7 -0
index 52479f1..5396751 100644
@@ -647,3 +647,10 @@ __be32 nfs4_callback_notify_lock(struct cb_notify_lock_args *args, void *dummy,
 	return htonl(NFS4_OK);
 }
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+__be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy,
+				struct cb_process_state *cps)
+{
+	return 0;
+}
+#endif /* CONFIG_NFS_V4_2 */
fs/nfs/callback_xdr.c +79 -1

105 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#5 lore raw

Re: [RFC v3 03/42] NFS CB_OFFLOAD xdr

Hi Olga,

Quoted 146 lines · Olga Kornievskaia
---
 fs/nfs/callback.h      | 13 ++++++++
 fs/nfs/callback_proc.c |  7 +++++
 fs/nfs/callback_xdr.c  | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 1 deletion(-)
fs/nfs/callback.h +13 -0
index c701c30..e4ab65d 100644
@@ -189,6 +189,19 @@ extern __be32 nfs4_callback_notify_lock(struct cb_notify_lock_args *args,
 					 void *dummy,
 					 struct cb_process_state *cps);
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+struct cb_offloadargs {
+	struct nfs_fh		coa_fh;
+	nfs4_stateid		coa_stateid;
+	uint32_t		error;
+	uint64_t		wr_count;
+	struct nfs_writeverf	wr_writeverf;
+};
+
+extern __be32 nfs4_callback_offload(
+	struct cb_offloadargs *args,
+	void *dummy, struct cb_process_state *cps);
+#endif /* CONFIG_NFS_V4_2 */
 extern int check_gss_callback_principal(struct nfs_client *, struct svc_rqst *);
 extern __be32 nfs4_callback_getattr(struct cb_getattrargs *args,
 				    struct cb_getattrres *res,
fs/nfs/callback_proc.c +7 -0
index 52479f1..5396751 100644
@@ -647,3 +647,10 @@ __be32 nfs4_callback_notify_lock(struct cb_notify_lock_args *args, void *dummy,
 	return htonl(NFS4_OK);
 }
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+__be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy,
+				struct cb_process_state *cps)
+{
+	return 0;
+}
+#endif /* CONFIG_NFS_V4_2 */
fs/nfs/callback_xdr.c +72 -1
index 390ac9c..be5ffa1 100644
@@ -37,6 +37,9 @@
 #define CB_OP_RECALLSLOT_RES_MAXSZ	(CB_OP_HDR_RES_MAXSZ)
 #define CB_OP_NOTIFY_LOCK_RES_MAXSZ	(CB_OP_HDR_RES_MAXSZ)
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+#define CB_OP_OFFLOAD_RES_MAXSZ		(CB_OP_HDR_RES_MAXSZ)
+#endif /* CONFIG_NFS_V4_2 */
 
 #define NFSDBG_FACILITY NFSDBG_CALLBACK
 
@@ -521,7 +524,72 @@ static __be32 decode_notify_lock_args(struct svc_rqst *rqstp, struct xdr_stream
 }
 
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+static __be32 decode_write_response(struct xdr_stream *xdr,
+					struct cb_offloadargs *args)
+{
+	__be32 *p;
+	__be32 dummy;
+
+	/* skip the always zero field */
+	p = read_buf(xdr, 4);
+	if (unlikely(!p))
+		goto out;
+	dummy = ntohl(*p++);
+
+	/* decode count, stable_how, verifier */
+	p = xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out;
+	p = xdr_decode_hyper(p, &args->wr_count);
+	args->wr_writeverf.committed = be32_to_cpup(p);
+	p = xdr_inline_decode(xdr, NFS4_VERIFIER_SIZE);
+	if (likely(p)) {
+		memcpy(&args->wr_writeverf.verifier.data[0], p,
+			NFS4_VERIFIER_SIZE);
+		return 0;
+	}
+out:
+	return htonl(NFS4ERR_RESOURCE);
+}
+
+static __be32 decode_offload_args(struct svc_rqst *rqstp,
+					struct xdr_stream *xdr,
+					struct cb_offloadargs *args)
+{
+	__be32 *p;
+	__be32 status;
+
+	/* decode fh */
+	status = decode_fh(xdr, &args->coa_fh);
+	if (unlikely(status != 0))
+		return status;
+
+	/* decode stateid */
+	status = decode_stateid(xdr, &args->coa_stateid);
+	if (unlikely(status != 0))
+		return status;
 
+	/* decode status */
+	p = read_buf(xdr, 4);
+	if (unlikely(!p))
+		goto out;
+	args->error = ntohl(*p++);
+	if (!args->error) {
+		status = decode_write_response(xdr, args);
+		if (unlikely(status != 0))
+			return status;
+	} else {
+		p = xdr_inline_decode(xdr, 8);
+		if (unlikely(!p))
+			goto out;
+		p = xdr_decode_hyper(p, &args->wr_count);
+	}
+	return 0;
+out:
+	return htonl(NFS4ERR_RESOURCE);
+}
+#endif /* CONFIG_NFS_V4_2 */
 static __be32 encode_string(struct xdr_stream *xdr, unsigned int len, const char *str)
 {
 	if (unlikely(xdr_stream_encode_opaque(xdr, str, len) < 0))
@@ -785,7 +853,10 @@ static void nfs4_cb_free_slot(struct cb_process_state *cps)
 	if (status != htonl(NFS4ERR_OP_ILLEGAL))
 		return status;
 
-	if (op_nr == OP_CB_OFFLOAD)
+	if (op_nr == OP_CB_OFFLOAD) {
+		*op = &callback_ops[op_nr];
+		return htonl(NFS_OK);
+	} else
 		return htonl(NFS4ERR_NOTSUPP);
 	return htonl(NFS4ERR_OP_ILLEGAL);
 }

fs/nfs/callback_xdr.c static __be32 nfs4_callback_compound(struct svc_rqst *rqstp, void *argp, void *r · line 1062

Quoted 10 lines
fs/nfs/callback_xdr.c +7 -0
@@ -982,6 +1053,13 @@ static __be32 nfs4_callback_compound(struct svc_rqst *rqstp, void *argp, void *r
 		.res_maxsize = CB_OP_NOTIFY_LOCK_RES_MAXSZ,
 	},
 #endif /* CONFIG_NFS_V4_1 */
+#ifdef CONFIG_NFS_V4_2
+	[OP_CB_OFFLOAD] = {
+		.process_op = (callback_process_op_t)nfs4_callback_offload,
+		.decode_args = (callback_decode_arg_t)decode_offload_args,
+		.res_maxsize = CB_OP_OFFLOAD_RES_MAXSZ,
+	},
+#endif /* CONFIG_NFS_V4_2 */

Just a heads up that the function signatures change in 4.13 and the casting is removed. The .process_op function now expects:

    __be32 func(void *argp, void *resp, struct cb_process_state *cps);

And .decode_args is now:

    __be32 func(struct svc_rqst *rqstp, struct xdr_stream *xdr, void *argp);

Thanks,
Anna

Quoted 4 lines
fs/nfs/callback_xdr.c +0 -0
 };
 
 /*
#6 lore raw

RFC v3 4/42 NFS OFFLOAD_STATUS xdr

In reply to Olga Kornievskaia
---
 fs/nfs/nfs42xdr.c         | 87 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4proc.c         |  1 +
 fs/nfs/nfs4xdr.c          |  1 +
 include/linux/nfs4.h      |  1 +
 include/linux/nfs_fs_sb.h |  1 +
 include/linux/nfs_xdr.h   | 12 +++++++
 6 files changed, 103 insertions(+)
fs/nfs/nfs42xdr.c +87 -0
index 528362f..e369de1 100644
@@ -25,6 +25,11 @@
 					 NFS42_WRITE_RES_SIZE + \
 					 1 /* cr_consecutive */ + \
 					 1 /* cr_synchronous */)
+#define encode_offload_status_maxsz	(op_encode_hdr_maxsz + \
+					 XDR_QUADLEN(NFS4_STATEID_SIZE))
+#define decode_offload_status_maxsz	(op_decode_hdr_maxsz + \
+					 2 /* osr_count */ + \
+					 1 + 1 /* osr_complete<1> */)
 #define encode_deallocate_maxsz		(op_encode_hdr_maxsz + \
 					 encode_fallocate_maxsz)
 #define decode_deallocate_maxsz		(op_decode_hdr_maxsz)
@@ -74,6 +79,12 @@
 					 decode_putfh_maxsz + \
 					 decode_copy_maxsz + \
 					 decode_commit_maxsz)
+#define NFS4_enc_offload_status_sz	(compound_encode_hdr_maxsz + \
+					 encode_putfh_maxsz + \
+					 encode_offload_status_maxsz)
+#define NFS4_dec_offload_status_sz	(compound_decode_hdr_maxsz + \
+					 decode_putfh_maxsz + \
+					 decode_offload_status_maxsz)
 #define NFS4_enc_deallocate_sz		(compound_encode_hdr_maxsz + \
 					 encode_putfh_maxsz + \
 					 encode_deallocate_maxsz + \
@@ -144,6 +155,14 @@ static void encode_copy(struct xdr_stream *xdr,
 	encode_uint32(xdr, 0); /* src server list */
 }
 
+static void encode_offload_status(struct xdr_stream *xdr,
+				  struct nfs42_offload_status_args *args,
+				  struct compound_hdr *hdr)
+{
+	encode_op_hdr(xdr, OP_OFFLOAD_STATUS, decode_offload_status_maxsz, hdr);
+	encode_nfs4_stateid(xdr, &args->osa_stateid);
+}
+
 static void encode_deallocate(struct xdr_stream *xdr,
 			      struct nfs42_falloc_args *args,
 			      struct compound_hdr *hdr)
@@ -258,6 +277,24 @@ static void nfs4_xdr_enc_copy(struct rpc_rqst *req,
 }
 
 /*
+ * Encode OFFLOAD_STATUS request
+ */
+static void nfs4_xdr_enc_offload_status(struct rpc_rqst *req,
+					struct xdr_stream *xdr,
+					struct nfs42_offload_status_args *args)
+{
+	struct compound_hdr hdr = {
+		.minorversion = nfs4_xdr_minorversion(&args->osa_seq_args),
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->osa_seq_args, &hdr);
+	encode_putfh(xdr, args->osa_src_fh, &hdr);
+	encode_offload_status(xdr, args, &hdr);
+	encode_nops(&hdr);
+}
+
+/*
  * Encode DEALLOCATE request
  */
 static void nfs4_xdr_enc_deallocate(struct rpc_rqst *req,
@@ -406,6 +443,31 @@ static int decode_copy(struct xdr_stream *xdr, struct nfs42_copy_res *res)
 	return decode_copy_requirements(xdr, res);
 }
 
+static int decode_offload_status(struct xdr_stream *xdr,
+				 struct nfs42_offload_status_res *res)
+{
+	__be32 *p;
+	uint32_t count;
+	int status;
+
+	status = decode_op_hdr(xdr, OP_OFFLOAD_STATUS);
+	if (status)
+		return status;
+	p = xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out_overflow;
+	p = xdr_decode_hyper(p, &res->osr_count);
+	count = be32_to_cpup(p++);
+	if (count) {
+		p = xdr_inline_decode(xdr, 4);
+		res->osr_status = nfs4_stat_to_errno(be32_to_cpup(p));
+	}
+	return 0;
+out_overflow:
+	print_overflow_msg(__func__, xdr);
+	return -EIO;
+}
+
 static int decode_deallocate(struct xdr_stream *xdr, struct nfs42_falloc_res *res)
 {
 	return decode_op_hdr(xdr, OP_DEALLOCATE);
@@ -504,6 +566,31 @@ static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp,
 }
 
 /*
+ * Decode OFFLOAD_STATUS response
+ */
+static int nfs4_xdr_dec_offload_status(struct rpc_rqst *rqstp,
+				       struct xdr_stream *xdr,
+				       struct nfs42_offload_status_res *res)
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status = decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status = decode_sequence(xdr, &res->osr_seq_res, rqstp);
+	if (status)
+		goto out;
+	status = decode_putfh(xdr);
+	if (status)
+		goto out;
+	status = decode_offload_status(xdr, res);
+
+out:
+	return status;
+}
+
+/*
  * Decode DEALLOCATE request
  */
 static int nfs4_xdr_dec_deallocate(struct rpc_rqst *rqstp,
fs/nfs/nfs4proc.c +1 -0
index dbfa189..90be5c1 100644
@@ -9229,6 +9229,7 @@ static bool nfs4_match_stateid(const nfs4_stateid *s1,
 		| NFS_CAP_ATOMIC_OPEN_V1
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
+		| NFS_CAP_OFFLOAD_STATUS
 		| NFS_CAP_DEALLOCATE
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4xdr.c +1 -0
index 3aebfdc..8b41f43 100644
@@ -7561,6 +7561,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(LAYOUTSTATS,	enc_layoutstats,	dec_layoutstats),
 	PROC(CLONE,		enc_clone,		dec_clone),
 	PROC(COPY,		enc_copy,		dec_copy),
+	PROC(OFFLOAD_STATUS,	enc_offload_status,	dec_offload_status),
 #endif /* CONFIG_NFS_V4_2 */
 };
 
include/linux/nfs4.h +1 -0
index 1b1ca04..d67c350 100644
@@ -523,6 +523,7 @@ enum {
 	NFSPROC4_CLNT_LAYOUTSTATS,
 	NFSPROC4_CLNT_CLONE,
 	NFSPROC4_CLNT_COPY,
+	NFSPROC4_CLNT_OFFLOAD_STATUS,
 };
 
 /* nfs41 types */
include/linux/nfs_fs_sb.h +1 -0
index e418a10..36535ad 100644
@@ -250,5 +250,6 @@ struct nfs_server {
 #define NFS_CAP_LAYOUTSTATS	(1U << 22)
 #define NFS_CAP_CLONE		(1U << 23)
 #define NFS_CAP_COPY		(1U << 24)
+#define NFS_CAP_OFFLOAD_STATUS	(1U << 25)
 
 #endif
include/linux/nfs_xdr.h +12 -0
index b28c834..eaaf540 100644
@@ -1386,6 +1386,18 @@ struct nfs42_copy_res {
 	struct nfs_commitres		commit_res;
 };
 
+struct nfs42_offload_status_args {
+	struct nfs4_sequence_args	osa_seq_args;
+	struct nfs_fh			*osa_src_fh;
+	nfs4_stateid			osa_stateid;
+};
+
+struct nfs42_offload_status_res {
+	struct nfs4_sequence_res	osr_seq_res;
+	uint64_t			osr_count;
+	int				osr_status;
+};
+
 struct nfs42_seek_args {
 	struct nfs4_sequence_args	seq_args;
 
Signature · 1 line
1.8.3.1
#7 lore raw

Re: [RFC v3 04/42] NFS OFFLOAD_STATUS xdr

Hi Olga,

Quoted 55 lines · Olga Kornievskaia
---
 fs/nfs/nfs42xdr.c         | 87 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4proc.c         |  1 +
 fs/nfs/nfs4xdr.c          |  1 +
 include/linux/nfs4.h      |  1 +
 include/linux/nfs_fs_sb.h |  1 +
 include/linux/nfs_xdr.h   | 12 +++++++
 6 files changed, 103 insertions(+)
fs/nfs/nfs42xdr.c +19 -0
index 528362f..e369de1 100644
@@ -25,6 +25,11 @@
 					 NFS42_WRITE_RES_SIZE + \
 					 1 /* cr_consecutive */ + \
 					 1 /* cr_synchronous */)
+#define encode_offload_status_maxsz	(op_encode_hdr_maxsz + \
+					 XDR_QUADLEN(NFS4_STATEID_SIZE))
+#define decode_offload_status_maxsz	(op_decode_hdr_maxsz + \
+					 2 /* osr_count */ + \
+					 1 + 1 /* osr_complete<1> */)
 #define encode_deallocate_maxsz		(op_encode_hdr_maxsz + \
 					 encode_fallocate_maxsz)
 #define decode_deallocate_maxsz		(op_decode_hdr_maxsz)
@@ -74,6 +79,12 @@
 					 decode_putfh_maxsz + \
 					 decode_copy_maxsz + \
 					 decode_commit_maxsz)
+#define NFS4_enc_offload_status_sz	(compound_encode_hdr_maxsz + \
+					 encode_putfh_maxsz + \
+					 encode_offload_status_maxsz)
+#define NFS4_dec_offload_status_sz	(compound_decode_hdr_maxsz + \
+					 decode_putfh_maxsz + \
+					 decode_offload_status_maxsz)
 #define NFS4_enc_deallocate_sz		(compound_encode_hdr_maxsz + \
 					 encode_putfh_maxsz + \
 					 encode_deallocate_maxsz + \
@@ -144,6 +155,14 @@ static void encode_copy(struct xdr_stream *xdr,
 	encode_uint32(xdr, 0); /* src server list */
 }
 
+static void encode_offload_status(struct xdr_stream *xdr,
+				  struct nfs42_offload_status_args *args,
+				  struct compound_hdr *hdr)
+{
+	encode_op_hdr(xdr, OP_OFFLOAD_STATUS, decode_offload_status_maxsz, hdr);
+	encode_nfs4_stateid(xdr, &args->osa_stateid);
+}
+
 static void encode_deallocate(struct xdr_stream *xdr,
 			      struct nfs42_falloc_args *args,
 			      struct compound_hdr *hdr)

fs/nfs/nfs42xdr.c static void nfs4_xdr_enc_copy(struct rpc_rqst *req, · line 284

Quoted 8 lines
fs/nfs/nfs42xdr.c +5 -0
@@ -258,6 +277,24 @@ static void nfs4_xdr_enc_copy(struct rpc_rqst *req,
 }
 
 /*
+ * Encode OFFLOAD_STATUS request
+ */
+static void nfs4_xdr_enc_offload_status(struct rpc_rqst *req,
+					struct xdr_stream *xdr,
+					struct nfs42_offload_status_args *args)

This last argument should now be: "const void *data"

Quoted 49 lines
fs/nfs/nfs42xdr.c +38 -0
+{
+	struct compound_hdr hdr = {
+		.minorversion = nfs4_xdr_minorversion(&args->osa_seq_args),
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->osa_seq_args, &hdr);
+	encode_putfh(xdr, args->osa_src_fh, &hdr);
+	encode_offload_status(xdr, args, &hdr);
+	encode_nops(&hdr);
+}
+
+/*
  * Encode DEALLOCATE request
  */
 static void nfs4_xdr_enc_deallocate(struct rpc_rqst *req,
@@ -406,6 +443,31 @@ static int decode_copy(struct xdr_stream *xdr, struct nfs42_copy_res *res)
 	return decode_copy_requirements(xdr, res);
 }
 
+static int decode_offload_status(struct xdr_stream *xdr,
+				 struct nfs42_offload_status_res *res)
+{
+	__be32 *p;
+	uint32_t count;
+	int status;
+
+	status = decode_op_hdr(xdr, OP_OFFLOAD_STATUS);
+	if (status)
+		return status;
+	p = xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out_overflow;
+	p = xdr_decode_hyper(p, &res->osr_count);
+	count = be32_to_cpup(p++);
+	if (count) {
+		p = xdr_inline_decode(xdr, 4);
+		res->osr_status = nfs4_stat_to_errno(be32_to_cpup(p));
+	}
+	return 0;
+out_overflow:
+	print_overflow_msg(__func__, xdr);
+	return -EIO;
+}
+
 static int decode_deallocate(struct xdr_stream *xdr, struct nfs42_falloc_res *res)
 {
 	return decode_op_hdr(xdr, OP_DEALLOCATE);

fs/nfs/nfs42xdr.c static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp, · line 573

Quoted 8 lines
fs/nfs/nfs42xdr.c +5 -0
@@ -504,6 +566,31 @@ static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp,
 }
 
 /*
+ * Decode OFFLOAD_STATUS response
+ */
+static int nfs4_xdr_dec_offload_status(struct rpc_rqst *rqstp,
+				       struct xdr_stream *xdr,
+				       struct nfs42_offload_status_res *res)

And this one should be "void *data"

Thanks,
Anna

Quoted 94 lines
fs/nfs/nfs42xdr.c +20 -0
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status = decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status = decode_sequence(xdr, &res->osr_seq_res, rqstp);
+	if (status)
+		goto out;
+	status = decode_putfh(xdr);
+	if (status)
+		goto out;
+	status = decode_offload_status(xdr, res);
+
+out:
+	return status;
+}
+
+/*
  * Decode DEALLOCATE request
  */
 static int nfs4_xdr_dec_deallocate(struct rpc_rqst *rqstp,
fs/nfs/nfs4proc.c +1 -0
index dbfa189..90be5c1 100644
@@ -9229,6 +9229,7 @@ static bool nfs4_match_stateid(const nfs4_stateid *s1,
 		| NFS_CAP_ATOMIC_OPEN_V1
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
+		| NFS_CAP_OFFLOAD_STATUS
 		| NFS_CAP_DEALLOCATE
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4xdr.c +1 -0
index 3aebfdc..8b41f43 100644
@@ -7561,6 +7561,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(LAYOUTSTATS,	enc_layoutstats,	dec_layoutstats),
 	PROC(CLONE,		enc_clone,		dec_clone),
 	PROC(COPY,		enc_copy,		dec_copy),
+	PROC(OFFLOAD_STATUS,	enc_offload_status,	dec_offload_status),
 #endif /* CONFIG_NFS_V4_2 */
 };
 
include/linux/nfs4.h +1 -0
index 1b1ca04..d67c350 100644
@@ -523,6 +523,7 @@ enum {
 	NFSPROC4_CLNT_LAYOUTSTATS,
 	NFSPROC4_CLNT_CLONE,
 	NFSPROC4_CLNT_COPY,
+	NFSPROC4_CLNT_OFFLOAD_STATUS,
 };
 
 /* nfs41 types */
include/linux/nfs_fs_sb.h +1 -0
index e418a10..36535ad 100644
@@ -250,5 +250,6 @@ struct nfs_server {
 #define NFS_CAP_LAYOUTSTATS	(1U << 22)
 #define NFS_CAP_CLONE		(1U << 23)
 #define NFS_CAP_COPY		(1U << 24)
+#define NFS_CAP_OFFLOAD_STATUS	(1U << 25)
 
 #endif
include/linux/nfs_xdr.h +12 -0
index b28c834..eaaf540 100644
@@ -1386,6 +1386,18 @@ struct nfs42_copy_res {
 	struct nfs_commitres		commit_res;
 };
 
+struct nfs42_offload_status_args {
+	struct nfs4_sequence_args	osa_seq_args;
+	struct nfs_fh			*osa_src_fh;
+	nfs4_stateid			osa_stateid;
+};
+
+struct nfs42_offload_status_res {
+	struct nfs4_sequence_res	osr_seq_res;
+	uint64_t			osr_count;
+	int				osr_status;
+};
+
 struct nfs42_seek_args {
 	struct nfs4_sequence_args	seq_args;
 
#8 lore raw

Re: [RFC v3 04/42] NFS OFFLOAD_STATUS xdr

Is there an ETA on when the casting patches will go in? If it=E2=80=99s = soon, I would rebase then and fix the new calls.

Quoted 1 line

On Jul 11, 2017, at 5:01 PM, Anna Schumaker <[email protected]> =

Quoted 7 lines · wrote:

=20
Hi Olga,
=20

Quoted 3 lines · Olga Kornievskaia

---
fs/nfs/nfs42xdr.c | 87 =

+++++++++++++++++++++++++++++++++++++++++++++++

Quoted 26 lines
Quoted 26 lines
fs/nfs/nfs4proc.c         |  1 +
fs/nfs/nfs4xdr.c          |  1 +
include/linux/nfs4.h      |  1 +
include/linux/nfs_fs_sb.h |  1 +
include/linux/nfs_xdr.h   | 12 +++++++
6 files changed, 103 insertions(+)

=20

fs/nfs/nfs42xdr.c +5 -0
index 528362f..e369de1 100644
@@ -25,6 +25,11 @@
 					 NFS42_WRITE_RES_SIZE + \
 					 1 /* cr_consecutive */ + \
 					 1 /* cr_synchronous */)
+#define encode_offload_status_maxsz	(op_encode_hdr_maxsz + \
+					 XDR_QUADLEN(NFS4_STATEID_SIZE))
+#define decode_offload_status_maxsz	(op_decode_hdr_maxsz + \
+					 2 /* osr_count */ + \
+					 1 + 1 /* osr_complete<1> */)
 #define encode_deallocate_maxsz		(op_encode_hdr_maxsz + \
 					 encode_fallocate_maxsz)
 #define decode_deallocate_maxsz		(op_decode_hdr_maxsz)
@@ -74,6 +79,12 @@
 					 decode_putfh_maxsz + \
 					 decode_copy_maxsz + \
Quoted 8 lines

fs/nfs/nfs42xdr.c

Quoted 8 lines
fs/nfs/nfs42xdr.c +6 -0
 					 decode_commit_maxsz)
+#define NFS4_enc_offload_status_sz	(compound_encode_hdr_maxsz + \
+					 encode_putfh_maxsz + \
+					 encode_offload_status_maxsz)
+#define NFS4_dec_offload_status_sz	(compound_decode_hdr_maxsz + \
+					 decode_putfh_maxsz + \
+					 decode_offload_status_maxsz)
 #define NFS4_enc_deallocate_sz		=

(compound_encode_hdr_maxsz + \

Quoted 8 lines

fs/nfs/nfs42xdr.c

Quoted 8 lines
fs/nfs/nfs42xdr.c +0 -0
 					 encode_putfh_maxsz + \
 					 encode_deallocate_maxsz + \
@@ -144,6 +155,14 @@ static void encode_copy(struct xdr_stream *xdr,
 	encode_uint32(xdr, 0); /* src server list */
 }

=20

fs/nfs/nfs42xdr.c +2 -0
+static void encode_offload_status(struct xdr_stream *xdr,
+				  struct nfs42_offload_status_args =

*args,

Quoted 3 lines

fs/nfs/nfs42xdr.c static void encode_copy(struct xdr_stream *xdr,

Quoted 3 lines
fs/nfs/nfs42xdr.c +3 -0
+				  struct compound_hdr *hdr)
+{
+	encode_op_hdr(xdr, OP_OFFLOAD_STATUS, =

decode_offload_status_maxsz, hdr);

Quoted 7 lines

fs/nfs/nfs42xdr.c static void nfs4_xdr_enc_copy(struct rpc_rqst =

Quoted 7 lines
fs/nfs/nfs42xdr.c +3 -0
+	encode_nfs4_stateid(xdr, &args->osa_stateid);
+}
+

static void encode_deallocate(struct xdr_stream *xdr,
struct nfs42_falloc_args *args, struct compound_hdr *hdr)

fs/nfs/nfs42xdr.c +0 -0
@@ -258,6 +277,24 @@ static void nfs4_xdr_enc_copy(struct rpc_rqst =

*req,

Quoted 8 lines

fs/nfs/nfs42xdr.c static void nfs4_xdr_enc_copy(struct rpc_rqst *req,

Quoted 8 lines
fs/nfs/nfs42xdr.c +0 -0
 }

=20
/*
+ * Encode OFFLOAD_STATUS request
+ */

fs/nfs/nfs42xdr.c +3 -0
+static void nfs4_xdr_enc_offload_status(struct rpc_rqst *req,
+					struct xdr_stream *xdr,
+					struct nfs42_offload_status_args =

*args)

Quoted 6 lines

=20
This last argument should now be: "const void *data" =20

fs/nfs/nfs42xdr.c

Quoted 3 lines
fs/nfs/nfs42xdr.c +3 -0
+{
+	struct compound_hdr hdr =3D {
+		.minorversion =3D =

nfs4_xdr_minorversion(&args->osa_seq_args),

Quoted 6 lines
Quoted 6 lines
fs/nfs/nfs42xdr.c +6 -0
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->osa_seq_args, &hdr);
+	encode_putfh(xdr, args->osa_src_fh, &hdr);
+	encode_offload_status(xdr, args, &hdr);
Quoted 8 lines

fs/nfs/nfs42xdr.c static int decode_copy(struct xdr_stream *xdr, =

Quoted 8 lines
fs/nfs/nfs42xdr.c +4 -0
+	encode_nops(&hdr);
+}
+
+/*
 * Encode DEALLOCATE request
 */

static void nfs4_xdr_enc_deallocate(struct rpc_rqst *req,

fs/nfs/nfs42xdr.c +0 -0
@@ -406,6 +443,31 @@ static int decode_copy(struct xdr_stream *xdr, =

struct nfs42_copy_res *res)

Quoted 21 lines
Quoted 21 lines
fs/nfs/nfs42xdr.c +0 -0
 	return decode_copy_requirements(xdr, res);
 }

=20

fs/nfs/nfs42xdr.c +18 -0
+static int decode_offload_status(struct xdr_stream *xdr,
+				 struct nfs42_offload_status_res *res)
+{
+	__be32 *p;
+	uint32_t count;
+	int status;
+
+	status =3D decode_op_hdr(xdr, OP_OFFLOAD_STATUS);
+	if (status)
+		return status;
+	p =3D xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out_overflow;
+	p =3D xdr_decode_hyper(p, &res->osr_count);
+	count =3D be32_to_cpup(p++);
+	if (count) {
+		p =3D xdr_inline_decode(xdr, 4);
+		res->osr_status =3D nfs4_stat_to_errno(be32_to_cpup(p));
Quoted 8 lines
Quoted 8 lines
fs/nfs/nfs42xdr.c +7 -0
+	}
+	return 0;
+out_overflow:
+	print_overflow_msg(__func__, xdr);
+	return -EIO;
+}
+

static int decode_deallocate(struct xdr_stream *xdr, struct =

nfs42_falloc_res *res)

Quoted 3 lines

fs/nfs/nfs42xdr.c static int nfs4_xdr_dec_copy(struct rpc_rqst =

Quoted 3 lines
{
	return decode_op_hdr(xdr, OP_DEALLOCATE);
fs/nfs/nfs42xdr.c +0 -0
@@ -504,6 +566,31 @@ static int nfs4_xdr_dec_copy(struct rpc_rqst =

*rqstp,

Quoted 8 lines

fs/nfs/nfs42xdr.c static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp,

Quoted 8 lines
fs/nfs/nfs42xdr.c +0 -0
 }

=20
/*
+ * Decode OFFLOAD_STATUS response
+ */

fs/nfs/nfs42xdr.c +3 -0
+static int nfs4_xdr_dec_offload_status(struct rpc_rqst *rqstp,
+				       struct xdr_stream *xdr,
+				       struct nfs42_offload_status_res =

*res)

Quoted 26 lines

=20
And this one should be "void *data"
=20
Thanks,
Anna
=20

Quoted 20 lines
fs/nfs/nfs42xdr.c +20 -0
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status =3D decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status =3D decode_sequence(xdr, &res->osr_seq_res, rqstp);
+	if (status)
+		goto out;
+	status =3D decode_putfh(xdr);
+	if (status)
+		goto out;
+	status =3D decode_offload_status(xdr, res);
+
+out:
+	return status;
+}
+
+/*
Quoted 8 lines

fs/nfs/nfs4proc.c static bool nfs4_match_stateid(const =

Quoted 8 lines

* Decode DEALLOCATE request
*/
static int nfs4_xdr_dec_deallocate(struct rpc_rqst *rqstp,

fs/nfs/nfs4proc.c +0 -0
index dbfa189..90be5c1 100644
@@ -9229,6 +9229,7 @@ static bool nfs4_match_stateid(const =

nfs4_stateid *s1,

Quoted 5 lines
Quoted 5 lines
fs/nfs/nfs4proc.c +1 -0
 		| NFS_CAP_ATOMIC_OPEN_V1
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
+		| NFS_CAP_OFFLOAD_STATUS
 		| NFS_CAP_DEALLOCATE
Quoted 8 lines

fs/nfs/nfs4xdr.c struct rpc_procinfo nfs4_procedures[] =3D { · line 7561

Quoted 8 lines
fs/nfs/nfs4proc.c +0 -0
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4xdr.c +0 -0
index 3aebfdc..8b41f43 100644
@@ -7561,6 +7561,7 @@ struct rpc_procinfo	nfs4_procedures[] =3D {
 	PROC(LAYOUTSTATS,	enc_layoutstats,	=

dec_layoutstats),

Quoted 3 lines

fs/nfs/nfs4xdr.c struct rpc_procinfo nfs4_procedures[] = {

Quoted 3 lines
fs/nfs/nfs4xdr.c +1 -0
 	PROC(CLONE,		enc_clone,		dec_clone),
 	PROC(COPY,		enc_copy,		dec_copy),
+	PROC(OFFLOAD_STATUS,	enc_offload_status,	=

dec_offload_status),

Quoted 50 lines
Quoted 50 lines
fs/nfs/nfs4xdr.c +0 -0
 #endif /* CONFIG_NFS_V4_2 */
 };

=20

include/linux/nfs4.h +1 -0
index 1b1ca04..d67c350 100644
@@ -523,6 +523,7 @@ enum {
 	NFSPROC4_CLNT_LAYOUTSTATS,
 	NFSPROC4_CLNT_CLONE,
 	NFSPROC4_CLNT_COPY,
+	NFSPROC4_CLNT_OFFLOAD_STATUS,
 };

=20
/* nfs41 types */

include/linux/nfs_fs_sb.h +1 -0
index e418a10..36535ad 100644
@@ -250,5 +250,6 @@ struct nfs_server {
 #define NFS_CAP_LAYOUTSTATS	(1U << 22)
 #define NFS_CAP_CLONE		(1U << 23)
 #define NFS_CAP_COPY		(1U << 24)
+#define NFS_CAP_OFFLOAD_STATUS	(1U << 25)

=20
#endif

include/linux/nfs_xdr.h +0 -0
index b28c834..eaaf540 100644
@@ -1386,6 +1386,18 @@ struct nfs42_copy_res {
 	struct nfs_commitres		commit_res;
 };

=20

include/linux/nfs_xdr.h +12 -0
+struct nfs42_offload_status_args {
+	struct nfs4_sequence_args	osa_seq_args;
+	struct nfs_fh			*osa_src_fh;
+	nfs4_stateid			osa_stateid;
+};
+
+struct nfs42_offload_status_res {
+	struct nfs4_sequence_res	osr_seq_res;
+	uint64_t			osr_count;
+	int				osr_status;
+};
+
struct nfs42_seek_args {
	struct nfs4_sequence_args	seq_args;

=20
=20

#9 lore raw

Re: [RFC v3 04/42] NFS OFFLOAD_STATUS xdr

Quoted 2 lines · Olga Kornievskaia

Is there an ETA on when the casting patches will go in? If it’s soon, I would rebase then and fix the new calls.

It goes in as part of the current merge window, so hold off until 4.13-rc1 :)

Quoted 230 lines
Quoted 229 lines · Anna Schumaker

Hi Olga,

Quoted 55 lines · Olga Kornievskaia

---

fs/nfs/nfs42xdr.c         | 87 +++++++++++++++++++++++++++++++++++++++++++++++
fs/nfs/nfs4proc.c         |  1 +
fs/nfs/nfs4xdr.c          |  1 +
include/linux/nfs4.h      |  1 +
include/linux/nfs_fs_sb.h |  1 +
include/linux/nfs_xdr.h   | 12 +++++++
6 files changed, 103 insertions(+)
fs/nfs/nfs42xdr.c +19 -0
index 528362f..e369de1 100644
@@ -25,6 +25,11 @@
 					 NFS42_WRITE_RES_SIZE + \
 					 1 /* cr_consecutive */ + \
 					 1 /* cr_synchronous */)
+#define encode_offload_status_maxsz	(op_encode_hdr_maxsz + \
+					 XDR_QUADLEN(NFS4_STATEID_SIZE))
+#define decode_offload_status_maxsz	(op_decode_hdr_maxsz + \
+					 2 /* osr_count */ + \
+					 1 + 1 /* osr_complete<1> */)
 #define encode_deallocate_maxsz		(op_encode_hdr_maxsz + \
 					 encode_fallocate_maxsz)
 #define decode_deallocate_maxsz		(op_decode_hdr_maxsz)
@@ -74,6 +79,12 @@
 					 decode_putfh_maxsz + \
 					 decode_copy_maxsz + \
 					 decode_commit_maxsz)
+#define NFS4_enc_offload_status_sz	(compound_encode_hdr_maxsz + \
+					 encode_putfh_maxsz + \
+					 encode_offload_status_maxsz)
+#define NFS4_dec_offload_status_sz	(compound_decode_hdr_maxsz + \
+					 decode_putfh_maxsz + \
+					 decode_offload_status_maxsz)
 #define NFS4_enc_deallocate_sz		(compound_encode_hdr_maxsz + \
 					 encode_putfh_maxsz + \
 					 encode_deallocate_maxsz + \
@@ -144,6 +155,14 @@ static void encode_copy(struct xdr_stream *xdr,
 	encode_uint32(xdr, 0); /* src server list */
 }
 
+static void encode_offload_status(struct xdr_stream *xdr,
+				  struct nfs42_offload_status_args *args,
+				  struct compound_hdr *hdr)
+{
+	encode_op_hdr(xdr, OP_OFFLOAD_STATUS, decode_offload_status_maxsz, hdr);
+	encode_nfs4_stateid(xdr, &args->osa_stateid);
+}
+
 static void encode_deallocate(struct xdr_stream *xdr,
 			      struct nfs42_falloc_args *args,
 			      struct compound_hdr *hdr)

fs/nfs/nfs42xdr.c static void nfs4_xdr_enc_copy(struct rpc_rqst *req, · line 284

Quoted 8 lines
fs/nfs/nfs42xdr.c +5 -0
@@ -258,6 +277,24 @@ static void nfs4_xdr_enc_copy(struct rpc_rqst *req,
 }
 
 /*
+ * Encode OFFLOAD_STATUS request
+ */
+static void nfs4_xdr_enc_offload_status(struct rpc_rqst *req,
+					struct xdr_stream *xdr,
+					struct nfs42_offload_status_args *args)

This last argument should now be: "const void *data"

Quoted 49 lines
fs/nfs/nfs42xdr.c +38 -0
+{
+	struct compound_hdr hdr = {
+		.minorversion = nfs4_xdr_minorversion(&args->osa_seq_args),
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->osa_seq_args, &hdr);
+	encode_putfh(xdr, args->osa_src_fh, &hdr);
+	encode_offload_status(xdr, args, &hdr);
+	encode_nops(&hdr);
+}
+
+/*
 * Encode DEALLOCATE request
 */
 static void nfs4_xdr_enc_deallocate(struct rpc_rqst *req,
@@ -406,6 +443,31 @@ static int decode_copy(struct xdr_stream *xdr, struct nfs42_copy_res *res)
 	return decode_copy_requirements(xdr, res);
 }
 
+static int decode_offload_status(struct xdr_stream *xdr,
+				 struct nfs42_offload_status_res *res)
+{
+	__be32 *p;
+	uint32_t count;
+	int status;
+
+	status = decode_op_hdr(xdr, OP_OFFLOAD_STATUS);
+	if (status)
+		return status;
+	p = xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out_overflow;
+	p = xdr_decode_hyper(p, &res->osr_count);
+	count = be32_to_cpup(p++);
+	if (count) {
+		p = xdr_inline_decode(xdr, 4);
+		res->osr_status = nfs4_stat_to_errno(be32_to_cpup(p));
+	}
+	return 0;
+out_overflow:
+	print_overflow_msg(__func__, xdr);
+	return -EIO;
+}
+
 static int decode_deallocate(struct xdr_stream *xdr, struct nfs42_falloc_res *res)
 {
 	return decode_op_hdr(xdr, OP_DEALLOCATE);

fs/nfs/nfs42xdr.c static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp, · line 573

Quoted 8 lines
fs/nfs/nfs42xdr.c +5 -0
@@ -504,6 +566,31 @@ static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp,
 }
 
 /*
+ * Decode OFFLOAD_STATUS response
+ */
+static int nfs4_xdr_dec_offload_status(struct rpc_rqst *rqstp,
+				       struct xdr_stream *xdr,
+				       struct nfs42_offload_status_res *res)

And this one should be "void *data"

Thanks,
Anna

Quoted 94 lines
fs/nfs/nfs42xdr.c +20 -0
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status = decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status = decode_sequence(xdr, &res->osr_seq_res, rqstp);
+	if (status)
+		goto out;
+	status = decode_putfh(xdr);
+	if (status)
+		goto out;
+	status = decode_offload_status(xdr, res);
+
+out:
+	return status;
+}
+
+/*
 * Decode DEALLOCATE request
 */
 static int nfs4_xdr_dec_deallocate(struct rpc_rqst *rqstp,
fs/nfs/nfs4proc.c +1 -0
index dbfa189..90be5c1 100644
@@ -9229,6 +9229,7 @@ static bool nfs4_match_stateid(const nfs4_stateid *s1,
 		| NFS_CAP_ATOMIC_OPEN_V1
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
+		| NFS_CAP_OFFLOAD_STATUS
 		| NFS_CAP_DEALLOCATE
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4xdr.c +1 -0
index 3aebfdc..8b41f43 100644
@@ -7561,6 +7561,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(LAYOUTSTATS,	enc_layoutstats,	dec_layoutstats),
 	PROC(CLONE,		enc_clone,		dec_clone),
 	PROC(COPY,		enc_copy,		dec_copy),
+	PROC(OFFLOAD_STATUS,	enc_offload_status,	dec_offload_status),
 #endif /* CONFIG_NFS_V4_2 */
 };
 
include/linux/nfs4.h +1 -0
index 1b1ca04..d67c350 100644
@@ -523,6 +523,7 @@ enum {
 	NFSPROC4_CLNT_LAYOUTSTATS,
 	NFSPROC4_CLNT_CLONE,
 	NFSPROC4_CLNT_COPY,
+	NFSPROC4_CLNT_OFFLOAD_STATUS,
 };
 
 /* nfs41 types */
include/linux/nfs_fs_sb.h +1 -0
index e418a10..36535ad 100644
@@ -250,5 +250,6 @@ struct nfs_server {
 #define NFS_CAP_LAYOUTSTATS	(1U << 22)
 #define NFS_CAP_CLONE		(1U << 23)
 #define NFS_CAP_COPY		(1U << 24)
+#define NFS_CAP_OFFLOAD_STATUS	(1U << 25)
 
 #endif
include/linux/nfs_xdr.h +12 -0
index b28c834..eaaf540 100644
@@ -1386,6 +1386,18 @@ struct nfs42_copy_res {
 	struct nfs_commitres		commit_res;
 };
 
+struct nfs42_offload_status_args {
+	struct nfs4_sequence_args	osa_seq_args;
+	struct nfs_fh			*osa_src_fh;
+	nfs4_stateid			osa_stateid;
+};
+
+struct nfs42_offload_status_res {
+	struct nfs4_sequence_res	osr_seq_res;
+	uint64_t			osr_count;
+	int				osr_status;
+};
+
 struct nfs42_seek_args {
 	struct nfs4_sequence_args	seq_args;
 
#10 lore raw

RFC v3 5/42 NFS OFFLOAD_STATUS op

In reply to Olga Kornievskaia
---
 fs/nfs/nfs42.h     |  5 ++++-
 fs/nfs/nfs42proc.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 1 deletion(-)
fs/nfs/nfs42.h +4 -1
index b6cd153..54b9941 100644
@@ -12,6 +12,7 @@
 #define PNFS_LAYOUTSTATS_MAXDEV (4)
 
 /* nfs4.2proc.c */
+#if defined(CONFIG_NFS_V4_2)
 int nfs42_proc_allocate(struct file *, loff_t, loff_t);
 ssize_t nfs42_proc_copy(struct file *, loff_t, struct file *, loff_t, size_t);
 int nfs42_proc_deallocate(struct file *, loff_t, loff_t);
@@ -19,5 +20,7 @@
 int nfs42_proc_layoutstats_generic(struct nfs_server *,
 				   struct nfs42_layoutstat_data *);
 int nfs42_proc_clone(struct file *, struct file *, loff_t, loff_t, loff_t);
-
+int nfs42_proc_offload_status(struct file *, nfs4_stateid *,
+			      struct nfs42_offload_status_res *);
+#endif /* CONFIG_NFS_V4_2) */
 #endif /* __LINUX_FS_NFS_NFS4_2_H */
fs/nfs/nfs42proc.c +42 -0

48 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#11 lore raw

Re: [RFC v3 05/42] NFS OFFLOAD_STATUS op

Hi Olga,

Quoted 11 lines · Olga Kornievskaia
---
 fs/nfs/nfs42.h     |  5 ++++-
 fs/nfs/nfs42proc.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 1 deletion(-)
fs/nfs/nfs42.h +0 -0
index b6cd153..54b9941 100644

fs/nfs/nfs42.h line 15

Quoted 4 lines
fs/nfs/nfs42.h +1 -0
@@ -12,6 +12,7 @@
 #define PNFS_LAYOUTSTATS_MAXDEV (4)
 
 /* nfs4.2proc.c */
+#if defined(CONFIG_NFS_V4_2)

This is just me being nitpicky. Can you move the "#if defined()" check to before the "/* nfs4.2proc.c */" comment? I think it would make the code easier to follow, especially if we ever add more v4.2 files.

Thanks,
Anna

Quoted 66 lines
fs/nfs/nfs42.h +3 -1
 int nfs42_proc_allocate(struct file *, loff_t, loff_t);
 ssize_t nfs42_proc_copy(struct file *, loff_t, struct file *, loff_t, size_t);
 int nfs42_proc_deallocate(struct file *, loff_t, loff_t);
@@ -19,5 +20,7 @@
 int nfs42_proc_layoutstats_generic(struct nfs_server *,
 				   struct nfs42_layoutstat_data *);
 int nfs42_proc_clone(struct file *, struct file *, loff_t, loff_t, loff_t);
-
+int nfs42_proc_offload_status(struct file *, nfs4_stateid *,
+			      struct nfs42_offload_status_res *);
+#endif /* CONFIG_NFS_V4_2) */
 #endif /* __LINUX_FS_NFS_NFS4_2_H */
fs/nfs/nfs42proc.c +42 -0
index 6c2db51..5c42e71 100644
@@ -263,6 +263,48 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 	return err;
 }
 
+int _nfs42_proc_offload_status(struct file *dst, nfs4_stateid *stateid,
+				struct nfs42_offload_status_res *res)
+{
+	struct nfs42_offload_status_args args;
+	struct nfs_server *dst_server = NFS_SERVER(file_inode(dst));
+	struct rpc_message msg = {
+		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_OFFLOAD_STATUS],
+		.rpc_resp = res,
+		.rpc_argp = &args,
+	};
+	int status;
+
+	args.osa_src_fh = NFS_FH(file_inode(dst));
+	memcpy(&args.osa_stateid, stateid, sizeof(args.osa_stateid));
+	status = nfs4_call_sync(dst_server->client, dst_server, &msg,
+				&args.osa_seq_args, &res->osr_seq_res, 0);
+	if (status == -ENOTSUPP)
+		dst_server->caps &= ~NFS_CAP_OFFLOAD_STATUS;
+
+	return status;
+}
+
+int nfs42_proc_offload_status(struct file *dst, nfs4_stateid *stateid,
+				struct nfs42_offload_status_res *res)
+{
+	struct nfs_server *dst_server = NFS_SERVER(file_inode(dst));
+	struct nfs4_exception exception = { };
+	int status;
+
+	if (!(dst_server->caps & NFS_CAP_OFFLOAD_STATUS))
+		return -EOPNOTSUPP;
+
+	do {
+		status = _nfs42_proc_offload_status(dst, stateid, res);
+		if (status == -ENOTSUPP)
+			return -EOPNOTSUPP;
+		status = nfs4_handle_exception(dst_server, status, &exception);
+	} while (exception.retry);
+
+	return status;
+}
+
 static loff_t _nfs42_proc_llseek(struct file *filep,
 		struct nfs_lock_context *lock, loff_t offset, int whence)
 {
#12 lore raw

RFC v3 6/42 NFS OFFLOAD_CANCEL xdr

In reply to Olga Kornievskaia
---
 fs/nfs/nfs42xdr.c         | 66 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4proc.c         |  1 +
 fs/nfs/nfs4xdr.c          |  1 +
 include/linux/nfs4.h      |  1 +
 include/linux/nfs_fs_sb.h |  1 +
 5 files changed, 70 insertions(+)
fs/nfs/nfs42xdr.c +66 -0
index e369de1..3c6ca02 100644
@@ -30,6 +30,9 @@
 #define decode_offload_status_maxsz	(op_decode_hdr_maxsz + \
 					 2 /* osr_count */ + \
 					 1 + 1 /* osr_complete<1> */)
+#define encode_offload_cancel_maxsz	(op_encode_hdr_maxsz + \
+					 XDR_QUADLEN(NFS4_STATEID_SIZE))
+#define decode_offload_cancel_maxsz	(op_decode_hdr_maxsz)
 #define encode_deallocate_maxsz		(op_encode_hdr_maxsz + \
 					 encode_fallocate_maxsz)
 #define decode_deallocate_maxsz		(op_decode_hdr_maxsz)
@@ -85,6 +88,12 @@
 #define NFS4_dec_offload_status_sz	(compound_decode_hdr_maxsz + \
 					 decode_putfh_maxsz + \
 					 decode_offload_status_maxsz)
+#define NFS4_enc_offload_cancel_sz	(compound_encode_hdr_maxsz + \
+					 encode_putfh_maxsz + \
+					 encode_offload_cancel_maxsz)
+#define NFS4_dec_offload_cancel_sz	(compound_decode_hdr_maxsz + \
+					 decode_putfh_maxsz + \
+					 decode_offload_cancel_maxsz)
 #define NFS4_enc_deallocate_sz		(compound_encode_hdr_maxsz + \
 					 encode_putfh_maxsz + \
 					 encode_deallocate_maxsz + \
@@ -163,6 +172,14 @@ static void encode_offload_status(struct xdr_stream *xdr,
 	encode_nfs4_stateid(xdr, &args->osa_stateid);
 }
 
+static void encode_offload_cancel(struct xdr_stream *xdr,
+				  struct nfs42_offload_status_args *args,
+				  struct compound_hdr *hdr)
+{
+	encode_op_hdr(xdr, OP_OFFLOAD_CANCEL, decode_offload_cancel_maxsz, hdr);
+	encode_nfs4_stateid(xdr, &args->osa_stateid);
+}
+
 static void encode_deallocate(struct xdr_stream *xdr,
 			      struct nfs42_falloc_args *args,
 			      struct compound_hdr *hdr)
@@ -295,6 +312,24 @@ static void nfs4_xdr_enc_offload_status(struct rpc_rqst *req,
 }
 
 /*
+ * Encode OFFLOAD_CANEL request
+ */
+static void nfs4_xdr_enc_offload_cancel(struct rpc_rqst *req,
+					struct xdr_stream *xdr,
+					struct nfs42_offload_status_args *args)
+{
+	struct compound_hdr hdr = {
+		.minorversion = nfs4_xdr_minorversion(&args->osa_seq_args),
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->osa_seq_args, &hdr);
+	encode_putfh(xdr, args->osa_src_fh, &hdr);
+	encode_offload_cancel(xdr, args, &hdr);
+	encode_nops(&hdr);
+}
+
+/*
  * Encode DEALLOCATE request
  */
 static void nfs4_xdr_enc_deallocate(struct rpc_rqst *req,
@@ -468,6 +503,12 @@ static int decode_offload_status(struct xdr_stream *xdr,
 	return -EIO;
 }
 
+static int decode_offload_cancel(struct xdr_stream *xdr,
+				 struct nfs42_offload_status_res *res)
+{
+	return decode_op_hdr(xdr, OP_OFFLOAD_CANCEL);
+}
+
 static int decode_deallocate(struct xdr_stream *xdr, struct nfs42_falloc_res *res)
 {
 	return decode_op_hdr(xdr, OP_DEALLOCATE);
@@ -591,6 +632,31 @@ static int nfs4_xdr_dec_offload_status(struct rpc_rqst *rqstp,
 }
 
 /*
+ * Decode OFFLOAD_CANCEL response
+ */
+static int nfs4_xdr_dec_offload_cancel(struct rpc_rqst *rqstp,
+				       struct xdr_stream *xdr,
+				       struct nfs42_offload_status_res *res)
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status = decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status = decode_sequence(xdr, &res->osr_seq_res, rqstp);
+	if (status)
+		goto out;
+	status = decode_putfh(xdr);
+	if (status)
+		goto out;
+	status = decode_offload_cancel(xdr, res);
+
+out:
+	return status;
+}
+
+/*
  * Decode DEALLOCATE request
  */
 static int nfs4_xdr_dec_deallocate(struct rpc_rqst *rqstp,
fs/nfs/nfs4proc.c +1 -0
index 90be5c1..b37e528 100644
@@ -9230,6 +9230,7 @@ static bool nfs4_match_stateid(const nfs4_stateid *s1,
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
 		| NFS_CAP_OFFLOAD_STATUS
+		| NFS_CAP_OFFLOAD_CANCEL
 		| NFS_CAP_DEALLOCATE
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4xdr.c +1 -0
index 8b41f43..9b66053 100644
@@ -7562,6 +7562,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(CLONE,		enc_clone,		dec_clone),
 	PROC(COPY,		enc_copy,		dec_copy),
 	PROC(OFFLOAD_STATUS,	enc_offload_status,	dec_offload_status),
+	PROC(OFFLOAD_CANCEL,    enc_offload_cancel,     dec_offload_cancel),
 #endif /* CONFIG_NFS_V4_2 */
 };
 
include/linux/nfs4.h +1 -0
index d67c350..7262908 100644
@@ -524,6 +524,7 @@ enum {
 	NFSPROC4_CLNT_CLONE,
 	NFSPROC4_CLNT_COPY,
 	NFSPROC4_CLNT_OFFLOAD_STATUS,
+	NFSPROC4_CLNT_OFFLOAD_CANCEL,
 };
 
 /* nfs41 types */
include/linux/nfs_fs_sb.h +1 -0
index 36535ad..41a2af5 100644
@@ -251,5 +251,6 @@ struct nfs_server {
 #define NFS_CAP_CLONE		(1U << 23)
 #define NFS_CAP_COPY		(1U << 24)
 #define NFS_CAP_OFFLOAD_STATUS	(1U << 25)
+#define NFS_CAP_OFFLOAD_CANCEL	(1U << 26)
 
 #endif
Signature · 1 line
1.8.3.1
#13 lore raw

RFC v3 7/42 NFS COPY xdr handle async reply

In reply to Olga Kornievskaia

If server returns async reply, it must include a callback stateid, wr_callback_id in the write_response4.

---
 fs/nfs/nfs42xdr.c       | 22 ++++++++++++----------
 include/linux/nfs_xdr.h |  1 +
 2 files changed, 13 insertions(+), 10 deletions(-)
fs/nfs/nfs42xdr.c +12 -10
index 3c6ca02..27da108 100644
@@ -418,21 +418,23 @@ static int decode_write_response(struct xdr_stream *xdr,
 				 struct nfs42_write_res *res)
 {
 	__be32 *p;
+	int status, count;
 
-	p = xdr_inline_decode(xdr, 4 + 8 + 4);
+	p = xdr_inline_decode(xdr, 4);
 	if (unlikely(!p))
 		goto out_overflow;
-
-	/*
-	 * We never use asynchronous mode, so warn if a server returns
-	 * a stateid.
-	 */
-	if (unlikely(*p != 0)) {
-		pr_err_once("%s: server has set unrequested "
-				"asynchronous mode\n", __func__);
+	count = be32_to_cpup(p);
+	if (count > 1)
 		return -EREMOTEIO;
+	else if (count == 1) {
+		status = decode_opaque_fixed(xdr, &res->stateid,
+				NFS4_STATEID_SIZE);
+		if (unlikely(status))
+			goto out_overflow;
 	}
-	p++;
+	p = xdr_inline_decode(xdr, 8 + 4);
+	if (unlikely(!p))
+		goto out_overflow;
 	p = xdr_decode_hyper(p, &res->count);
 	res->verifier.committed = be32_to_cpup(p);
 	return decode_verifier(xdr, &res->verifier.verifier);
include/linux/nfs_xdr.h +1 -0
index eaaf540..60eafa0 100644
@@ -1374,6 +1374,7 @@ struct nfs42_copy_args {
 };
 
 struct nfs42_write_res {
+	nfs4_stateid		stateid;
 	u64			count;
 	struct nfs_writeverf	verifier;
 };
Signature · 1 line
1.8.3.1
#14 lore raw

RFC v3 8/42 NFS add support for asynchronous COPY

In reply to Olga Kornievskaia

Change xdr to always send COPY asynchronously.

Keep the list copies send in a list under a server structure. Once copy is sent, it waits on a completion structure that will be signalled by the callback thread that receives CB_OFFLOAD.

---
 fs/nfs/callback_proc.c    | 34 ++++++++++++++++++++++++++++
 fs/nfs/client.c           |  1 +
 fs/nfs/nfs42proc.c        | 56 ++++++++++++++++++++++++++++++++++++++++++-----
 fs/nfs/nfs42xdr.c         |  8 ++++---
 include/linux/nfs_fs.h    |  9 ++++++++
 include/linux/nfs_fs_sb.h |  1 +
 include/linux/nfs_xdr.h   |  1 +
 7 files changed, 102 insertions(+), 8 deletions(-)
fs/nfs/callback_proc.c +34 -0
index 5396751..7e211ed 100644
@@ -648,9 +648,43 @@ __be32 nfs4_callback_notify_lock(struct cb_notify_lock_args *args, void *dummy,
 }
 #endif /* CONFIG_NFS_V4_1 */
 #ifdef CONFIG_NFS_V4_2
+static void nfs4_copy_cb_args(struct nfs4_copy_state *cp_state,
+				struct cb_offloadargs *args)
+{
+	cp_state->count = args->wr_count;
+	cp_state->error = args->error;
+	if (!args->error) {
+		cp_state->verf.committed = args->wr_writeverf.committed;
+		memcpy(&cp_state->verf.verifier.data[0],
+			&args->wr_writeverf.verifier.data[0],
+			NFS4_VERIFIER_SIZE);
+	}
+}
+
 __be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy,
 				struct cb_process_state *cps)
 {
+	struct nfs_server *server;
+	struct nfs4_copy_state *copy;
+
+	rcu_read_lock();
+	list_for_each_entry_rcu(server, &cps->clp->cl_superblocks,
+				client_link) {
+		spin_lock(&server->nfs_client->cl_lock);
+		list_for_each_entry(copy, &server->ss_copies, copies) {
+			if (memcmp(args->coa_stateid.other,
+					copy->stateid.other,
+					sizeof(args->coa_stateid.other)))
+				continue;
+			nfs4_copy_cb_args(copy, args);
+			complete(&copy->completion);
+			spin_unlock(&server->nfs_client->cl_lock);
+			goto out;
+		}
+		spin_unlock(&server->nfs_client->cl_lock);
+	}
+out:
+	rcu_read_unlock();
 	return 0;
 }
 #endif /* CONFIG_NFS_V4_2 */
fs/nfs/client.c +1 -0
index ee5ddbd..6bde98a 100644
@@ -875,6 +875,7 @@ struct nfs_server *nfs_alloc_server(void)
 	INIT_LIST_HEAD(&server->delegations);
 	INIT_LIST_HEAD(&server->layouts);
 	INIT_LIST_HEAD(&server->state_owners_lru);
+	INIT_LIST_HEAD(&server->ss_copies);
 
 	atomic_set(&server->active, 0);
 
fs/nfs/nfs42proc.c +51 -5

90 lines not on this page · raw message

fs/nfs/nfs42xdr.c +5 -3
index 27da108..f507fe4 100644
@@ -160,7 +160,7 @@ static void encode_copy(struct xdr_stream *xdr,
 	encode_uint64(xdr, args->count);
 
 	encode_uint32(xdr, 1); /* consecutive = true */
-	encode_uint32(xdr, 1); /* synchronous = true */
+	encode_uint32(xdr, args->sync);
 	encode_uint32(xdr, 0); /* src server list */
 }
 
@@ -289,7 +289,8 @@ static void nfs4_xdr_enc_copy(struct rpc_rqst *req,
 	encode_savefh(xdr, &hdr);
 	encode_putfh(xdr, args->dst_fh, &hdr);
 	encode_copy(xdr, args, &hdr);
-	encode_copy_commit(xdr, args, &hdr);
+	if (args->sync)
+		encode_copy_commit(xdr, args, &hdr);
 	encode_nops(&hdr);
 }
 
@@ -603,7 +604,8 @@ static int nfs4_xdr_dec_copy(struct rpc_rqst *rqstp,
 	status = decode_copy(xdr, res);
 	if (status)
 		goto out;
-	status = decode_commit(xdr, &res->commit_res);
+	if (res->commit_res.verf)
+		status = decode_commit(xdr, &res->commit_res);
 out:
 	return status;
 }
include/linux/nfs_fs.h +9 -0
index bb0eb2c..8b0192f 100644
@@ -183,6 +183,15 @@ struct nfs_inode {
 	struct inode		vfs_inode;
 };
 
+struct nfs4_copy_state {
+	struct list_head	copies;
+	nfs4_stateid		stateid;
+	struct completion	completion;
+	uint64_t		count;
+	struct nfs_writeverf	verf;
+	int			error;
+};
+
 /*
  * Cache validity bit flags
  */
include/linux/nfs_fs_sb.h +1 -0
index 41a2af5..d4a2385 100644
@@ -205,6 +205,7 @@ struct nfs_server {
 	struct list_head	state_owners_lru;
 	struct list_head	layouts;
 	struct list_head	delegations;
+	struct list_head	ss_copies;
 
 	unsigned long		mig_gen;
 	unsigned long		mig_status;
include/linux/nfs_xdr.h +1 -0
index 60eafa0..4875928 100644
@@ -1371,6 +1371,7 @@ struct nfs42_copy_args {
 	u64				dst_pos;
 
 	u64				count;
+	bool				sync;
 };
 
 struct nfs42_write_res {
Signature · 1 line
1.8.3.1
#15 lore raw

RFC v3 9/42 NFS handle COPY reply CB_OFFLOAD call race

In reply to Olga Kornievskaia

It's possible that server replies back with CB_OFFLOAD call and COPY reply at the same time such that client will process CB_OFFLOAD before reply to COPY. For that keep a list of pending callback stateids received and them before waiting on completion check the pending list.

Cleanup any pending copies on the client shutdown.

---
 fs/nfs/callback_proc.c    | 17 ++++++++++++++---
 fs/nfs/nfs42proc.c        | 24 +++++++++++++++++++++---
 fs/nfs/nfs4client.c       | 15 +++++++++++++++
 include/linux/nfs_fs_sb.h |  1 +
 4 files changed, 51 insertions(+), 6 deletions(-)
fs/nfs/callback_proc.c +14 -3
index 7e211ed..bac6022 100644
@@ -666,11 +666,12 @@ __be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy,
 {
 	struct nfs_server *server;
 	struct nfs4_copy_state *copy;
+	bool found = false;
 
+	spin_lock(&cps->clp->cl_lock);
 	rcu_read_lock();
 	list_for_each_entry_rcu(server, &cps->clp->cl_superblocks,
 				client_link) {
-		spin_lock(&server->nfs_client->cl_lock);
 		list_for_each_entry(copy, &server->ss_copies, copies) {
 			if (memcmp(args->coa_stateid.other,
 					copy->stateid.other,
@@ -678,13 +679,23 @@ __be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy,
 				continue;
 			nfs4_copy_cb_args(copy, args);
 			complete(&copy->completion);
-			spin_unlock(&server->nfs_client->cl_lock);
+			found = true;
 			goto out;
 		}
-		spin_unlock(&server->nfs_client->cl_lock);
 	}
 out:
 	rcu_read_unlock();
+	if (!found) {
+		copy = kzalloc(sizeof(struct nfs4_copy_state), GFP_NOFS);
+		if (!copy) {
+			spin_unlock(&cps->clp->cl_lock);
+			return -ENOMEM;
+		}
+		memcpy(&copy->stateid, &args->coa_stateid, NFS4_STATEID_SIZE);
+		nfs4_copy_cb_args(copy, args);
+		list_add_tail(&copy->copies, &cps->clp->pending_cb_stateids);
+	}
+	spin_unlock(&cps->clp->cl_lock);
 	return 0;
 }
 #endif /* CONFIG_NFS_V4_2 */
fs/nfs/nfs42proc.c +21 -3
index d92ce46..9acbde5 100644
@@ -137,15 +137,32 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 			     uint64_t *ret_count)
 {
 	struct nfs4_copy_state *copy;
-	int status;
+	int status = NFS4_OK;
+	bool found_pending = false;
+
+	spin_lock(&server->nfs_client->cl_lock);
+	list_for_each_entry(copy, &server->nfs_client->pending_cb_stateids,
+				copies) {
+		if (memcmp(&res->write_res.stateid, &copy->stateid,
+				NFS4_STATEID_SIZE))
+			continue;
+		found_pending = true;
+		list_del(&copy->copies);
+		break;
+	}
+	if (found_pending) {
+		spin_unlock(&server->nfs_client->cl_lock);
+		goto out;
+	}
 
 	copy = kzalloc(sizeof(struct nfs4_copy_state), GFP_NOFS);
-	if (!copy)
+	if (!copy) {
+		spin_unlock(&server->nfs_client->cl_lock);
 		return -ENOMEM;
+	}
 	memcpy(&copy->stateid, &res->write_res.stateid, NFS4_STATEID_SIZE);
 	init_completion(&copy->completion);
 
-	spin_lock(&server->nfs_client->cl_lock);
 	list_add_tail(&copy->copies, &server->ss_copies);
 	spin_unlock(&server->nfs_client->cl_lock);
 
@@ -153,6 +170,7 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	spin_lock(&server->nfs_client->cl_lock);
 	list_del_init(&copy->copies);
 	spin_unlock(&server->nfs_client->cl_lock);
+out:
 	*ret_count = copy->count;
 	memcpy(&res->write_res.verifier, &copy->verf, sizeof(copy->verf));
 	status = -copy->error;
fs/nfs/nfs4client.c +15 -0
index 66776f0..89b6414 100644
@@ -156,9 +156,23 @@ struct rpc_clnt *
 	}
 }
 
+static void
+nfs4_cleanup_callback(struct nfs_client *clp)
+{
+	struct nfs4_copy_state *cp_state;
+
+	while (!list_empty(&clp->pending_cb_stateids)) {
+		cp_state = list_entry(clp->pending_cb_stateids.next,
+					struct nfs4_copy_state, copies);
+		list_del(&cp_state->copies);
+		kfree(cp_state);
+	}
+}
+
 void nfs41_shutdown_client(struct nfs_client *clp)
 {
 	if (nfs4_has_session(clp)) {
+		nfs4_cleanup_callback(clp);
 		nfs4_shutdown_ds_clients(clp);
 		nfs4_destroy_session(clp->cl_session);
 		nfs4_destroy_clientid(clp);
@@ -202,6 +216,7 @@ struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init)
 #if IS_ENABLED(CONFIG_NFS_V4_1)
 	init_waitqueue_head(&clp->cl_lock_waitq);
 #endif
+	INIT_LIST_HEAD(&clp->pending_cb_stateids);
 	return clp;
 
 error:
include/linux/nfs_fs_sb.h +1 -0
index d4a2385..2b0397d 100644
@@ -118,6 +118,7 @@ struct nfs_client {
 #endif
 
 	struct net		*cl_net;
+	struct list_head	pending_cb_stateids;
 };
 
 /*
Signature · 1 line
1.8.3.1
#16 lore raw

RFC v3 10/42 NFS export nfs4_async_handle_error

In reply to Olga Kornievskaia

Make this function available to nfs42proc.c

---
 fs/nfs/nfs4_fs.h  | 3 +++
 fs/nfs/nfs4proc.c | 3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
fs/nfs/nfs4_fs.h +3 -0
index af285cc..f5d014e 100644
@@ -244,6 +244,9 @@ int nfs4_replace_transport(struct nfs_server *server,
 
 /* nfs4proc.c */
 extern int nfs4_handle_exception(struct nfs_server *, int, struct nfs4_exception *);
+extern int nfs4_async_handle_error(struct rpc_task *task,
+				   struct nfs_server *server,
+				   struct nfs4_state *state, long *timeout);
 extern int nfs4_call_sync(struct rpc_clnt *, struct nfs_server *,
 			  struct rpc_message *, struct nfs4_sequence_args *,
 			  struct nfs4_sequence_res *, int);
fs/nfs/nfs4proc.c +2 -1
index b37e528..fdb6b76 100644
@@ -548,7 +548,7 @@ int nfs4_handle_exception(struct nfs_server *server, int errorcode, struct nfs4_
 	return ret;
 }
 
-static int
+int
 nfs4_async_handle_error(struct rpc_task *task, struct nfs_server *server,
 			struct nfs4_state *state, long *timeout)
 {
@@ -569,6 +569,7 @@ int nfs4_handle_exception(struct nfs_server *server, int errorcode, struct nfs4_
 		return -EAGAIN;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nfs4_async_handle_error);
 
 /*
  * Return 'true' if 'clp' is using an rpc_client that is integrity protected
Signature · 1 line
1.8.3.1
#17 lore raw

Re: [RFC v3 10/42] NFS export nfs4_async_handle_error

Hi Olga,

Quoted 1 line · Olga Kornievskaia

Make this function available to nfs42proc.c

I'm confused about why this patch is needed. Aren't NFS v4.2 and nfs4_async_handle_error() both built into the NFSv4.ko module?

Anna

Quoted 43 lines
---
 fs/nfs/nfs4_fs.h  | 3 +++
 fs/nfs/nfs4proc.c | 3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)
fs/nfs/nfs4_fs.h +3 -0
index af285cc..f5d014e 100644
@@ -244,6 +244,9 @@ int nfs4_replace_transport(struct nfs_server *server,
 
 /* nfs4proc.c */
 extern int nfs4_handle_exception(struct nfs_server *, int, struct nfs4_exception *);
+extern int nfs4_async_handle_error(struct rpc_task *task,
+				   struct nfs_server *server,
+				   struct nfs4_state *state, long *timeout);
 extern int nfs4_call_sync(struct rpc_clnt *, struct nfs_server *,
 			  struct rpc_message *, struct nfs4_sequence_args *,
 			  struct nfs4_sequence_res *, int);
fs/nfs/nfs4proc.c +2 -1
index b37e528..fdb6b76 100644
@@ -548,7 +548,7 @@ int nfs4_handle_exception(struct nfs_server *server, int errorcode, struct nfs4_
 	return ret;
 }
 
-static int
+int
 nfs4_async_handle_error(struct rpc_task *task, struct nfs_server *server,
 			struct nfs4_state *state, long *timeout)
 {
@@ -569,6 +569,7 @@ int nfs4_handle_exception(struct nfs_server *server, int errorcode, struct nfs4_
 		return -EAGAIN;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nfs4_async_handle_error);
 
 /*
  * Return 'true' if 'clp' is using an rpc_client that is integrity protected
#18 lore raw

Re: [RFC v3 10/42] NFS export nfs4_async_handle_error

Quoted 2 lines

On Jul 12, 2017, at 9:56 AM, Anna Schumaker <[email protected]> =

Quoted 7 lines · wrote:

=20
Hi Olga,
=20

Quoted 1 line · Olga Kornievskaia

Make this function available to nfs42proc.c

=20
I'm confused about why this patch is needed. Aren't NFS v4.2 and =

nfs4_async_handle_error()

Quoted 2 lines

both built into the NFSv4.ko module?
=20

You are right I don=E2=80=99t need the EXPORT_SYMBOL chunk but I need = the function to be callable from outside of nfs4proc.c
=20

Quoted 14 lines

Anna
=20

fs/nfs/nfs4_fs.h int nfs4_replace_transport(struct nfs_server =

Quoted 12 lines

=20

---
fs/nfs/nfs4_fs.h | 3 +++
fs/nfs/nfs4proc.c | 3 ++-

2 files changed, 5 insertions(+), 1 deletion(-)

=20

fs/nfs/nfs4_fs.h +0 -0
index af285cc..f5d014e 100644
@@ -244,6 +244,9 @@ int nfs4_replace_transport(struct nfs_server =

*server,

Quoted 3 lines
Quoted 3 lines

=20
/* nfs4proc.c */
extern int nfs4_handle_exception(struct nfs_server *, int, struct =

nfs4_exception *);

Quoted 3 lines

fs/nfs/nfs4_fs.h int nfs4_replace_transport(struct nfs_server *server,

Quoted 3 lines
fs/nfs/nfs4_fs.h +3 -0
+extern int nfs4_async_handle_error(struct rpc_task *task,
+				   struct nfs_server *server,
+				   struct nfs4_state *state, long =

*timeout);

Quoted 2 lines
Quoted 2 lines

extern int nfs4_call_sync(struct rpc_clnt *, struct nfs_server *,
struct rpc_message *, struct =

nfs4_sequence_args *,

Quoted 6 lines

fs/nfs/nfs4proc.c int nfs4_handle_exception(struct nfs_server =

Quoted 6 lines
			  struct nfs4_sequence_res *, int);
fs/nfs/nfs4proc.c +0 -0
index b37e528..fdb6b76 100644
@@ -548,7 +548,7 @@ int nfs4_handle_exception(struct nfs_server =

*server, int errorcode, struct nfs4_

Quoted 6 lines
Quoted 6 lines
fs/nfs/nfs4proc.c +0 -0
 	return ret;
 }

=20

fs/nfs/nfs4proc.c +1 -1
-static int
+int

nfs4_async_handle_error(struct rpc_task *task, struct nfs_server =

*server,

Quoted 3 lines

fs/nfs/nfs4proc.c int nfs4_handle_exception(struct nfs_server =

Quoted 3 lines

struct nfs4_state *state, long *timeout)
{

fs/nfs/nfs4proc.c +0 -0
@@ -569,6 +569,7 @@ int nfs4_handle_exception(struct nfs_server =

*server, int errorcode, struct nfs4_

Quoted 7 lines
Quoted 7 lines
fs/nfs/nfs4proc.c +1 -0
 		return -EAGAIN;
 	return 0;
 }
+EXPORT_SYMBOL_GPL(nfs4_async_handle_error);

=20
/*
* Return 'true' if 'clp' is using an rpc_client that is integrity =

protected

Quoted 1 line
Quoted 1 line

=20

#19 lore raw

RFC v3 11/42 NFS test for intra vs inter COPY

In reply to Olga Kornievskaia
---
 fs/nfs/internal.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
fs/nfs/internal.h +10 -0
index 3e24392..dcbcffb 100644
@@ -7,6 +7,7 @@
 #include <linux/security.h>
 #include <linux/crc32.h>
 #include <linux/nfs_page.h>
+#include <linux/sunrpc/addr.h>
 
 #define NFS_MS_MASK (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_SYNCHRONOUS)
 
@@ -766,3 +767,12 @@ static inline bool nfs_error_is_fatal(int err)
 		return false;
 	}
 }
+
+static inline bool nfs42_files_from_same_server(struct file *in, struct file *out)
+{
+	struct nfs_client *c_in = (NFS_SERVER(file_inode(in)))->nfs_client;
+	struct nfs_client *c_out = (NFS_SERVER(file_inode(out)))->nfs_client;
+
+	return rpc_cmp_addr((struct sockaddr *)&c_in->cl_addr,
+				(struct sockaddr *)&c_out->cl_addr);
+}
Signature · 1 line
1.8.3.1
#20 lore raw

Re: [RFC v3 11/42] NFS test for intra vs inter COPY

Hi Olga,

Quoted 20 lines · Olga Kornievskaia
---
 fs/nfs/internal.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
fs/nfs/internal.h +1 -0
index 3e24392..dcbcffb 100644
@@ -7,6 +7,7 @@
 #include <linux/security.h>
 #include <linux/crc32.h>
 #include <linux/nfs_page.h>
+#include <linux/sunrpc/addr.h>
 
 #define NFS_MS_MASK (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_SYNCHRONOUS)
 
@@ -766,3 +767,12 @@ static inline bool nfs_error_is_fatal(int err)
 		return false;
 	}

fs/nfs/internal.h static inline bool nfs_error_is_fatal(int err) · line 778

Quoted 10 lines
fs/nfs/internal.h +9 -0
 }
+
+static inline bool nfs42_files_from_same_server(struct file *in, struct file *out)
+{
+	struct nfs_client *c_in = (NFS_SERVER(file_inode(in)))->nfs_client;
+	struct nfs_client *c_out = (NFS_SERVER(file_inode(out)))->nfs_client;
+
+	return rpc_cmp_addr((struct sockaddr *)&c_in->cl_addr,
+				(struct sockaddr *)&c_out->cl_addr);
+}

Can you move this to nfs42.h instead of internal.h?

Thanks,
Anna

Quoted 1 line
#21 lore raw

Re: [RFC v3 11/42] NFS test for intra vs inter COPY

Quoted 2 lines

On Jul 12, 2017, at 10:06 AM, Anna Schumaker =

Quoted 4 lines · <[email protected]> wrote:

=20
Hi Olga,
=20
On 07/11/2017 12:43 PM, Olga Kornievskaia wrote:

Quoted 16 lines
Quoted 14 lines

---
fs/nfs/internal.h | 10 ++++++++++

1 file changed, 10 insertions(+)

=20

fs/nfs/internal.h +1 -0
index 3e24392..dcbcffb 100644
@@ -7,6 +7,7 @@
 #include <linux/security.h>
 #include <linux/crc32.h>
 #include <linux/nfs_page.h>
+#include <linux/sunrpc/addr.h>
Quoted 2 lines

=20
#define NFS_MS_MASK =

(MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_SYNCHRONOUS)

Quoted 7 lines

fs/nfs/internal.h static inline bool nfs_error_is_fatal(int err) · line 771

Quoted 7 lines

=20

fs/nfs/internal.h +2 -0
@@ -766,3 +767,12 @@ static inline bool nfs_error_is_fatal(int err)
 		return false;
 	}
 }
+
+static inline bool nfs42_files_from_same_server(struct file *in, =

struct file *out)

Quoted 2 lines

fs/nfs/internal.h static inline bool nfs_error_is_fatal(int err) · line 773

Quoted 2 lines
fs/nfs/internal.h +2 -0
+{
+	struct nfs_client *c_in =3D =

(NFS_SERVER(file_inode(in)))->nfs_client;

Quoted 1 line

fs/nfs/internal.h static inline bool nfs_error_is_fatal(int err) · line 774

Quoted 1 line
fs/nfs/internal.h +1 -0
+	struct nfs_client *c_out =3D =

(NFS_SERVER(file_inode(out)))->nfs_client;

Quoted 7 lines

fs/nfs/internal.h static inline bool nfs_error_is_fatal(int err) · line 778

Quoted 4 lines
fs/nfs/internal.h +4 -0
+
+	return rpc_cmp_addr((struct sockaddr *)&c_in->cl_addr,
+				(struct sockaddr *)&c_out->cl_addr);
+}

=20
Can you move this to nfs42.h instead of internal.h? =20

Ok.

Quoted 4 lines

Thanks,
Anna
=20

Quoted 1 line

=20

#22 lore raw

RFC v3 12/42 NFS send OFFLOAD_CANCEL when COPY killed

In reply to Olga Kornievskaia

When COPY is killed send OFFLOAD_CANCEL to both destination and source servers.

---
 fs/nfs/nfs42proc.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 97 insertions(+), 3 deletions(-)
fs/nfs/nfs42proc.c +97 -3
index 9acbde5..8223726 100644
@@ -16,6 +16,7 @@
 #include "internal.h"
 
 #define NFSDBG_FACILITY NFSDBG_PROC
+static int nfs42_do_offload_cancel_async(struct file *dst, nfs4_stateid *std);
 
 static int _nfs42_proc_fallocate(struct rpc_message *msg, struct file *filep,
 		struct nfs_lock_context *lock, loff_t offset, loff_t len)
@@ -166,10 +167,15 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	list_add_tail(&copy->copies, &server->ss_copies);
 	spin_unlock(&server->nfs_client->cl_lock);
 
-	wait_for_completion_interruptible(&copy->completion);
+	status = wait_for_completion_interruptible(&copy->completion);
 	spin_lock(&server->nfs_client->cl_lock);
 	list_del_init(&copy->copies);
 	spin_unlock(&server->nfs_client->cl_lock);
+	if (status == -ERESTARTSYS) {
+		nfs42_do_offload_cancel_async(dst, &copy->stateid);
+		kfree(copy);
+		return status;
+	}
 out:
 	*ret_count = copy->count;
 	memcpy(&res->write_res.verifier, &copy->verf, sizeof(copy->verf));
@@ -226,8 +232,6 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 	}
 	status = nfs4_call_sync(server->client, server, &msg,
 				&args->seq_args, &res->seq_res, 0);
-	if (status == -ENOTSUPP)
-		server->caps &= ~NFS_CAP_COPY;
 	if (status)
 		goto out;
 
@@ -304,6 +308,14 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 				dst, dst_lock,
 				&args, &res);
 		inode_unlock(file_inode(dst));
+		switch (err) {
+		case -ENOTSUPP:
+			server->caps &= ~NFS_CAP_COPY;
+			if (nfs42_files_from_same_server(src, dst))
+				break;
+		case -ERESTARTSYS:
+			nfs42_do_offload_cancel_async(src, &args.src_stateid);
+		}
 
 		if (err >= 0)
 			break;
@@ -327,6 +339,88 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 	return err;
 }
 
+struct nfs42_offloadcancel_data {
+	struct nfs_server *seq_server;
+	struct nfs42_offload_status_args args;
+	struct nfs42_offload_status_res res;
+};
+
+static void nfs42_offload_cancel_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs42_offloadcancel_data *data = calldata;
+
+	nfs4_setup_sequence(data->seq_server->nfs_client,
+				&data->args.osa_seq_args,
+				&data->res.osr_seq_res, task);
+}
+
+static void nfs42_offload_cancel_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs42_offloadcancel_data *data = calldata;
+
+	nfs41_sequence_done(task, &data->res.osr_seq_res);
+	if (task->tk_status &&
+		nfs4_async_handle_error(task, data->seq_server, NULL,
+			NULL) == -EAGAIN)
+		rpc_restart_call_prepare(task);
+}
+
+static void nfs42_free_offloadcancel_data(void *data)
+{
+	kfree(data);
+}
+
+static const struct rpc_call_ops nfs42_offload_cancel_ops = {
+	.rpc_call_prepare = nfs42_offload_cancel_prepare,
+	.rpc_call_done = nfs42_offload_cancel_done,
+	.rpc_release = nfs42_free_offloadcancel_data,
+};
+
+static int nfs42_do_offload_cancel_async(struct file *dst,
+				nfs4_stateid *stateid)
+{
+	struct nfs_server *dst_server = NFS_SERVER(file_inode(dst));
+	struct nfs42_offloadcancel_data *data = NULL;
+	struct nfs_open_context *ctx = nfs_file_open_context(dst);
+	struct rpc_task *task;
+	struct rpc_message msg = {
+		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_OFFLOAD_CANCEL],
+		.rpc_cred = ctx->cred,
+	};
+	struct rpc_task_setup task_setup_data = {
+		.rpc_client = dst_server->client,
+		.rpc_message = &msg,
+		.callback_ops = &nfs42_offload_cancel_ops,
+		.workqueue = nfsiod_workqueue,
+		.flags = RPC_TASK_ASYNC,
+	};
+	int status;
+
+	if (!(dst_server->caps & NFS_CAP_OFFLOAD_CANCEL))
+		return -EOPNOTSUPP;
+
+	data = kzalloc(sizeof(struct nfs42_offloadcancel_data), GFP_NOFS);
+	if (data == NULL)
+		return -ENOMEM;
+
+	data->seq_server = dst_server;
+	data->args.osa_src_fh = NFS_FH(file_inode(dst));
+	memcpy(&data->args.osa_stateid, stateid,
+		sizeof(data->args.osa_stateid));
+	msg.rpc_argp = &data->args;
+	msg.rpc_resp = &data->res;
+	task_setup_data.callback_data = data;
+	nfs4_init_sequence(&data->args.osa_seq_args, &data->res.osr_seq_res, 1);
+	task = rpc_run_task(&task_setup_data);
+	if (IS_ERR(task))
+		return PTR_ERR(task);
+	status = rpc_wait_for_completion_task(task);
+	if (status == -ENOTSUPP)
+		dst_server->caps &= ~NFS_CAP_OFFLOAD_CANCEL;
+	rpc_put_task(task);
+	return status;
+}
+
 int _nfs42_proc_offload_status(struct file *dst, nfs4_stateid *stateid,
 				struct nfs42_offload_status_res *res)
 {
Signature · 1 line
1.8.3.1
#23 lore raw

RFC v3 13/42 NFS handle COPY ERR_OFFLOAD_NO_REQS

In reply to Olga Kornievskaia

If client sent async COPY and server replied with ERR_OFFLOAD_NO_REQS, client should retry with a synchronous copy.

---
 fs/nfs/nfs42proc.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfs/nfs42proc.c +7 -0
index 8223726..dba2b62 100644
@@ -309,6 +309,13 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 				&args, &res);
 		inode_unlock(file_inode(dst));
 		switch (err) {
+		case -NFS4ERR_OFFLOAD_NO_REQS:
+			if (!args.sync) {
+				args.sync = true;
+				dst_exception.retry = 1;
+				continue;
+			}
+			break;
 		case -ENOTSUPP:
 			server->caps &= ~NFS_CAP_COPY;
 			if (nfs42_files_from_same_server(src, dst))
Signature · 1 line
1.8.3.1
#24 lore raw

RFC v3 14/42 NFS if we got partial copy ignore errors

In reply to Olga Kornievskaia
---
 fs/nfs/nfs42proc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfs/nfs42proc.c +2 -1
index dba2b62..dfd43bc 100644
@@ -179,7 +179,8 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 out:
 	*ret_count = copy->count;
 	memcpy(&res->write_res.verifier, &copy->verf, sizeof(copy->verf));
-	status = -copy->error;
+	if (copy->count <= 0)
+		status = -copy->error;
 
 	kfree(copy);
 	return status;
Signature · 1 line
1.8.3.1
#25 lore raw

Re: [RFC v3 14/42] NFS if we got partial copy ignore errors

Hi Olga,

Quoted 10 lines · Olga Kornievskaia
---
 fs/nfs/nfs42proc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/nfs/nfs42proc.c +0 -0
index dba2b62..dfd43bc 100644

fs/nfs/nfs42proc.c static int handle_async_copy(struct nfs42_copy_res *res, · line 183

Quoted 6 lines
fs/nfs/nfs42proc.c +2 -1
@@ -179,7 +179,8 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 out:
 	*ret_count = copy->count;
 	memcpy(&res->write_res.verifier, &copy->verf, sizeof(copy->verf));
-	status = -copy->error;
+	if (copy->count <= 0)
+		status = -copy->error;

This is a pretty small change. Can you squash this patch into patch 8 ("NFS add support for asynchronous COPY") that way the code is correct from the time it's added?

Thanks,
Anna

Quoted 4 lines
fs/nfs/nfs42proc.c +0 -0
 	kfree(copy);
 	return status;
 
#26 lore raw

Re: [RFC v3 14/42] NFS if we got partial copy ignore errors

Quoted 2 lines

On Jul 12, 2017, at 10:52 AM, Anna Schumaker =

Quoted 4 lines · <[email protected]> wrote:

=20
Hi Olga,
=20
On 07/11/2017 12:43 PM, Olga Kornievskaia wrote:

Quoted 10 lines

fs/nfs/nfs42proc.c static int handle_async_copy(struct =

Quoted 10 lines

---
fs/nfs/nfs42proc.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)

=20

fs/nfs/nfs42proc.c +0 -0
index dba2b62..dfd43bc 100644
@@ -179,7 +179,8 @@ static int handle_async_copy(struct =

nfs42_copy_res *res,

Quoted 3 lines
Quoted 3 lines

out:

	*ret_count =3D copy->count;
	memcpy(&res->write_res.verifier, &copy->verf, =

sizeof(copy->verf));

Quoted 6 lines

fs/nfs/nfs42proc.c

Quoted 3 lines
fs/nfs/nfs42proc.c +2 -1
-	status =3D -copy->error;
+	if (copy->count <=3D 0)
+		status =3D -copy->error;

=20
This is a pretty small change. Can you squash this patch into patch 8 ("NFS add support for asynchronous COPY") that way the code is correct =

from

Quoted 1 line

the time it's added?

Ok.

Quoted 8 lines

=20
Thanks,
Anna
=20

Quoted 4 lines

=20

	kfree(copy);
	return status;

=20

#27 lore raw

RFC v3 15/42 NFS recover from destination server reboot for copies

In reply to Olga Kornievskaia

Mark the destination state to indicate a server-side copy is happening. On detecting a reboot and recovering open state check if any state is engaged in a server-side copy, if so, find the copy and mark it and then signal the waiting thread. Upon wakeup, if copy was marked then propage EAGAIN to the nfsd_copy_file_range and restart the copy from scratch (no partial state is being queried at this point).

---
 fs/nfs/nfs42proc.c     | 16 +++++++++++++---
 fs/nfs/nfs4_fs.h       |  4 ++++
 fs/nfs/nfs4file.c      |  9 +++++++--
 fs/nfs/nfs4state.c     | 15 +++++++++++++++
 include/linux/nfs_fs.h |  2 ++
 5 files changed, 41 insertions(+), 5 deletions(-)
fs/nfs/nfs42proc.c +13 -3
index dfd43bc..e145479 100644
@@ -140,6 +140,7 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	struct nfs4_copy_state *copy;
 	int status = NFS4_OK;
 	bool found_pending = false;
+	struct nfs_open_context *ctx = nfs_file_open_context(dst);
 
 	spin_lock(&server->nfs_client->cl_lock);
 	list_for_each_entry(copy, &server->nfs_client->pending_cb_stateids,
@@ -163,6 +164,7 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	}
 	memcpy(&copy->stateid, &res->write_res.stateid, NFS4_STATEID_SIZE);
 	init_completion(&copy->completion);
+	copy->parent_state = ctx->state;
 
 	list_add_tail(&copy->copies, &server->ss_copies);
 	spin_unlock(&server->nfs_client->cl_lock);
@@ -172,9 +174,10 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	list_del_init(&copy->copies);
 	spin_unlock(&server->nfs_client->cl_lock);
 	if (status == -ERESTARTSYS) {
-		nfs42_do_offload_cancel_async(dst, &copy->stateid);
-		kfree(copy);
-		return status;
+		goto out_cancel;
+	} else if (copy->flags) {
+		status = -EAGAIN;
+		goto out_cancel;
 	}
 out:
 	*ret_count = copy->count;
@@ -184,6 +187,10 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 
 	kfree(copy);
 	return status;
+out_cancel:
+	nfs42_do_offload_cancel_async(dst, &copy->stateid);
+	kfree(copy);
+	return status;
 }
 
 static ssize_t _nfs42_proc_copy(struct file *src,
@@ -231,6 +238,9 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 		if (!res->commit_res.verf)
 			return -ENOMEM;
 	}
+	set_bit(NFS_CLNT_DST_SSC_COPY_STATE,
+		&dst_lock->open_context->state->flags);
+
 	status = nfs4_call_sync(server->client, server, &msg,
 				&args->seq_args, &res->seq_res, 0);
 	if (status)
fs/nfs/nfs4_fs.h +4 -0
index f5d014e..54e8778 100644
@@ -161,6 +161,10 @@ enum {
 	NFS_STATE_POSIX_LOCKS,		/* Posix locks are supported */
 	NFS_STATE_RECOVERY_FAILED,	/* OPEN stateid state recovery failed */
 	NFS_STATE_MAY_NOTIFY_LOCK,	/* server may CB_NOTIFY_LOCK */
+#ifdef CONFIG_NFS_V4_2
+	NFS_CLNT_DST_SSC_COPY_STATE,	/* dst server open state on client*/
+#endif /* CONFIG_NFS_V4_2 */
+
 };
 
 struct nfs4_state {
fs/nfs/nfs4file.c +7 -2
index 0efba77..881b819 100644
@@ -133,10 +133,15 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in,
 				    struct file *file_out, loff_t pos_out,
 				    size_t count, unsigned int flags)
 {
+	ssize_t ret;
+
 	if (file_inode(file_in) == file_inode(file_out))
 		return -EINVAL;
-
-	return nfs42_proc_copy(file_in, pos_in, file_out, pos_out, count);
+retry:
+	ret = nfs42_proc_copy(file_in, pos_in, file_out, pos_out, count);
+	if (ret == -EAGAIN)
+		goto retry;
+	return ret;
 }
 
 static loff_t nfs4_file_llseek(struct file *filep, loff_t offset, int whence)
fs/nfs/nfs4state.c +15 -0
index cbf82b0..81f9d52 100644
@@ -1540,6 +1540,21 @@ static int nfs4_reclaim_open_state(struct nfs4_state_owner *sp, const struct nfs
 					&state->flags);
 				nfs4_put_open_state(state);
 				spin_lock(&sp->so_lock);
+#ifdef CONFIG_NFS_V4_2
+				if (test_bit(NFS_CLNT_DST_SSC_COPY_STATE, &state->flags)) {
+					struct nfs4_copy_state *copy;
+
+					spin_lock(&sp->so_server->nfs_client->cl_lock);
+					list_for_each_entry(copy, &sp->so_server->ss_copies, copies) {
+						if (memcmp(&state->stateid.other, &copy->parent_state->stateid.other, NFS4_STATEID_SIZE))
+							continue;
+						copy->flags = 1;
+						complete(&copy->completion);
+						break;
+					}
+					spin_unlock(&sp->so_server->nfs_client->cl_lock);
+				}
+#endif /* CONFIG_NFS_V4_2 */
 				goto restart;
 			}
 		}
include/linux/nfs_fs.h +2 -0
index 8b0192f..f26a35b 100644
@@ -190,6 +190,8 @@ struct nfs4_copy_state {
 	uint64_t		count;
 	struct nfs_writeverf	verf;
 	int			error;
+	int			flags;
+	struct nfs4_state	*parent_state;
 };
 
 /*
Signature · 1 line
1.8.3.1
#28 lore raw

RFC v3 16/42 NFS NFSD defining nl4_servers structure needed by both

In reply to Olga Kornievskaia

These structures are needed by COPY_NOTIFY on the client and needed by the nfsd as well

---
 include/linux/nfs4.h | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
include/linux/nfs4.h +33 -0
index 7262908..4179c78 100644
@@ -15,6 +15,7 @@
 #include <linux/list.h>
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
+#include <linux/sunrpc/msg_prot.h>
 
 enum nfs4_acl_whotype {
 	NFS4_ACL_WHO_NAMED = 0,
@@ -659,4 +660,36 @@ struct nfs4_op_map {
 	} u;
 };
 
+struct nfs42_netaddr {
+	unsigned int	na_netid_len;
+	char		na_netid[RPCBIND_MAXNETIDLEN + 1];
+	unsigned int	na_uaddr_len;
+	char		na_uaddr[RPCBIND_MAXUADDRLEN + 1];
+};
+
+enum netloc_type4 {
+	NL4_NAME		= 1,
+	NL4_URL			= 2,
+	NL4_NETADDR		= 3,
+};
+
+struct nl4_server {
+	enum netloc_type4	nl4_type;
+	union {
+		struct { /* NL4_NAME, NL4_URL */
+			int	nl4_str_sz;
+			char	nl4_str[NFS4_OPAQUE_LIMIT + 1];
+		};
+		struct nfs42_netaddr	nl4_addr; /* NL4_NETADDR */
+	} u;
+};
+
+/*  support 1 nl4_server for now */
+#define NFS42_MAX_SSC_SRC       1
+
+struct nl4_servers {
+	int			nl_nsvr;
+	struct nl4_server	*nl_svr;
+};
+
 #endif
Signature · 1 line
1.8.3.1
#29 lore raw

Re: [RFC v3 16/42] NFS NFSD defining nl4_servers structure needed by both

Quoted 40 lines · Olga Kornievskaia

These structures are needed by COPY_NOTIFY on the client and needed by the nfsd as well

---
 include/linux/nfs4.h | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
include/linux/nfs4.h +17 -0
index 7262908..4179c78 100644
@@ -15,6 +15,7 @@
 #include <linux/list.h>
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
+#include <linux/sunrpc/msg_prot.h>
 
 enum nfs4_acl_whotype {
 	NFS4_ACL_WHO_NAMED = 0,
@@ -659,4 +660,36 @@ struct nfs4_op_map {
 	} u;
 };
 
+struct nfs42_netaddr {
+	unsigned int	na_netid_len;
+	char		na_netid[RPCBIND_MAXNETIDLEN + 1];
+	unsigned int	na_uaddr_len;
+	char		na_uaddr[RPCBIND_MAXUADDRLEN + 1];
+};
+
+enum netloc_type4 {
+	NL4_NAME		= 1,
+	NL4_URL			= 2,
+	NL4_NETADDR		= 3,
+};
+
+struct nl4_server {
+	enum netloc_type4	nl4_type;
+	union {

include/linux/nfs4.h struct nfs4_op_map { · line 688

Quoted 10 lines
include/linux/nfs4.h +10 -0
+		struct { /* NL4_NAME, NL4_URL */
+			int	nl4_str_sz;
+			char	nl4_str[NFS4_OPAQUE_LIMIT + 1];
+		};
+		struct nfs42_netaddr	nl4_addr; /* NL4_NETADDR */
+	} u;
+};
+
+/*  support 1 nl4_server for now */
+#define NFS42_MAX_SSC_SRC       1

In that case, let's just build that assumption into the data structures and embed struct nl4_server in nl4_servers and drop the unnecessary kmalloc()s.

--b.

Quoted 14 lines
include/linux/nfs4.h +6 -0
+
+struct nl4_servers {
+	int			nl_nsvr;
+	struct nl4_server	*nl_svr;
+};
+
 #endif
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#30 lore raw

Re: [RFC v3 16/42] NFS NFSD defining nl4_servers structure needed by both

Quoted 47 lines · J. Bruce Fields
Quoted 46 lines · Olga Kornievskaia

These structures are needed by COPY_NOTIFY on the client and needed by the nfsd as well

---
 include/linux/nfs4.h | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
include/linux/nfs4.h +23 -0
index 7262908..4179c78 100644
@@ -15,6 +15,7 @@
 #include <linux/list.h>
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
+#include <linux/sunrpc/msg_prot.h>
 
 enum nfs4_acl_whotype {
      NFS4_ACL_WHO_NAMED = 0,
@@ -659,4 +660,36 @@ struct nfs4_op_map {
      } u;
 };
 
+struct nfs42_netaddr {
+     unsigned int    na_netid_len;
+     char            na_netid[RPCBIND_MAXNETIDLEN + 1];
+     unsigned int    na_uaddr_len;
+     char            na_uaddr[RPCBIND_MAXUADDRLEN + 1];
+};
+
+enum netloc_type4 {
+     NL4_NAME                = 1,
+     NL4_URL                 = 2,
+     NL4_NETADDR             = 3,
+};
+
+struct nl4_server {
+     enum netloc_type4       nl4_type;
+     union {
+             struct { /* NL4_NAME, NL4_URL */
+                     int     nl4_str_sz;
+                     char    nl4_str[NFS4_OPAQUE_LIMIT + 1];
+             };
+             struct nfs42_netaddr    nl4_addr; /* NL4_NETADDR */
+     } u;
Quoted 8 lines

include/linux/nfs4.h struct nfs4_op_map { · line 688

Quoted 4 lines
include/linux/nfs4.h +4 -0
+};
+
+/*  support 1 nl4_server for now */
+#define NFS42_MAX_SSC_SRC       1

In that case, let's just build that assumption into the data structures and embed struct nl4_server in nl4_servers and drop the unnecessary kmalloc()s.

Wouldn't we still want to malloc as the structure is too big to be allocated on the stack?

Anna has also suggested dropping nl4_servers since the code sets maximum to 1. I have resisted making the change because I felt like it was easier to increase the
MAX_SSC_SRC to say something like 10 and support a multi-home server. Then the destination server can add smarts to chose between the available address to connect to the source server with.

Is this not a good argument to keep the structures? Should I change value to 2 to
indicate desire to support multi-path?

Quoted 21 lines

--b.

Quoted 10 lines
include/linux/nfs4.h +6 -1
+
+struct nl4_servers {
+     int                     nl_nsvr;
+     struct nl4_server       *nl_svr;
+};
+
 #endif
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#31 lore raw

Re: [RFC v3 16/42] NFS NFSD defining nl4_servers structure needed by both

Quoted 57 lines · Olga Kornievskaia
Quoted 47 lines · J. Bruce Fields
Quoted 46 lines · Olga Kornievskaia

These structures are needed by COPY_NOTIFY on the client and needed by the nfsd as well

---
 include/linux/nfs4.h | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
include/linux/nfs4.h +23 -0
index 7262908..4179c78 100644
@@ -15,6 +15,7 @@
 #include <linux/list.h>
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
+#include <linux/sunrpc/msg_prot.h>
 
 enum nfs4_acl_whotype {
      NFS4_ACL_WHO_NAMED = 0,
@@ -659,4 +660,36 @@ struct nfs4_op_map {
      } u;
 };
 
+struct nfs42_netaddr {
+     unsigned int    na_netid_len;
+     char            na_netid[RPCBIND_MAXNETIDLEN + 1];
+     unsigned int    na_uaddr_len;
+     char            na_uaddr[RPCBIND_MAXUADDRLEN + 1];
+};
+
+enum netloc_type4 {
+     NL4_NAME                = 1,
+     NL4_URL                 = 2,
+     NL4_NETADDR             = 3,
+};
+
+struct nl4_server {
+     enum netloc_type4       nl4_type;
+     union {
+             struct { /* NL4_NAME, NL4_URL */
+                     int     nl4_str_sz;
+                     char    nl4_str[NFS4_OPAQUE_LIMIT + 1];
+             };
+             struct nfs42_netaddr    nl4_addr; /* NL4_NETADDR */
+     } u;
Quoted 8 lines

include/linux/nfs4.h struct nfs4_op_map { · line 688

Quoted 4 lines
include/linux/nfs4.h +4 -0
+};
+
+/*  support 1 nl4_server for now */
+#define NFS42_MAX_SSC_SRC       1

In that case, let's just build that assumption into the data structures and embed struct nl4_server in nl4_servers and drop the unnecessary kmalloc()s.

Quoted 2 lines

Wouldn't we still want to malloc as the structure is too big to be allocated on the stack?

Is it allocated on the stack? I thought it was embedded in nfsd4_compoundargs.

Anyway, I'm not that fixated on removing kmalloc()'s, it's OK if you need them.

Looking at that definition again, NFS4_OPAQUE_LIMIT is 1K, that's bigger than I'd want embedded in nfsd4_compoundargs too, so yes, it might still be worth kmalloc()'ing. Or consider whether we really need the maximum size for our current use (just ascii-encoded IP addresses?), or whether we should do the parsing earlier and just store the resulting socket address.

Quoted 10 lines

Anna has also suggested dropping nl4_servers since the code sets maximum to 1. I have resisted making the change because I felt like it was easier to increase the
MAX_SSC_SRC to say something like 10 and support a multi-home server. Then the destination server can add smarts to chose between the available address to connect to the source server with.

Is this not a good argument to keep the structures? Should I change value to 2 to
indicate desire to support multi-path?

In the first version of this patchset I recommend dropping anything that's not necessary.

It's easy enough to add this back in later. The hard part is going to be getting in any server-to-server patches at all, so we need to make them as simple as possible.

The only reason to keep this in the initial version, I think, would be if you think it's an important enough feature that server-to-server copy isn't useful at all without it.

--b.

#32 lore raw

RFC v3 17/42 NFS add COPY_NOTIFY operation

In reply to Olga Kornievskaia

Try using the delegation stateid, then the open stateid.

Only NL4_NETATTR, No support for NL4_NAME and NL4_URL. Allow only one source server address to be returned for now.

---
 fs/nfs/nfs42.h            |   2 +
 fs/nfs/nfs42proc.c        |  91 +++++++++++++++++++++++
 fs/nfs/nfs42xdr.c         | 186 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4_fs.h          |   1 +
 fs/nfs/nfs4file.c         |  28 +++++++
 fs/nfs/nfs4proc.c         |   1 +
 fs/nfs/nfs4state.c        |   2 +-
 fs/nfs/nfs4xdr.c          |   1 +
 include/linux/nfs4.h      |   1 +
 include/linux/nfs_fs_sb.h |   1 +
 include/linux/nfs_xdr.h   |  18 +++++
 11 files changed, 331 insertions(+), 1 deletion(-)
fs/nfs/nfs42.h +2 -0
index 54b9941..a0d6d45 100644
@@ -22,5 +22,7 @@ int nfs42_proc_layoutstats_generic(struct nfs_server *,
 int nfs42_proc_clone(struct file *, struct file *, loff_t, loff_t, loff_t);
 int nfs42_proc_offload_status(struct file *, nfs4_stateid *,
 			      struct nfs42_offload_status_res *);
+int nfs42_proc_copy_notify(struct file *, struct file *,
+			   struct nfs42_copy_notify_res *);
 #endif /* CONFIG_NFS_V4_2) */
 #endif /* __LINUX_FS_NFS_NFS4_2_H */
fs/nfs/nfs42proc.c +91 -0

113 lines not on this page · raw message

fs/nfs/nfs42xdr.c +186 -0

234 lines not on this page · raw message

fs/nfs/nfs4_fs.h +1 -0
index 54e8778..70fa0e4 100644
@@ -453,6 +453,7 @@ extern void nfs41_handle_server_scope(struct nfs_client *,
 extern int nfs4_select_rw_stateid(struct nfs4_state *, fmode_t,
 		const struct nfs_lock_context *, nfs4_stateid *,
 		struct rpc_cred **);
+extern void nfs4_copy_open_stateid(nfs4_stateid *, struct nfs4_state *);
 
 extern struct nfs_seqid *nfs_alloc_seqid(struct nfs_seqid_counter *counter, gfp_t gfp_mask);
 extern int nfs_wait_on_sequence(struct nfs_seqid *seqid, struct rpc_task *task);
fs/nfs/nfs4file.c +28 -0
index 881b819..b7b8c47 100644
@@ -133,12 +133,40 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in,
 				    struct file *file_out, loff_t pos_out,
 				    size_t count, unsigned int flags)
 {
+	struct nfs42_copy_notify_res *cn_resp = NULL;
 	ssize_t ret;
 
 	if (file_inode(file_in) == file_inode(file_out))
 		return -EINVAL;
 retry:
+	if (nfs42_files_from_same_server(file_in, file_out)) {  /* Intra-ssc */
+		if (file_in->f_op != file_out->f_op)
+			return -EXDEV;
+	} else {  /* Inter-ssc */
+		cn_resp = kzalloc(sizeof(struct nfs42_copy_notify_res),
+				  GFP_NOFS);
+		if (unlikely(cn_resp == NULL))
+			return -ENOMEM;
+
+		cn_resp->cnr_src.nl_svr = kzalloc(NFS42_MAX_SSC_SRC *
+						sizeof(struct nl4_server),
+						GFP_NOFS);
+		if (unlikely(cn_resp->cnr_src.nl_svr == NULL)) {
+			kfree(cn_resp);
+			return -ENOMEM;
+		}
+
+		ret = nfs42_proc_copy_notify(file_in, file_out, cn_resp);
+		if (ret)
+			goto out;
+	}
 	ret = nfs42_proc_copy(file_in, pos_in, file_out, pos_out, count);
+
+out:
+	if (cn_resp) {
+		kfree(cn_resp->cnr_src.nl_svr);
+		kfree(cn_resp);
+	}
 	if (ret == -EAGAIN)
 		goto retry;
 	return ret;
fs/nfs/nfs4proc.c +1 -0
index fdb6b76..94913fd 100644
@@ -9232,6 +9232,7 @@ static bool nfs4_match_stateid(const nfs4_stateid *s1,
 		| NFS_CAP_COPY
 		| NFS_CAP_OFFLOAD_STATUS
 		| NFS_CAP_OFFLOAD_CANCEL
+		| NFS_CAP_COPY_NOTIFY
 		| NFS_CAP_DEALLOCATE
 		| NFS_CAP_SEEK
 		| NFS_CAP_LAYOUTSTATS
fs/nfs/nfs4state.c +1 -1
index 81f9d52..f640050 100644
@@ -979,7 +979,7 @@ static int nfs4_copy_lock_stateid(nfs4_stateid *dst,
 	return ret;
 }
 
-static void nfs4_copy_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
+void nfs4_copy_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
 {
 	const nfs4_stateid *src;
 	int seq;
fs/nfs/nfs4xdr.c +1 -0
index 9b66053..8c72bb6 100644
@@ -7563,6 +7563,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(COPY,		enc_copy,		dec_copy),
 	PROC(OFFLOAD_STATUS,	enc_offload_status,	dec_offload_status),
 	PROC(OFFLOAD_CANCEL,    enc_offload_cancel,     dec_offload_cancel),
+	PROC(COPY_NOTIFY,	enc_copy_notify,	dec_copy_notify),
 #endif /* CONFIG_NFS_V4_2 */
 };
 
include/linux/nfs4.h +1 -0
index 4179c78..5aa36ac 100644
@@ -526,6 +526,7 @@ enum {
 	NFSPROC4_CLNT_COPY,
 	NFSPROC4_CLNT_OFFLOAD_STATUS,
 	NFSPROC4_CLNT_OFFLOAD_CANCEL,
+	NFSPROC4_CLNT_COPY_NOTIFY,
 };
 
 /* nfs41 types */
include/linux/nfs_fs_sb.h +1 -0
index 2b0397d..7099855 100644
@@ -254,5 +254,6 @@ struct nfs_server {
 #define NFS_CAP_COPY		(1U << 24)
 #define NFS_CAP_OFFLOAD_STATUS	(1U << 25)
 #define NFS_CAP_OFFLOAD_CANCEL	(1U << 26)
+#define NFS_CAP_COPY_NOTIFY	(1U << 27)
 
 #endif
include/linux/nfs_xdr.h +18 -0
index 4875928..300289b 100644
@@ -1400,6 +1400,24 @@ struct nfs42_offload_status_res {
 	int				osr_status;
 };
 
+struct nfs42_copy_notify_args {
+	struct nfs4_sequence_args	cna_seq_args;
+
+	struct nfs_fh		*cna_src_fh;
+	nfs4_stateid		cna_src_stateid;
+	/* cna_destiniation_server */
+	struct nl4_server	cna_dst;
+};
+
+struct nfs42_copy_notify_res {
+	struct nfs4_sequence_res	cnr_seq_res;
+
+	struct nfstime4		cnr_lease_time;
+	nfs4_stateid		cnr_stateid;
+	/* cnr_source_server, for now, always 1 */
+	struct nl4_servers	cnr_src;
+};
+
 struct nfs42_seek_args {
 	struct nfs4_sequence_args	seq_args;
 
Signature · 1 line
1.8.3.1
#33 lore raw

RFC v3 18/42 NFS add ca_source_server<> to COPY

In reply to Olga Kornievskaia

Support only one source server address: the same address that the client and source server use.

---
 fs/nfs/nfs42.h          |  3 ++-
 fs/nfs/nfs42proc.c      | 26 +++++++++++++++++---------
 fs/nfs/nfs42xdr.c       | 19 +++++++++++++++++--
 fs/nfs/nfs4file.c       |  7 ++++++-
 include/linux/nfs_xdr.h |  1 +
 5 files changed, 43 insertions(+), 13 deletions(-)
fs/nfs/nfs42.h +2 -1
index a0d6d45..6bf5775 100644
@@ -14,7 +14,8 @@
 /* nfs4.2proc.c */
 #if defined(CONFIG_NFS_V4_2)
 int nfs42_proc_allocate(struct file *, loff_t, loff_t);
-ssize_t nfs42_proc_copy(struct file *, loff_t, struct file *, loff_t, size_t);
+ssize_t nfs42_proc_copy(struct file *, loff_t, struct file *, loff_t, size_t,
+			struct nl4_servers *, nfs4_stateid *);
 int nfs42_proc_deallocate(struct file *, loff_t, loff_t);
 loff_t nfs42_proc_llseek(struct file *, loff_t, int);
 int nfs42_proc_layoutstats_generic(struct nfs_server *,
fs/nfs/nfs42proc.c +17 -9
index d133533..36b265a 100644
@@ -219,7 +219,9 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 				struct file *dst,
 				struct nfs_lock_context *dst_lock,
 				struct nfs42_copy_args *args,
-				struct nfs42_copy_res *res)
+				struct nfs42_copy_res *res,
+				struct nl4_servers *nss,
+				nfs4_stateid *cnr_stateid)
 {
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_COPY],
@@ -233,11 +235,15 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 	size_t count = args->count;
 	ssize_t status;
 
-	status = nfs4_set_rw_stateid(&args->src_stateid, src_lock->open_context,
-				     src_lock, FMODE_READ);
-	if (status)
-		return status;
-
+	if (nss) {
+		args->cp_src = nss;
+		nfs4_stateid_copy(&args->src_stateid, cnr_stateid);
+	} else {
+		status = nfs4_set_rw_stateid(&args->src_stateid,
+				src_lock->open_context, src_lock, FMODE_READ);
+		if (status)
+			return status;
+	}
 	status = nfs_filemap_write_and_wait_range(file_inode(src)->i_mapping,
 			pos_src, pos_src + (loff_t)count - 1);
 	if (status)
@@ -292,8 +298,9 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 }
 
 ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
-			struct file *dst, loff_t pos_dst,
-			size_t count)
+			struct file *dst, loff_t pos_dst, size_t count,
+			struct nl4_servers *nss,
+			nfs4_stateid *cnr_stateid)
 {
 	struct nfs_server *server = NFS_SERVER(file_inode(dst));
 	struct nfs_lock_context *src_lock;
@@ -338,7 +345,8 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 		inode_lock(file_inode(dst));
 		err = _nfs42_proc_copy(src, src_lock,
 				dst, dst_lock,
-				&args, &res);
+				&args, &res,
+				nss, cnr_stateid);
 		inode_unlock(file_inode(dst));
 		switch (err) {
 		case -NFS4ERR_OFFLOAD_NO_REQS:
fs/nfs/nfs42xdr.c +17 -2
index ae84bae..68ba3cc 100644
@@ -20,7 +20,10 @@
 #define encode_copy_maxsz		(op_encode_hdr_maxsz +          \
 					 XDR_QUADLEN(NFS4_STATEID_SIZE) + \
 					 XDR_QUADLEN(NFS4_STATEID_SIZE) + \
-					 2 + 2 + 2 + 1 + 1 + 1)
+					 2 + 2 + 2 + 1 + 1 + 1 +\
+					 1 + /* One cnr_source_server */\
+					 1 + /* nl4_type */ \
+					 1 + XDR_QUADLEN(NFS4_OPAQUE_LIMIT))
 #define decode_copy_maxsz		(op_decode_hdr_maxsz + \
 					 NFS42_WRITE_RES_SIZE + \
 					 1 /* cr_consecutive */ + \
@@ -186,6 +189,9 @@ static void encode_copy(struct xdr_stream *xdr,
 			struct nfs42_copy_args *args,
 			struct compound_hdr *hdr)
 {
+	struct nl4_server *ns;
+	int i;
+
 	encode_op_hdr(xdr, OP_COPY, decode_copy_maxsz, hdr);
 	encode_nfs4_stateid(xdr, &args->src_stateid);
 	encode_nfs4_stateid(xdr, &args->dst_stateid);
@@ -196,7 +202,16 @@ static void encode_copy(struct xdr_stream *xdr,
 
 	encode_uint32(xdr, 1); /* consecutive = true */
 	encode_uint32(xdr, args->sync);
-	encode_uint32(xdr, 0); /* src server list */
+	if (args->cp_src == NULL) { /* intra-ssc */
+		encode_uint32(xdr, 0); /* no src server list */
+		return;
+	}
+	encode_uint32(xdr, args->cp_src->nl_nsvr);
+	ns = args->cp_src->nl_svr;
+	for (i = 0; i < args->cp_src->nl_nsvr; i++) {
+		encode_nl4_server(xdr, args->cp_src->nl_svr);
+		ns++;
+	}
 }
 
 static void encode_offload_status(struct xdr_stream *xdr,
fs/nfs/nfs4file.c +6 -1
index b7b8c47..0e15a6d 100644
@@ -134,6 +134,8 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in,
 				    size_t count, unsigned int flags)
 {
 	struct nfs42_copy_notify_res *cn_resp = NULL;
+	struct nl4_servers *nss = NULL;
+	nfs4_stateid *cnrs = NULL;
 	ssize_t ret;
 
 	if (file_inode(file_in) == file_inode(file_out))
@@ -159,8 +161,11 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in,
 		ret = nfs42_proc_copy_notify(file_in, file_out, cn_resp);
 		if (ret)
 			goto out;
+		nss = &cn_resp->cnr_src;
+		cnrs = &cn_resp->cnr_stateid;
 	}
-	ret = nfs42_proc_copy(file_in, pos_in, file_out, pos_out, count);
+	ret = nfs42_proc_copy(file_in, pos_in, file_out, pos_out, count, nss,
+				cnrs);
 
 out:
 	if (cn_resp) {
include/linux/nfs_xdr.h +1 -0
index 300289b..658aaff 100644
@@ -1372,6 +1372,7 @@ struct nfs42_copy_args {
 
 	u64				count;
 	bool				sync;
+	struct nl4_servers		*cp_src;
 };
 
 struct nfs42_write_res {
Signature · 1 line
1.8.3.1
#34 lore raw

RFC v3 19/42 NFS also send OFFLOAD_CANCEL to source server

In reply to Olga Kornievskaia

In case of copy is cancelled, also send OFFLOAD_CANCEL to the source server.

---
 fs/nfs/nfs42proc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
fs/nfs/nfs42proc.c +4 -2
index 36b265a..b7c08a9 100644
@@ -206,12 +206,14 @@ static int handle_async_copy(struct nfs42_copy_res *res,
 	if (copy->count <= 0)
 		status = -copy->error;
 
+out_free:
 	kfree(copy);
 	return status;
 out_cancel:
 	nfs42_do_offload_cancel_async(dst, &copy->stateid);
-	kfree(copy);
-	return status;
+	if (!nfs42_files_from_same_server(src, dst))
+		nfs42_do_offload_cancel_async(src, src_stateid);
+	goto out_free;
 }
 
 static ssize_t _nfs42_proc_copy(struct file *src,
Signature · 1 line
1.8.3.1
#35 lore raw

RFC v3 20/42 NFS inter ssc open

In reply to Olga Kornievskaia

NFSv4.2 inter server to server copy requires the destination server to READ the data from the source server using the provided stateid and file handle.

Given an NFSv4 stateid and filehandle from the COPY operaion, provide the destination server with an NFS client function to create a struct file suitable for the destiniation server to READ the data to be copied.

---
 fs/nfs/nfs4_fs.h  |   7 ++++
 fs/nfs/nfs4file.c | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4proc.c |   5 ++-
 3 files changed, 115 insertions(+), 3 deletions(-)
fs/nfs/nfs4_fs.h +7 -0
index 70fa0e4..822c4cb 100644
@@ -278,6 +278,13 @@ extern int nfs4_set_rw_stateid(nfs4_stateid *stateid,
 		const struct nfs_open_context *ctx,
 		const struct nfs_lock_context *l_ctx,
 		fmode_t fmode);
+extern int nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle,
+				struct nfs_fattr *fattr,
+				struct nfs4_label *label);
+extern int update_open_stateid(struct nfs4_state *state,
+				const nfs4_stateid *open_stateid,
+				const nfs4_stateid *deleg_stateid,
+				fmode_t fmode);
 
 #if defined(CONFIG_NFS_V4_1)
 extern int nfs41_sequence_done(struct rpc_task *, struct nfs4_sequence_res *);
fs/nfs/nfs4file.c +106 -0

118 lines not on this page · raw message

fs/nfs/nfs4proc.c +2 -3
index 94913fd..ecaadb0 100644
@@ -89,7 +89,6 @@
 static int _nfs4_recover_proc_open(struct nfs4_opendata *data);
 static int nfs4_do_fsinfo(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
 static void nfs_fixup_referral_attributes(struct nfs_fattr *fattr);
-static int nfs4_proc_getattr(struct nfs_server *, struct nfs_fh *, struct nfs_fattr *, struct nfs4_label *label);
 static int _nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle, struct nfs_fattr *fattr, struct nfs4_label *label);
 static int nfs4_do_setattr(struct inode *inode, struct rpc_cred *cred,
 			    struct nfs_fattr *fattr, struct iattr *sattr,
@@ -1461,7 +1460,7 @@ static void __update_open_stateid(struct nfs4_state *state,
 	spin_unlock(&state->owner->so_lock);
 }
 
-static int update_open_stateid(struct nfs4_state *state,
+int update_open_stateid(struct nfs4_state *state,
 		const nfs4_stateid *open_stateid,
 		const nfs4_stateid *delegation,
 		fmode_t fmode)
@@ -3615,7 +3614,7 @@ static int _nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle,
 	return nfs4_call_sync(server->client, server, &msg, &args.seq_args, &res.seq_res, 0);
 }
 
-static int nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle,
+int nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle,
 				struct nfs_fattr *fattr, struct nfs4_label *label)
 {
 	struct nfs4_exception exception = { };
Signature · 1 line
1.8.3.1
#36 lore raw

RFC v3 21/42 NFS skip recovery of copy open on dest server

In reply to Olga Kornievskaia

Mark the open created for the source file on the destination server. Then if this open is going thru a recovery, then fail the recovery as we don't need to be recoving a "fake" open. We need to fail the ongoing READs and vfs_copy_file_range().

---
 fs/nfs/nfs4_fs.h   |  1 +
 fs/nfs/nfs4file.c  |  2 +-
 fs/nfs/nfs4state.c | 14 ++++++++++++++
 3 files changed, 16 insertions(+), 1 deletion(-)
fs/nfs/nfs4_fs.h +1 -0
index 822c4cb..4ca9657 100644
@@ -163,6 +163,7 @@ enum {
 	NFS_STATE_MAY_NOTIFY_LOCK,	/* server may CB_NOTIFY_LOCK */
 #ifdef CONFIG_NFS_V4_2
 	NFS_CLNT_DST_SSC_COPY_STATE,	/* dst server open state on client*/
+	NFS_SRV_SSC_COPY_STATE,		/* ssc state on the dst server */
 #endif /* CONFIG_NFS_V4_2 */
 
 };
fs/nfs/nfs4file.c +1 -1
index fa6ce82..49b6d15 100644
@@ -349,7 +349,7 @@ struct file *
 	ctx->state = nfs4_get_open_state(r_ino, sp);
 	if (ctx->state == NULL)
 		goto out_stateowner;
-
+	set_bit(NFS_SRV_SSC_COPY_STATE, &ctx->state->flags);
 	update_open_stateid(ctx->state, stateid, NULL, filep->f_mode);
 
 	nfs_file_set_open_context(filep, ctx);
fs/nfs/nfs4state.c +14 -0
index f640050..2c8556e 100644
@@ -1501,6 +1501,9 @@ static int nfs4_reclaim_open_state(struct nfs4_state_owner *sp, const struct nfs
 	struct nfs4_state *state;
 	struct nfs4_lock_state *lock;
 	int status = 0;
+#ifdef CONFIG_NFS_V4_2
+	bool found_ssc_copy_state = false;
+#endif /* CONFIG_NFS_V4_2 */
 
 	/* Note: we rely on the sp->so_states list being ordered 
 	 * so that we always reclaim open(O_RDWR) and/or open(O_WRITE)
@@ -1520,6 +1523,13 @@ static int nfs4_reclaim_open_state(struct nfs4_state_owner *sp, const struct nfs
 			continue;
 		if (state->state == 0)
 			continue;
+#ifdef CONFIG_NFS_V4_2
+		if (test_bit(NFS_SRV_SSC_COPY_STATE, &state->flags)) {
+			nfs4_state_mark_recovery_failed(state, -EIO);
+			found_ssc_copy_state = true;
+			continue;
+		}
+#endif /* CONFIG_NFS_V4_2 */
 		atomic_inc(&state->count);
 		spin_unlock(&sp->so_lock);
 		status = ops->recover_open(sp, state);
@@ -1597,6 +1607,10 @@ static int nfs4_reclaim_open_state(struct nfs4_state_owner *sp, const struct nfs
 	}
 	raw_write_seqcount_end(&sp->so_reclaim_seqcount);
 	spin_unlock(&sp->so_lock);
+#ifdef CONFIG_NFS_V4_2
+	if (found_ssc_copy_state)
+		return -EIO;
+#endif /* CONFIG_NFS_V4_2 */
 	return 0;
 out_err:
 	nfs4_put_open_state(state);
Signature · 1 line
1.8.3.1
#37 lore raw

RFC v3 22/42 NFS for "inter" copy treat ESTALE as ENOTSUPP

In reply to Olga Kornievskaia

If the client sends an "inter" copy to the destination server but it only supports "intra" copy, it can return ESTALE (since it doesn't know anything about the file handle from the other server and does not recognize the special case of "inter" copy). Translate this error as ENOTSUPP and also send OFFLOAD_CANCEL to teh source server so that it can clean up state.

---
 fs/nfs/nfs42proc.c | 7 +++++++
 1 file changed, 7 insertions(+)
fs/nfs/nfs42proc.c +7 -0
index b7c08a9..09f653d 100644
@@ -358,6 +358,13 @@ ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,
 				continue;
 			}
 			break;
+		case -ESTALE:
+			if (!nfs42_files_from_same_server(src, dst)) {
+				err = -ENOTSUPP;
+				nfs42_do_offload_cancel_async(src,
+					&args.src_stateid);
+			}
+			break;
 		case -ENOTSUPP:
 			server->caps &= ~NFS_CAP_COPY;
 			if (nfs42_files_from_same_server(src, dst))
Signature · 1 line
1.8.3.1
#38 lore raw

RFC v3 23/42 NFS add a simple sync nfs4_proc_commit after async COPY

In reply to Olga Kornievskaia

A COPY with unstable write data needs a simple commit that doesnt deal with inodes

---
 fs/nfs/nfs42proc.c | 22 ++++++++++++++++++++++
 fs/nfs/nfs4_fs.h   |  2 +-
 fs/nfs/nfs4proc.c  | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+), 1 deletion(-)
fs/nfs/nfs42proc.c +22 -0
index 09f653d..590bd50 100644
@@ -289,6 +289,28 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 			return status;
 	}
 
+	if ((!res->synchronous || !args->sync) &&
+			res->write_res.verifier.committed != NFS_FILE_SYNC) {
+		struct nfs_commitres cres;
+
+		cres.verf = kzalloc(sizeof(struct nfs_writeverf), GFP_NOFS);
+		if (!cres.verf)
+			return -ENOMEM;
+
+		status = nfs4_proc_commit(dst, pos_dst, res->write_res.count,
+					  &cres);
+		if (status) {
+			kfree(cres.verf);
+			return status;
+		}
+		if (!nfs_write_verifier_cmp(&res->write_res.verifier.verifier,
+					    &cres.verf->verifier)) {
+			/* what are we suppose to do here ? */
+			dprintk("commit verf differs from copy verf\n");
+		}
+		kfree(cres.verf);
+	}
+
 	truncate_pagecache_range(dst_inode, pos_dst,
 				 pos_dst + res->write_res.count);
 
fs/nfs/nfs4_fs.h +1 -1
index 4ca9657..1b817b4 100644
@@ -477,7 +477,7 @@ extern int nfs4_sequence_done(struct rpc_task *task,
 			      struct nfs4_sequence_res *res);
 
 extern void nfs4_free_lock_state(struct nfs_server *server, struct nfs4_lock_state *lsp);
-
+extern int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 count, struct nfs_commitres *res);
 extern const nfs4_stateid zero_stateid;
 
 /* nfs4super.c */
fs/nfs/nfs4proc.c +33 -0
index ecaadb0..c477333 100644
@@ -4749,6 +4749,39 @@ static void nfs4_proc_commit_setup(struct nfs_commit_data *data, struct rpc_mess
 	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 }
 
+static int _nfs4_proc_commit(struct file *dst, struct nfs_commitargs *args,
+				struct nfs_commitres *res)
+{
+	struct inode *dst_inode = file_inode(dst);
+	struct nfs_server *server = NFS_SERVER(dst_inode);
+	struct rpc_message msg = {
+		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_COMMIT],
+		.rpc_argp = args,
+		.rpc_resp = res,
+	};
+	return nfs4_call_sync(server->client, server, &msg,
+			&args->seq_args, &res->seq_res, 1);
+}
+
+int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 count, struct nfs_commitres *res)
+{
+	struct nfs_commitargs args = {
+		.fh = NFS_FH(file_inode(dst)),
+		.offset = offset,
+		.count = count,
+	};
+	struct nfs_server *dst_server = NFS_SERVER(file_inode(dst));
+	struct nfs4_exception exception = { };
+	int status;
+
+	do {
+		status = _nfs4_proc_commit(dst, &args, res);
+		status = nfs4_handle_exception(dst_server, status, &exception);
+	} while (exception.retry);
+
+	return status;
+}
+
 struct nfs4_renewdata {
 	struct nfs_client	*client;
 	unsigned long		timestamp;
Signature · 1 line
1.8.3.1
#39 lore raw

Re: [RFC v3 23/42] NFS add a simple sync nfs4_proc_commit after async COPY

Hi Olga,

Quoted 81 lines · Olga Kornievskaia

A COPY with unstable write data needs a simple commit that doesnt deal with inodes

---
 fs/nfs/nfs42proc.c | 22 ++++++++++++++++++++++
 fs/nfs/nfs4_fs.h   |  2 +-
 fs/nfs/nfs4proc.c  | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+), 1 deletion(-)
fs/nfs/nfs42proc.c +22 -0
index 09f653d..590bd50 100644
@@ -289,6 +289,28 @@ static ssize_t _nfs42_proc_copy(struct file *src,
 			return status;
 	}
 
+	if ((!res->synchronous || !args->sync) &&
+			res->write_res.verifier.committed != NFS_FILE_SYNC) {
+		struct nfs_commitres cres;
+
+		cres.verf = kzalloc(sizeof(struct nfs_writeverf), GFP_NOFS);
+		if (!cres.verf)
+			return -ENOMEM;
+
+		status = nfs4_proc_commit(dst, pos_dst, res->write_res.count,
+					  &cres);
+		if (status) {
+			kfree(cres.verf);
+			return status;
+		}
+		if (!nfs_write_verifier_cmp(&res->write_res.verifier.verifier,
+					    &cres.verf->verifier)) {
+			/* what are we suppose to do here ? */
+			dprintk("commit verf differs from copy verf\n");
+		}
+		kfree(cres.verf);
+	}
+
 	truncate_pagecache_range(dst_inode, pos_dst,
 				 pos_dst + res->write_res.count);
 
fs/nfs/nfs4_fs.h +1 -1
index 4ca9657..1b817b4 100644
@@ -477,7 +477,7 @@ extern int nfs4_sequence_done(struct rpc_task *task,
 			      struct nfs4_sequence_res *res);
 
 extern void nfs4_free_lock_state(struct nfs_server *server, struct nfs4_lock_state *lsp);
-
+extern int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 count, struct nfs_commitres *res);
 extern const nfs4_stateid zero_stateid;
 
 /* nfs4super.c */
fs/nfs/nfs4proc.c +17 -0
index ecaadb0..c477333 100644
@@ -4749,6 +4749,39 @@ static void nfs4_proc_commit_setup(struct nfs_commit_data *data, struct rpc_mess
 	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 }
 
+static int _nfs4_proc_commit(struct file *dst, struct nfs_commitargs *args,
+				struct nfs_commitres *res)
+{
+	struct inode *dst_inode = file_inode(dst);
+	struct nfs_server *server = NFS_SERVER(dst_inode);
+	struct rpc_message msg = {
+		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_COMMIT],
+		.rpc_argp = args,
+		.rpc_resp = res,
+	};
+	return nfs4_call_sync(server->client, server, &msg,
+			&args->seq_args, &res->seq_res, 1);
+}
+
+int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 count, struct nfs_commitres *res)
+{
+	struct nfs_commitargs args = {

fs/nfs/nfs4proc.c static void nfs4_proc_commit_setup(struct nfs_commit_data *data, struct rpc_mess · line 4778

Quoted 10 lines
fs/nfs/nfs4proc.c +10 -0
+		.fh = NFS_FH(file_inode(dst)),
+		.offset = offset,
+		.count = count,
+	};
+	struct nfs_server *dst_server = NFS_SERVER(file_inode(dst));
+	struct nfs4_exception exception = { };
+	int status;
+
+	do {
+		status = _nfs4_proc_commit(dst, &args, res);
                ^^^^^^

Do you want to check the status here before continuing on to nfs4_handle_exception()?

Thanks,
Anna

Quoted 10 lines
fs/nfs/nfs4proc.c +6 -0
+		status = nfs4_handle_exception(dst_server, status, &exception);
+	} while (exception.retry);
+
+	return status;
+}
+
 struct nfs4_renewdata {
 	struct nfs_client	*client;
 	unsigned long		timestamp;
#40 lore raw

Re: [RFC v3 23/42] NFS add a simple sync nfs4_proc_commit after async COPY

Quoted 2 lines

On Jul 12, 2017, at 1:13 PM, Anna Schumaker <[email protected]> =

Quoted 4 lines · wrote:

=20
Hi Olga,
=20
On 07/11/2017 12:43 PM, Olga Kornievskaia wrote:

Quoted 15 lines

fs/nfs/nfs42proc.c static ssize_t _nfs42_proc_copy(struct file =

Quoted 15 lines

A COPY with unstable write data needs a simple commit that doesnt deal with inodes
=20

---

fs/nfs/nfs42proc.c | 22 ++++++++++++++++++++++
fs/nfs/nfs4_fs.h   |  2 +-
fs/nfs/nfs4proc.c  | 33 +++++++++++++++++++++++++++++++++
3 files changed, 56 insertions(+), 1 deletion(-)

=20

fs/nfs/nfs42proc.c +0 -0
index 09f653d..590bd50 100644
@@ -289,6 +289,28 @@ static ssize_t _nfs42_proc_copy(struct file =

*src,

Quoted 5 lines

fs/nfs/nfs42proc.c

Quoted 5 lines
fs/nfs/nfs42proc.c +0 -0
 			return status;
 	}

=20

fs/nfs/nfs42proc.c +2 -0
+	if ((!res->synchronous || !args->sync) &&
+			res->write_res.verifier.committed !=3D =

NFS_FILE_SYNC) {

Quoted 3 lines

fs/nfs/nfs42proc.c static ssize_t _nfs42_proc_copy(struct file *src,

Quoted 3 lines
fs/nfs/nfs42proc.c +3 -0
+		struct nfs_commitres cres;
+
+		cres.verf =3D kzalloc(sizeof(struct nfs_writeverf), =

GFP_NOFS);

Quoted 4 lines

fs/nfs/nfs42proc.c static ssize_t _nfs42_proc_copy(struct file *src,

Quoted 4 lines
fs/nfs/nfs42proc.c +4 -0
+		if (!cres.verf)
+			return -ENOMEM;
+
+		status =3D nfs4_proc_commit(dst, pos_dst, =

res->write_res.count,

Quoted 6 lines

fs/nfs/nfs42proc.c static ssize_t _nfs42_proc_copy(struct file *src,

Quoted 6 lines
fs/nfs/nfs42proc.c +6 -0
+					  &cres);
+		if (status) {
+			kfree(cres.verf);
+			return status;
+		}
+		if =

(!nfs_write_verifier_cmp(&res->write_res.verifier.verifier,

Quoted 7 lines
Quoted 7 lines
fs/nfs/nfs42proc.c +7 -0
+					    &cres.verf->verifier)) {
+			/* what are we suppose to do here ? */
+			dprintk("commit verf differs from copy verf\n");
+		}
+		kfree(cres.verf);
+	}
+
Quoted 8 lines

fs/nfs/nfs4_fs.h extern int nfs4_sequence_done(struct rpc_task =

Quoted 8 lines
	truncate_pagecache_range(dst_inode, pos_dst,
				 pos_dst + res->write_res.count);

=20

fs/nfs/nfs4_fs.h +0 -0
index 4ca9657..1b817b4 100644
@@ -477,7 +477,7 @@ extern int nfs4_sequence_done(struct rpc_task =

*task,

Quoted 3 lines
Quoted 3 lines
fs/nfs/nfs4_fs.h +0 -0
 			      struct nfs4_sequence_res *res);

=20
extern void nfs4_free_lock_state(struct nfs_server *server, struct =

nfs4_lock_state *lsp);

Quoted 2 lines

fs/nfs/nfs4_fs.h

Quoted 2 lines
fs/nfs/nfs4_fs.h +1 -1
-
+extern int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 =

count, struct nfs_commitres *res);

Quoted 8 lines

fs/nfs/nfs4proc.c static void nfs4_proc_commit_setup(struct =

Quoted 8 lines

extern const nfs4_stateid zero_stateid;
=20
/* nfs4super.c */

fs/nfs/nfs4proc.c +0 -0
index ecaadb0..c477333 100644
@@ -4749,6 +4749,39 @@ static void nfs4_proc_commit_setup(struct =

nfs_commit_data *data, struct rpc_mess

Quoted 4 lines
Quoted 4 lines
fs/nfs/nfs4proc.c +0 -0
 	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 }

=20
+static int _nfs4_proc_commit(struct file *dst, struct nfs_commitargs =

*args,

Quoted 6 lines
Quoted 6 lines
fs/nfs/nfs4proc.c +6 -0
+				struct nfs_commitres *res)
+{
+	struct inode *dst_inode =3D file_inode(dst);
+	struct nfs_server *server =3D NFS_SERVER(dst_inode);
+	struct rpc_message msg =3D {
+		.rpc_proc =3D &nfs4_procedures[NFSPROC4_CLNT_COMMIT],
Quoted 8 lines

fs/nfs/nfs4proc.c static void nfs4_proc_commit_setup(struct nfs_commit_data *data, struct rpc_mess

Quoted 8 lines
fs/nfs/nfs4proc.c +8 -0
+		.rpc_argp =3D args,
+		.rpc_resp =3D res,
+	};
+	return nfs4_call_sync(server->client, server, &msg,
+			&args->seq_args, &res->seq_res, 1);
+}
+
+int nfs4_proc_commit(struct file *dst, __u64 offset, __u32 count, =

struct nfs_commitres *res)

Quoted 13 lines

fs/nfs/nfs4proc.c

Quoted 12 lines
fs/nfs/nfs4proc.c +12 -0
+{
+	struct nfs_commitargs args =3D {
+		.fh =3D NFS_FH(file_inode(dst)),
+		.offset =3D offset,
+		.count =3D count,
+	};
+	struct nfs_server *dst_server =3D NFS_SERVER(file_inode(dst));
+	struct nfs4_exception exception =3D { };
+	int status;
+
+	do {
+		status =3D _nfs4_proc_commit(dst, &args, res);
               ^^^^^^
Quoted 1 line

Do you want to check the status here before continuing on to =

nfs4_handle_exception()?

No, the exception handler should be called with the status and deal with = it and the loop exit will be determined by it.=20

Quoted 5 lines

=20
Thanks,
Anna
=20

fs/nfs/nfs4proc.c

Quoted 1 line
fs/nfs/nfs4proc.c +1 -0
+		status =3D nfs4_handle_exception(dst_server, status, =

&exception);

Quoted 9 lines
Quoted 9 lines
fs/nfs/nfs4proc.c +5 -0
+	} while (exception.retry);
+
+	return status;
+}
+
struct nfs4_renewdata {
	struct nfs_client	*client;
	unsigned long		timestamp;

=20

#41 lore raw

RFC v3 24/42 NFSD add ca_source_server<> to COPY

In reply to Olga Kornievskaia

From: Andy Adamson <[email protected]>

Note: followed conventions and have struct nfsd4_compoundargs pointer as a parameter even though it is unused.

---
 fs/nfsd/nfs4xdr.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 fs/nfsd/xdr4.h    |  4 +++
 2 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4xdr.c +73 -2
index 26780d5..031d06d 100644
@@ -41,6 +41,7 @@
 #include <linux/utsname.h>
 #include <linux/pagemap.h>
 #include <linux/sunrpc/svcauth_gss.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "acl.h"
@@ -1726,11 +1727,58 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str
 	DECODE_TAIL;
 }
 
+static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
+				      struct nl4_server *ns)
+{
+	DECODE_HEAD;
+	struct nfs42_netaddr *naddr;
+
+	READ_BUF(4);
+	ns->nl4_type = be32_to_cpup(p++);
+
+	/* currently support for 1 inter-server source server */
+	switch (ns->nl4_type) {
+	case NL4_NAME:
+	case NL4_URL:
+		READ_BUF(4);
+		ns->u.nl4_str_sz = be32_to_cpup(p++);
+		if (ns->u.nl4_str_sz > NFS4_OPAQUE_LIMIT)
+			goto xdr_error;
+
+		READ_BUF(ns->u.nl4_str_sz);
+		COPYMEM(ns->u.nl4_str,
+			ns->u.nl4_str_sz);
+		break;
+	case NL4_NETADDR:
+		naddr = &ns->u.nl4_addr;
+
+		READ_BUF(4);
+		naddr->na_netid_len = be32_to_cpup(p++);
+		if (naddr->na_netid_len > RPCBIND_MAXNETIDLEN)
+			goto xdr_error;
+
+		READ_BUF(naddr->na_netid_len + 4); /* 4 for uaddr len */
+		COPYMEM(naddr->na_netid, naddr->na_netid_len);
+
+		naddr->na_uaddr_len = be32_to_cpup(p++);
+		if (naddr->na_uaddr_len > RPCBIND_MAXUADDRLEN)
+			goto xdr_error;
+
+		READ_BUF(naddr->na_uaddr_len);
+		COPYMEM(naddr->na_uaddr, naddr->na_uaddr_len);
+		break;
+	default:
+		goto xdr_error;
+	}
+	DECODE_TAIL;
+}
+
 static __be32
 nfsd4_decode_copy(struct nfsd4_compoundargs *argp, struct nfsd4_copy *copy)
 {
 	DECODE_HEAD;
-	unsigned int tmp;
+	struct nl4_server *ns;
+	int i;
 
 	status = nfsd4_decode_stateid(argp, &copy->cp_src_stateid);
 	if (status)
@@ -1745,8 +1793,29 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str
 	p = xdr_decode_hyper(p, &copy->cp_count);
 	copy->cp_consecutive = be32_to_cpup(p++);
 	copy->cp_synchronous = be32_to_cpup(p++);
-	tmp = be32_to_cpup(p); /* Source server list not supported */
+	copy->cp_src.nl_nsvr = be32_to_cpup(p++);
 
+	if (copy->cp_src.nl_nsvr == 0) /* intra-server copy */
+		goto intra;
+
+	/** Support NFSD4_MAX_SSC_SRC number of source servers.
+	 * freed in nfsd4_encode_copy
+	 */
+	if (copy->cp_src.nl_nsvr > NFSD4_MAX_SSC_SRC)
+		copy->cp_src.nl_nsvr = NFSD4_MAX_SSC_SRC;
+	copy->cp_src.nl_svr = kmalloc(copy->cp_src.nl_nsvr *
+					sizeof(struct nl4_server), GFP_KERNEL);
+	if (copy->cp_src.nl_svr == NULL)
+		return nfserrno(-ENOMEM);
+
+	ns = copy->cp_src.nl_svr;
+	for (i = 0; i < copy->cp_src.nl_nsvr; i++) {
+		status = nfsd4_decode_nl4_server(argp, ns);
+		if (status)
+			return status;
+		ns++;
+	}
+intra:
 	DECODE_TAIL;
 }
 
@@ -4300,6 +4369,8 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 		*p++ = cpu_to_be32(copy->cp_consecutive);
 		*p++ = cpu_to_be32(copy->cp_synchronous);
 	}
+	/* allocated in nfsd4_decode_copy */
+	kfree(copy->cp_src.nl_svr);
 	return nfserr;
 }
 
fs/nfsd/xdr4.h +4 -0
index 8fda4ab..6b1a61fc 100644
@@ -509,6 +509,9 @@ struct nfsd42_write_res {
 	nfs4_verifier		wr_verifier;
 };
 
+/*  support 1 source server for now */
+#define NFSD4_MAX_SSC_SRC       1
+
 struct nfsd4_copy {
 	/* request */
 	stateid_t	cp_src_stateid;
@@ -516,6 +519,7 @@ struct nfsd4_copy {
 	u64		cp_src_pos;
 	u64		cp_dst_pos;
 	u64		cp_count;
+	struct nl4_servers cp_src;
 
 	/* both */
 	bool		cp_consecutive;
Signature · 1 line
1.8.3.1
#42 lore raw

RFC v3 25/42 NFSD add COPY_NOTIFY operation

In reply to Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c |  98 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/xdr4.h     |  13 +++++++
 3 files changed, 221 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +98 -0
index dadb3bf..ed6b9f2 100644
@@ -35,6 +35,7 @@
 #include <linux/file.h>
 #include <linux/falloc.h>
 #include <linux/slab.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "cache.h"
@@ -1087,6 +1088,82 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 }
 
 static __be32
+nfsd4_set_src_nl4_netaddr(struct svc_rqst *rqstp, struct nl4_servers *svrs)
+{
+	const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr;
+	const struct sockaddr_in *sin = (const struct sockaddr_in *)addr;
+	const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr;
+	int uaddr_len = rqstp->rq_daddrlen + 4 + 1; /* port (4) and '\0' (1) */
+	struct nfs42_netaddr *naddr;
+	size_t ret;
+	unsigned short port;
+
+	/* freed in nfsd4_encode_copy_notify */
+	svrs->nl_svr = kmalloc_array(svrs->nl_nsvr, sizeof(struct nl4_server),
+				GFP_KERNEL);
+	if (svrs->nl_svr == NULL)
+		return nfserrno(-ENOMEM);
+
+	svrs->nl_svr->nl4_type = NL4_NETADDR;
+	naddr = &svrs->nl_svr->u.nl4_addr;
+
+	switch (addr->sa_family) {
+	case AF_INET:
+		port = ntohs(sin->sin_port);
+		sprintf(naddr->na_netid, "tcp");
+		naddr->na_netid_len = 3;
+		break;
+	case AF_INET6:
+		port = ntohs(sin6->sin6_port);
+		sprintf(naddr->na_netid, "tcp6");
+		naddr->na_netid_len = 4;
+		break;
+	default:
+		dprintk("NFSD  nfsd4_set_notify_src: unknown address type: %d",
+			addr->sa_family);
+		kfree(svrs->nl_svr);
+		return nfserrno(-EINVAL);
+	}
+	ret = rpc_ntop(addr, naddr->na_uaddr, sizeof(naddr->na_uaddr));
+	snprintf(naddr->na_uaddr + ret, uaddr_len, ".%u.%u",
+		 port >> 8, port & 255);
+	naddr->na_uaddr_len = strlen(naddr->na_uaddr);
+	return 0;
+}
+
+static __be32
+nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+		  struct nfsd4_copy_notify *cn)
+{
+	__be32 status;
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+
+	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+					&cn->cpn_src_stateid, RD_STATE, NULL,
+					NULL);
+	if (status)
+		return status;
+
+	cn->cpn_sec = nn->nfsd4_lease;
+	cn->cpn_nsec = 0;
+
+
+	/** XXX Save cpn_src_statid, cpn_src, and any other returned source
+	 * server addresses on which the source server is williing to accept
+	 * connections from the destination e.g. what is returned in cpn_src,
+	 * to verify READ from dest server.
+	 */
+
+	/**
+	 * For now, only return one server address in cpn_src, the
+	 * address used by the client to connect to this server.
+	 */
+	cn->cpn_src.nl_nsvr = 1;
+
+	return nfsd4_set_src_nl4_netaddr(rqstp, &cn->cpn_src);
+}
+
+static __be32
 nfsd4_fallocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		struct nfsd4_fallocate *fallocate, int flags)
 {
@@ -2042,6 +2119,21 @@ static inline u32 nfsd4_copy_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		1 /* cr_synchronous */) * sizeof(__be32);
 }
 
+static inline u32 nfsd4_copy_notify_rsize(struct svc_rqst *rqstp,
+					struct nfsd4_op *op)
+{
+	return (op_encode_hdr_size +
+		3 /* cnr_lease_time */ +
+		1 /* We support one cnr_source_server */ +
+		1 /* cnr_stateid seq */ +
+		op_encode_stateid_maxsz /* cnr_stateid */ +
+		1 /* num cnr_source_server*/ +
+		1 /* nl4_type */ +
+		1 /* nl4 size */ +
+		XDR_QUADLEN(NFS4_OPAQUE_LIMIT) /*nl4_loc + nl4_loc_sz */)
+		* sizeof(__be32);
+}
+
 #ifdef CONFIG_NFSD_PNFS
 static inline u32 nfsd4_getdeviceinfo_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 {
@@ -2446,6 +2538,12 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		.op_name = "OP_SEEK",
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_seek_rsize,
 	},
+	[OP_COPY_NOTIFY] = {
+		.op_func = (nfsd4op_func)nfsd4_copy_notify,
+		.op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME,
+		.op_name = "OP_COPY_NOTIFY",
+		.op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize,
+	},
 };
 
 /**
fs/nfsd/nfs4xdr.c +110 -2

142 lines not on this page · raw message

fs/nfsd/xdr4.h +13 -0
index 6b1a61fc..5db7cd8 100644
@@ -540,6 +540,18 @@ struct nfsd4_seek {
 	loff_t		seek_pos;
 };
 
+struct nfsd4_copy_notify {
+	/* request */
+	stateid_t		cpn_src_stateid;
+	struct nl4_server	cpn_dst;
+
+	/* response */
+	/* Note: cpn_src_stateid is used for cnr_stateid */
+	u64			cpn_sec;
+	u32			cpn_nsec;
+	struct nl4_servers	cpn_src;
+};
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -595,6 +607,7 @@ struct nfsd4_op {
 		struct nfsd4_fallocate		deallocate;
 		struct nfsd4_clone		clone;
 		struct nfsd4_copy		copy;
+		struct nfsd4_copy_notify	copy_notify;
 		struct nfsd4_seek		seek;
 	} u;
 	struct nfs4_replay *			replay;
Signature · 1 line
1.8.3.1
#43 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

Quoted 53 lines · Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c |  98 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/xdr4.h     |  13 +++++++
 3 files changed, 221 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +31 -0
index dadb3bf..ed6b9f2 100644
@@ -35,6 +35,7 @@
 #include <linux/file.h>
 #include <linux/falloc.h>
 #include <linux/slab.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "cache.h"
@@ -1087,6 +1088,82 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 }
 
 static __be32
+nfsd4_set_src_nl4_netaddr(struct svc_rqst *rqstp, struct nl4_servers *svrs)
+{
+	const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr;
+	const struct sockaddr_in *sin = (const struct sockaddr_in *)addr;
+	const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr;
+	int uaddr_len = rqstp->rq_daddrlen + 4 + 1; /* port (4) and '\0' (1) */
+	struct nfs42_netaddr *naddr;
+	size_t ret;
+	unsigned short port;
+
+	/* freed in nfsd4_encode_copy_notify */
+	svrs->nl_svr = kmalloc_array(svrs->nl_nsvr, sizeof(struct nl4_server),
+				GFP_KERNEL);
+	if (svrs->nl_svr == NULL)
+		return nfserrno(-ENOMEM);
+
+	svrs->nl_svr->nl4_type = NL4_NETADDR;
+	naddr = &svrs->nl_svr->u.nl4_addr;
+
+	switch (addr->sa_family) {
+	case AF_INET:
+		port = ntohs(sin->sin_port);
+		sprintf(naddr->na_netid, "tcp");
+		naddr->na_netid_len = 3;
+		break;
+	case AF_INET6:
+		port = ntohs(sin6->sin6_port);
+		sprintf(naddr->na_netid, "tcp6");
+		naddr->na_netid_len = 4;
+		break;

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1130

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	default:
+		dprintk("NFSD  nfsd4_set_notify_src: unknown address type: %d",
+			addr->sa_family);
+		kfree(svrs->nl_svr);
+		return nfserrno(-EINVAL);
+	}
+	ret = rpc_ntop(addr, naddr->na_uaddr, sizeof(naddr->na_uaddr));
+	snprintf(naddr->na_uaddr + ret, uaddr_len, ".%u.%u",
+		 port >> 8, port & 255);
+	naddr->na_uaddr_len = strlen(naddr->na_uaddr);

I'm a little surprised there isn't already a standard helper for this.... Actually, does net/sunrpc:addr.c:rpc_sockaddr2uaddr() do it?

(Looks like there's other potential users in fs/nfsd/flexfilelayout.c:nfsd4_ff_proc_getdeviceinfo() and in flexfiles code on client side and probably some other places too.)

--b.

Quoted 263 lines
fs/nfsd/nfs4proc.c +57 -0
+	return 0;
+}
+
+static __be32
+nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+		  struct nfsd4_copy_notify *cn)
+{
+	__be32 status;
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+
+	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+					&cn->cpn_src_stateid, RD_STATE, NULL,
+					NULL);
+	if (status)
+		return status;
+
+	cn->cpn_sec = nn->nfsd4_lease;
+	cn->cpn_nsec = 0;
+
+
+	/** XXX Save cpn_src_statid, cpn_src, and any other returned source
+	 * server addresses on which the source server is williing to accept
+	 * connections from the destination e.g. what is returned in cpn_src,
+	 * to verify READ from dest server.
+	 */
+
+	/**
+	 * For now, only return one server address in cpn_src, the
+	 * address used by the client to connect to this server.
+	 */
+	cn->cpn_src.nl_nsvr = 1;
+
+	return nfsd4_set_src_nl4_netaddr(rqstp, &cn->cpn_src);
+}
+
+static __be32
 nfsd4_fallocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		struct nfsd4_fallocate *fallocate, int flags)
 {
@@ -2042,6 +2119,21 @@ static inline u32 nfsd4_copy_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		1 /* cr_synchronous */) * sizeof(__be32);
 }
 
+static inline u32 nfsd4_copy_notify_rsize(struct svc_rqst *rqstp,
+					struct nfsd4_op *op)
+{
+	return (op_encode_hdr_size +
+		3 /* cnr_lease_time */ +
+		1 /* We support one cnr_source_server */ +
+		1 /* cnr_stateid seq */ +
+		op_encode_stateid_maxsz /* cnr_stateid */ +
+		1 /* num cnr_source_server*/ +
+		1 /* nl4_type */ +
+		1 /* nl4 size */ +
+		XDR_QUADLEN(NFS4_OPAQUE_LIMIT) /*nl4_loc + nl4_loc_sz */)
+		* sizeof(__be32);
+}
+
 #ifdef CONFIG_NFSD_PNFS
 static inline u32 nfsd4_getdeviceinfo_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 {
@@ -2446,6 +2538,12 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		.op_name = "OP_SEEK",
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_seek_rsize,
 	},
+	[OP_COPY_NOTIFY] = {
+		.op_func = (nfsd4op_func)nfsd4_copy_notify,
+		.op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME,
+		.op_name = "OP_COPY_NOTIFY",
+		.op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize,
+	},
 };
 
 /**
fs/nfsd/nfs4xdr.c +110 -2
index 031d06d..3e08c15 100644
@@ -1820,6 +1820,22 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 }
 
 static __be32
+nfsd4_decode_copy_notify(struct nfsd4_compoundargs *argp,
+			 struct nfsd4_copy_notify *cn)
+{
+	int status;
+
+	status = nfsd4_decode_stateid(argp, &cn->cpn_src_stateid);
+	if (status)
+		return status;
+	status = nfsd4_decode_nl4_server(argp, &cn->cpn_dst);
+	if (status)
+		return status;
+
+	return status;
+}
+
+static __be32
 nfsd4_decode_seek(struct nfsd4_compoundargs *argp, struct nfsd4_seek *seek)
 {
 	DECODE_HEAD;
@@ -1920,7 +1936,7 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 	/* new operations for NFSv4.2 */
 	[OP_ALLOCATE]		= (nfsd4_dec)nfsd4_decode_fallocate,
 	[OP_COPY]		= (nfsd4_dec)nfsd4_decode_copy,
-	[OP_COPY_NOTIFY]	= (nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_COPY_NOTIFY]	= (nfsd4_dec)nfsd4_decode_copy_notify,
 	[OP_DEALLOCATE]		= (nfsd4_dec)nfsd4_decode_fallocate,
 	[OP_IO_ADVISE]		= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_LAYOUTERROR]	= (nfsd4_dec)nfsd4_decode_notsupp,
@@ -4355,6 +4371,52 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd42_encode_nl4_server(struct nfsd4_compoundres *resp, struct nl4_server *ns)
+{
+	struct xdr_stream *xdr = &resp->xdr;
+	struct nfs42_netaddr *addr;
+	__be32 *p;
+
+	p = xdr_reserve_space(xdr, 4);
+	*p++ = cpu_to_be32(ns->nl4_type);
+
+	switch (ns->nl4_type) {
+	case NL4_NAME:
+	case NL4_URL:
+		p = xdr_reserve_space(xdr, 4 /* url or name len */ +
+				      (XDR_QUADLEN(ns->u.nl4_str_sz) * 4));
+		if (!p)
+			return nfserr_resource;
+		*p++ = cpu_to_be32(ns->u.nl4_str_sz);
+		p = xdr_encode_opaque_fixed(p, ns->u.nl4_str, ns->u.nl4_str_sz);
+			break;
+	case NL4_NETADDR:
+		addr = &ns->u.nl4_addr;
+
+		/** netid_len, netid, uaddr_len, uaddr (port included
+		 * in RPCBIND_MAXUADDRLEN)
+		 */
+		p = xdr_reserve_space(xdr,
+			4 /* netid len */ +
+			(XDR_QUADLEN(addr->na_netid_len) * 4) +
+			4 /* uaddr len */ +
+			(XDR_QUADLEN(addr->na_uaddr_len) * 4));
+		if (!p)
+			return nfserr_resource;
+
+		*p++ = cpu_to_be32(addr->na_netid_len);
+		p = xdr_encode_opaque_fixed(p, addr->na_netid,
+					    addr->na_netid_len);
+		*p++ = cpu_to_be32(addr->na_uaddr_len);
+		p = xdr_encode_opaque_fixed(p, addr->na_uaddr,
+					addr->na_uaddr_len);
+		break;
+	}
+
+	return 0;
+}
+
+static __be32
 nfsd4_encode_copy(struct nfsd4_compoundres *resp, __be32 nfserr,
 		  struct nfsd4_copy *copy)
 {
@@ -4375,6 +4437,52 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd4_encode_copy_notify(struct nfsd4_compoundres *resp, __be32 nfserr,
+			 struct nfsd4_copy_notify *cn)
+{
+	struct xdr_stream *xdr = &resp->xdr;
+	struct nl4_server *ns;
+	__be32 *p;
+	int i;
+
+	if (nfserr)
+		return nfserr;
+
+	/* 8 sec, 4 nsec */
+	p = xdr_reserve_space(xdr, 12);
+	if (!p)
+		return nfserr_resource;
+
+	/* cnr_lease_time */
+	p = xdr_encode_hyper(p, cn->cpn_sec);
+	*p++ = cpu_to_be32(cn->cpn_nsec);
+
+	/* cnr_stateid */
+	nfserr = nfsd4_encode_stateid(xdr, &cn->cpn_src_stateid);
+	if (nfserr)
+		return nfserr;
+
+	/* cnr_src.nl_nsvr */
+	p = xdr_reserve_space(xdr, 4);
+	if (!p)
+		return nfserr_resource;
+
+	*p++ = cpu_to_be32(cn->cpn_src.nl_nsvr);
+
+	ns = cn->cpn_src.nl_svr;
+	for (i = 0; i < cn->cpn_src.nl_nsvr; i++) {
+		nfserr = nfsd42_encode_nl4_server(resp, ns);
+		if (nfserr)
+			return nfserr;
+		ns++;
+	}
+
+	/* allocated in nfsd4_copy_notify */
+	kfree(cn->cpn_src.nl_svr);
+	return nfserr;
+}
+
+static __be32
 nfsd4_encode_seek(struct nfsd4_compoundres *resp, __be32 nfserr,
 		  struct nfsd4_seek *seek)
 {
@@ -4474,7 +4582,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 	/* NFSv4.2 operations */
 	[OP_ALLOCATE]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_COPY]		= (nfsd4_enc)nfsd4_encode_copy,
-	[OP_COPY_NOTIFY]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_COPY_NOTIFY]	= (nfsd4_enc)nfsd4_encode_copy_notify,
 	[OP_DEALLOCATE]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_IO_ADVISE]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_LAYOUTERROR]	= (nfsd4_enc)nfsd4_encode_noop,
fs/nfsd/xdr4.h +13 -0
index 6b1a61fc..5db7cd8 100644
@@ -540,6 +540,18 @@ struct nfsd4_seek {
 	loff_t		seek_pos;
 };
 
+struct nfsd4_copy_notify {
+	/* request */
+	stateid_t		cpn_src_stateid;
+	struct nl4_server	cpn_dst;
+
+	/* response */
+	/* Note: cpn_src_stateid is used for cnr_stateid */
+	u64			cpn_sec;
+	u32			cpn_nsec;
+	struct nl4_servers	cpn_src;
+};
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -595,6 +607,7 @@ struct nfsd4_op {
 		struct nfsd4_fallocate		deallocate;
 		struct nfsd4_clone		clone;
 		struct nfsd4_copy		copy;
+		struct nfsd4_copy_notify	copy_notify;
 		struct nfsd4_seek		seek;
 	} u;
 	struct nfs4_replay *			replay;
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#44 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

Quoted 59 lines · J. Bruce Fields
Quoted 58 lines · Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c |  98 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/xdr4.h     |  13 +++++++
 3 files changed, 221 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +36 -0
index dadb3bf..ed6b9f2 100644
@@ -35,6 +35,7 @@
 #include <linux/file.h>
 #include <linux/falloc.h>
 #include <linux/slab.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "cache.h"
@@ -1087,6 +1088,82 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 }
 
 static __be32
+nfsd4_set_src_nl4_netaddr(struct svc_rqst *rqstp, struct nl4_servers *svrs)
+{
+     const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr;
+     const struct sockaddr_in *sin = (const struct sockaddr_in *)addr;
+     const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr;
+     int uaddr_len = rqstp->rq_daddrlen + 4 + 1; /* port (4) and '\0' (1) */
+     struct nfs42_netaddr *naddr;
+     size_t ret;
+     unsigned short port;
+
+     /* freed in nfsd4_encode_copy_notify */
+     svrs->nl_svr = kmalloc_array(svrs->nl_nsvr, sizeof(struct nl4_server),
+                             GFP_KERNEL);
+     if (svrs->nl_svr == NULL)
+             return nfserrno(-ENOMEM);
+
+     svrs->nl_svr->nl4_type = NL4_NETADDR;
+     naddr = &svrs->nl_svr->u.nl4_addr;
+
+     switch (addr->sa_family) {
+     case AF_INET:
+             port = ntohs(sin->sin_port);
+             sprintf(naddr->na_netid, "tcp");
+             naddr->na_netid_len = 3;
+             break;
+     case AF_INET6:
+             port = ntohs(sin6->sin6_port);
+             sprintf(naddr->na_netid, "tcp6");
+             naddr->na_netid_len = 4;
+             break;
+     default:
+             dprintk("NFSD  nfsd4_set_notify_src: unknown address type: %d",
+                     addr->sa_family);
+             kfree(svrs->nl_svr);
+             return nfserrno(-EINVAL);
Quoted 8 lines

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1130

Quoted 5 lines
fs/nfsd/nfs4proc.c +5 -0
+     }
+     ret = rpc_ntop(addr, naddr->na_uaddr, sizeof(naddr->na_uaddr));
+     snprintf(naddr->na_uaddr + ret, uaddr_len, ".%u.%u",
+              port >> 8, port & 255);
+     naddr->na_uaddr_len = strlen(naddr->na_uaddr);

I'm a little surprised there isn't already a standard helper for this.... Actually, does net/sunrpc:addr.c:rpc_sockaddr2uaddr() do it?

That only creates the string which is just a part of the data structure. So I don't think it would be useful.

Quoted 3 lines

(Looks like there's other potential users in fs/nfsd/flexfilelayout.c:nfsd4_ff_proc_getdeviceinfo() and in flexfiles code on client side and probably some other places too.)

Both flex files and copy can be re-written to share code though. They are setting up similar .._netaddr structure.

Quoted 269 lines

--b.

Quoted 259 lines
fs/nfsd/nfs4proc.c +57 -0
+     return 0;
+}
+
+static __be32
+nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+               struct nfsd4_copy_notify *cn)
+{
+     __be32 status;
+     struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+
+     status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+                                     &cn->cpn_src_stateid, RD_STATE, NULL,
+                                     NULL);
+     if (status)
+             return status;
+
+     cn->cpn_sec = nn->nfsd4_lease;
+     cn->cpn_nsec = 0;
+
+
+     /** XXX Save cpn_src_statid, cpn_src, and any other returned source
+      * server addresses on which the source server is williing to accept
+      * connections from the destination e.g. what is returned in cpn_src,
+      * to verify READ from dest server.
+      */
+
+     /**
+      * For now, only return one server address in cpn_src, the
+      * address used by the client to connect to this server.
+      */
+     cn->cpn_src.nl_nsvr = 1;
+
+     return nfsd4_set_src_nl4_netaddr(rqstp, &cn->cpn_src);
+}
+
+static __be32
 nfsd4_fallocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
              struct nfsd4_fallocate *fallocate, int flags)
 {
@@ -2042,6 +2119,21 @@ static inline u32 nfsd4_copy_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
              1 /* cr_synchronous */) * sizeof(__be32);
 }
 
+static inline u32 nfsd4_copy_notify_rsize(struct svc_rqst *rqstp,
+                                     struct nfsd4_op *op)
+{
+     return (op_encode_hdr_size +
+             3 /* cnr_lease_time */ +
+             1 /* We support one cnr_source_server */ +
+             1 /* cnr_stateid seq */ +
+             op_encode_stateid_maxsz /* cnr_stateid */ +
+             1 /* num cnr_source_server*/ +
+             1 /* nl4_type */ +
+             1 /* nl4 size */ +
+             XDR_QUADLEN(NFS4_OPAQUE_LIMIT) /*nl4_loc + nl4_loc_sz */)
+             * sizeof(__be32);
+}
+
 #ifdef CONFIG_NFSD_PNFS
 static inline u32 nfsd4_getdeviceinfo_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 {
@@ -2446,6 +2538,12 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
              .op_name = "OP_SEEK",
              .op_rsize_bop = (nfsd4op_rsize)nfsd4_seek_rsize,
      },
+     [OP_COPY_NOTIFY] = {
+             .op_func = (nfsd4op_func)nfsd4_copy_notify,
+             .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME,
+             .op_name = "OP_COPY_NOTIFY",
+             .op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize,
+     },
 };
 
 /**
fs/nfsd/nfs4xdr.c +110 -2
index 031d06d..3e08c15 100644
@@ -1820,6 +1820,22 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 }
 
 static __be32
+nfsd4_decode_copy_notify(struct nfsd4_compoundargs *argp,
+                      struct nfsd4_copy_notify *cn)
+{
+     int status;
+
+     status = nfsd4_decode_stateid(argp, &cn->cpn_src_stateid);
+     if (status)
+             return status;
+     status = nfsd4_decode_nl4_server(argp, &cn->cpn_dst);
+     if (status)
+             return status;
+
+     return status;
+}
+
+static __be32
 nfsd4_decode_seek(struct nfsd4_compoundargs *argp, struct nfsd4_seek *seek)
 {
      DECODE_HEAD;
@@ -1920,7 +1936,7 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
      /* new operations for NFSv4.2 */
      [OP_ALLOCATE]           = (nfsd4_dec)nfsd4_decode_fallocate,
      [OP_COPY]               = (nfsd4_dec)nfsd4_decode_copy,
-     [OP_COPY_NOTIFY]        = (nfsd4_dec)nfsd4_decode_notsupp,
+     [OP_COPY_NOTIFY]        = (nfsd4_dec)nfsd4_decode_copy_notify,
      [OP_DEALLOCATE]         = (nfsd4_dec)nfsd4_decode_fallocate,
      [OP_IO_ADVISE]          = (nfsd4_dec)nfsd4_decode_notsupp,
      [OP_LAYOUTERROR]        = (nfsd4_dec)nfsd4_decode_notsupp,
@@ -4355,6 +4371,52 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd42_encode_nl4_server(struct nfsd4_compoundres *resp, struct nl4_server *ns)
+{
+     struct xdr_stream *xdr = &resp->xdr;
+     struct nfs42_netaddr *addr;
+     __be32 *p;
+
+     p = xdr_reserve_space(xdr, 4);
+     *p++ = cpu_to_be32(ns->nl4_type);
+
+     switch (ns->nl4_type) {
+     case NL4_NAME:
+     case NL4_URL:
+             p = xdr_reserve_space(xdr, 4 /* url or name len */ +
+                                   (XDR_QUADLEN(ns->u.nl4_str_sz) * 4));
+             if (!p)
+                     return nfserr_resource;
+             *p++ = cpu_to_be32(ns->u.nl4_str_sz);
+             p = xdr_encode_opaque_fixed(p, ns->u.nl4_str, ns->u.nl4_str_sz);
+                     break;
+     case NL4_NETADDR:
+             addr = &ns->u.nl4_addr;
+
+             /** netid_len, netid, uaddr_len, uaddr (port included
+              * in RPCBIND_MAXUADDRLEN)
+              */
+             p = xdr_reserve_space(xdr,
+                     4 /* netid len */ +
+                     (XDR_QUADLEN(addr->na_netid_len) * 4) +
+                     4 /* uaddr len */ +
+                     (XDR_QUADLEN(addr->na_uaddr_len) * 4));
+             if (!p)
+                     return nfserr_resource;
+
+             *p++ = cpu_to_be32(addr->na_netid_len);
+             p = xdr_encode_opaque_fixed(p, addr->na_netid,
+                                         addr->na_netid_len);
+             *p++ = cpu_to_be32(addr->na_uaddr_len);
+             p = xdr_encode_opaque_fixed(p, addr->na_uaddr,
+                                     addr->na_uaddr_len);
+             break;
+     }
+
+     return 0;
+}
+
+static __be32
 nfsd4_encode_copy(struct nfsd4_compoundres *resp, __be32 nfserr,
                struct nfsd4_copy *copy)
 {
@@ -4375,6 +4437,52 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd4_encode_copy_notify(struct nfsd4_compoundres *resp, __be32 nfserr,
+                      struct nfsd4_copy_notify *cn)
+{
+     struct xdr_stream *xdr = &resp->xdr;
+     struct nl4_server *ns;
+     __be32 *p;
+     int i;
+
+     if (nfserr)
+             return nfserr;
+
+     /* 8 sec, 4 nsec */
+     p = xdr_reserve_space(xdr, 12);
+     if (!p)
+             return nfserr_resource;
+
+     /* cnr_lease_time */
+     p = xdr_encode_hyper(p, cn->cpn_sec);
+     *p++ = cpu_to_be32(cn->cpn_nsec);
+
+     /* cnr_stateid */
+     nfserr = nfsd4_encode_stateid(xdr, &cn->cpn_src_stateid);
+     if (nfserr)
+             return nfserr;
+
+     /* cnr_src.nl_nsvr */
+     p = xdr_reserve_space(xdr, 4);
+     if (!p)
+             return nfserr_resource;
+
+     *p++ = cpu_to_be32(cn->cpn_src.nl_nsvr);
+
+     ns = cn->cpn_src.nl_svr;
+     for (i = 0; i < cn->cpn_src.nl_nsvr; i++) {
+             nfserr = nfsd42_encode_nl4_server(resp, ns);
+             if (nfserr)
+                     return nfserr;
+             ns++;
+     }
+
+     /* allocated in nfsd4_copy_notify */
+     kfree(cn->cpn_src.nl_svr);
+     return nfserr;
+}
+
+static __be32
 nfsd4_encode_seek(struct nfsd4_compoundres *resp, __be32 nfserr,
                struct nfsd4_seek *seek)
 {
@@ -4474,7 +4582,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
      /* NFSv4.2 operations */
      [OP_ALLOCATE]           = (nfsd4_enc)nfsd4_encode_noop,
      [OP_COPY]               = (nfsd4_enc)nfsd4_encode_copy,
-     [OP_COPY_NOTIFY]        = (nfsd4_enc)nfsd4_encode_noop,
+     [OP_COPY_NOTIFY]        = (nfsd4_enc)nfsd4_encode_copy_notify,
      [OP_DEALLOCATE]         = (nfsd4_enc)nfsd4_encode_noop,
      [OP_IO_ADVISE]          = (nfsd4_enc)nfsd4_encode_noop,
      [OP_LAYOUTERROR]        = (nfsd4_enc)nfsd4_encode_noop,
fs/nfsd/xdr4.h +13 -1
index 6b1a61fc..5db7cd8 100644
@@ -540,6 +540,18 @@ struct nfsd4_seek {
      loff_t          seek_pos;
 };
 
+struct nfsd4_copy_notify {
+     /* request */
+     stateid_t               cpn_src_stateid;
+     struct nl4_server       cpn_dst;
+
+     /* response */
+     /* Note: cpn_src_stateid is used for cnr_stateid */
+     u64                     cpn_sec;
+     u32                     cpn_nsec;
+     struct nl4_servers      cpn_src;
+};
+
 struct nfsd4_op {
      int                                     opnum;
      __be32                                  status;
@@ -595,6 +607,7 @@ struct nfsd4_op {
              struct nfsd4_fallocate          deallocate;
              struct nfsd4_clone              clone;
              struct nfsd4_copy               copy;
+             struct nfsd4_copy_notify        copy_notify;
              struct nfsd4_seek               seek;
      } u;
      struct nfs4_replay *                    replay;
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#45 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

In reply to Olga Kornievskaia
Quoted 78 lines · Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c |  98 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/xdr4.h     |  13 +++++++
 3 files changed, 221 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +56 -0
index dadb3bf..ed6b9f2 100644
@@ -35,6 +35,7 @@
 #include <linux/file.h>
 #include <linux/falloc.h>
 #include <linux/slab.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "cache.h"
@@ -1087,6 +1088,82 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 }
 
 static __be32
+nfsd4_set_src_nl4_netaddr(struct svc_rqst *rqstp, struct nl4_servers *svrs)
+{
+	const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr;
+	const struct sockaddr_in *sin = (const struct sockaddr_in *)addr;
+	const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr;
+	int uaddr_len = rqstp->rq_daddrlen + 4 + 1; /* port (4) and '\0' (1) */
+	struct nfs42_netaddr *naddr;
+	size_t ret;
+	unsigned short port;
+
+	/* freed in nfsd4_encode_copy_notify */
+	svrs->nl_svr = kmalloc_array(svrs->nl_nsvr, sizeof(struct nl4_server),
+				GFP_KERNEL);
+	if (svrs->nl_svr == NULL)
+		return nfserrno(-ENOMEM);
+
+	svrs->nl_svr->nl4_type = NL4_NETADDR;
+	naddr = &svrs->nl_svr->u.nl4_addr;
+
+	switch (addr->sa_family) {
+	case AF_INET:
+		port = ntohs(sin->sin_port);
+		sprintf(naddr->na_netid, "tcp");
+		naddr->na_netid_len = 3;
+		break;
+	case AF_INET6:
+		port = ntohs(sin6->sin6_port);
+		sprintf(naddr->na_netid, "tcp6");
+		naddr->na_netid_len = 4;
+		break;
+	default:
+		dprintk("NFSD  nfsd4_set_notify_src: unknown address type: %d",
+			addr->sa_family);
+		kfree(svrs->nl_svr);
+		return nfserrno(-EINVAL);
+	}
+	ret = rpc_ntop(addr, naddr->na_uaddr, sizeof(naddr->na_uaddr));
+	snprintf(naddr->na_uaddr + ret, uaddr_len, ".%u.%u",
+		 port >> 8, port & 255);
+	naddr->na_uaddr_len = strlen(naddr->na_uaddr);
+	return 0;
+}
+
+static __be32
+nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+		  struct nfsd4_copy_notify *cn)
+{
+	__be32 status;
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+
+	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+					&cn->cpn_src_stateid, RD_STATE, NULL,
+					NULL);
+	if (status)
+		return status;

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1155

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+
+	cn->cpn_sec = nn->nfsd4_lease;
+	cn->cpn_nsec = 0;
+
+
+	/** XXX Save cpn_src_statid, cpn_src, and any other returned source
+	 * server addresses on which the source server is williing to accept
+	 * connections from the destination e.g. what is returned in cpn_src,
+	 * to verify READ from dest server.
+	 */

Either we should fix the stuff mentioned in the comment or figure out why it's not necessary.

I guess I don't see what the value would be of verifying the IP address. Our choice of IP address to return here is a little arbitrary anyway. And I assume the eventualy READ calls are going to be verified with all the usual export and file permission checks.

If we don't save the cna_src_stateid, then I guess that means the COPY might still succeed after the original OPEN/LOCK stateid is no longer valid? I can't decide if that will cause problems in practice or not.

Quoted 10 lines
fs/nfsd/nfs4proc.c +9 -0
 static __be32
+nfsd42_encode_nl4_server(struct nfsd4_compoundres *resp, struct nl4_server *ns)
+{
+	struct xdr_stream *xdr = &resp->xdr;
+	struct nfs42_netaddr *addr;
+	__be32 *p;
+
+	p = xdr_reserve_space(xdr, 4);
+	*p++ = cpu_to_be32(ns->nl4_type);
+

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	switch (ns->nl4_type) {
+	case NL4_NAME:
+	case NL4_URL:
+		p = xdr_reserve_space(xdr, 4 /* url or name len */ +
+				      (XDR_QUADLEN(ns->u.nl4_str_sz) * 4));
+		if (!p)
+			return nfserr_resource;
+		*p++ = cpu_to_be32(ns->u.nl4_str_sz);
+		p = xdr_encode_opaque_fixed(p, ns->u.nl4_str, ns->u.nl4_str_sz);
+			break;

This code will never be used as far as I can tell. We could BUG_ON(nl4_type != NL4_NETADDR) (or even just remove nl4_type).

--b.

#46 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

Quoted 84 lines · J. Bruce Fields
Quoted 83 lines · Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c |  98 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/xdr4.h     |  13 +++++++
 3 files changed, 221 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +61 -0
index dadb3bf..ed6b9f2 100644
@@ -35,6 +35,7 @@
 #include <linux/file.h>
 #include <linux/falloc.h>
 #include <linux/slab.h>
+#include <linux/sunrpc/addr.h>
 
 #include "idmap.h"
 #include "cache.h"
@@ -1087,6 +1088,82 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 }
 
 static __be32
+nfsd4_set_src_nl4_netaddr(struct svc_rqst *rqstp, struct nl4_servers *svrs)
+{
+     const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr;
+     const struct sockaddr_in *sin = (const struct sockaddr_in *)addr;
+     const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr;
+     int uaddr_len = rqstp->rq_daddrlen + 4 + 1; /* port (4) and '\0' (1) */
+     struct nfs42_netaddr *naddr;
+     size_t ret;
+     unsigned short port;
+
+     /* freed in nfsd4_encode_copy_notify */
+     svrs->nl_svr = kmalloc_array(svrs->nl_nsvr, sizeof(struct nl4_server),
+                             GFP_KERNEL);
+     if (svrs->nl_svr == NULL)
+             return nfserrno(-ENOMEM);
+
+     svrs->nl_svr->nl4_type = NL4_NETADDR;
+     naddr = &svrs->nl_svr->u.nl4_addr;
+
+     switch (addr->sa_family) {
+     case AF_INET:
+             port = ntohs(sin->sin_port);
+             sprintf(naddr->na_netid, "tcp");
+             naddr->na_netid_len = 3;
+             break;
+     case AF_INET6:
+             port = ntohs(sin6->sin6_port);
+             sprintf(naddr->na_netid, "tcp6");
+             naddr->na_netid_len = 4;
+             break;
+     default:
+             dprintk("NFSD  nfsd4_set_notify_src: unknown address type: %d",
+                     addr->sa_family);
+             kfree(svrs->nl_svr);
+             return nfserrno(-EINVAL);
+     }
+     ret = rpc_ntop(addr, naddr->na_uaddr, sizeof(naddr->na_uaddr));
+     snprintf(naddr->na_uaddr + ret, uaddr_len, ".%u.%u",
+              port >> 8, port & 255);
+     naddr->na_uaddr_len = strlen(naddr->na_uaddr);
+     return 0;
+}
+
+static __be32
+nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+               struct nfsd4_copy_notify *cn)
+{
+     __be32 status;
+     struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+
+     status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+                                     &cn->cpn_src_stateid, RD_STATE, NULL,
+                                     NULL);
+     if (status)
+             return status;
+
+     cn->cpn_sec = nn->nfsd4_lease;
+     cn->cpn_nsec = 0;
+
+
Quoted 8 lines

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1155

Quoted 5 lines
fs/nfsd/nfs4proc.c +5 -0
+     /** XXX Save cpn_src_statid, cpn_src, and any other returned source
+      * server addresses on which the source server is williing to accept
+      * connections from the destination e.g. what is returned in cpn_src,
+      * to verify READ from dest server.
+      */

Either we should fix the stuff mentioned in the comment or figure out why it's not necessary.

Sorry yes this comment should be removed from this patch. Another patch (29) removes it.

Quoted 1 line

I guess I don't see what the value would be of verifying the IP address.

Correct no value is there. Also the READ request can come from a different IP then what the client sends in COPY_NOTIFY. Destination server can be multi-homed and share a high speed link with the source server and not with the client.

Quoted 1 line

Our choice of IP address to return here is a little arbitrary anyway.

What the source server is returning shouldn't be arbitrary it should return all the IPs (networks) that it might want to receive requests from the destination
server.

Quoted 6 lines

And I assume the eventualy READ calls are going to be verified with all the usual export and file permission checks.

If we don't save the cna_src_stateid, then I guess that means the COPY might still succeed after the original OPEN/LOCK stateid is no longer valid? I can't decide if that will cause problems in practice or not.

No we store the stateid (it's a part of the nfsd4_copy_notify structure and gets saved from nfs4_preprocess_stateid_op. Then in patch 29 we generate the unique stateid to be used for the copy itself and its validity is tied to the
open/lock stateid and also it has to start to be used within the configured lease time (nfsd_net->nfsd4_lease value).

Quoted 16 lines
Quoted 16 lines
fs/nfsd/nfs4proc.c +14 -0
 static __be32
+nfsd42_encode_nl4_server(struct nfsd4_compoundres *resp, struct nl4_server *ns)
+{
+     struct xdr_stream *xdr = &resp->xdr;
+     struct nfs42_netaddr *addr;
+     __be32 *p;
+
+     p = xdr_reserve_space(xdr, 4);
+     *p++ = cpu_to_be32(ns->nl4_type);
+
+     switch (ns->nl4_type) {
+     case NL4_NAME:
+     case NL4_URL:
+             p = xdr_reserve_space(xdr, 4 /* url or name len */ +
+                                   (XDR_QUADLEN(ns->u.nl4_str_sz) * 4));
Quoted 8 lines

fs/nfsd/nfs4proc.c

Quoted 5 lines
fs/nfsd/nfs4proc.c +5 -0
+             if (!p)
+                     return nfserr_resource;
+             *p++ = cpu_to_be32(ns->u.nl4_str_sz);
+             p = xdr_encode_opaque_fixed(p, ns->u.nl4_str, ns->u.nl4_str_sz);
+                     break;

This code will never be used as far as I can tell. We could BUG_ON(nl4_type != NL4_NETADDR) (or even just remove nl4_type).

Sure.

Quoted 6 lines

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#47 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

In reply to Olga Kornievskaia

fs/nfsd/xdr4.h struct nfsd4_seek { · line 549

Quoted 7 lines · Olga Kornievskaia
fs/nfsd/xdr4.h +7 -0
+struct nfsd4_copy_notify {
+	/* request */
+	stateid_t		cpn_src_stateid;
+	struct nl4_server	cpn_dst;
+
+	/* response */
+	/* Note: cpn_src_stateid is used for cnr_stateid */

That might be worth a little more explanation.

--b.

Quoted 5 lines
fs/nfsd/xdr4.h +5 -0
+	u64			cpn_sec;
+	u32			cpn_nsec;
+	struct nl4_servers	cpn_src;
+};
+
#48 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

Quoted 10 lines · J. Bruce Fields

fs/nfsd/xdr4.h struct nfsd4_seek { · line 549

Quoted 7 lines · Olga Kornievskaia
fs/nfsd/xdr4.h +7 -0
+struct nfsd4_copy_notify {
+     /* request */
+     stateid_t               cpn_src_stateid;
+     struct nl4_server       cpn_dst;
+
+     /* response */
+     /* Note: cpn_src_stateid is used for cnr_stateid */

That might be worth a little more explanation.

I'm opting for removing it from this patch? It's just valid for that patch saying that for the
moment the same cpn_src_stateid is returned fort he cnr_stateid in the later patch (29)
we will add the generation of the unique stateid used for the copy.

Quoted 12 lines

--b.

fs/nfsd/xdr4.h struct nfsd4_seek { · line 554

Quoted 5 lines
fs/nfsd/xdr4.h +5 -0
+     u64                     cpn_sec;
+     u32                     cpn_nsec;
+     struct nl4_servers      cpn_src;
+};
+

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#49 lore raw

Re: [RFC v3 25/42] NFSD add COPY_NOTIFY operation

Quoted 12 lines · Olga Kornievskaia
Quoted 10 lines · J. Bruce Fields

fs/nfsd/xdr4.h struct nfsd4_seek { · line 549

Quoted 7 lines · Olga Kornievskaia
fs/nfsd/xdr4.h +7 -0
+struct nfsd4_copy_notify {
+     /* request */
+     stateid_t               cpn_src_stateid;
+     struct nl4_server       cpn_dst;
+
+     /* response */
+     /* Note: cpn_src_stateid is used for cnr_stateid */

That might be worth a little more explanation.

Quoted 5 lines

I'm opting for removing it from this patch? It's just valid for that patch saying that for the
moment the same cpn_src_stateid is returned fort he cnr_stateid in the later patch (29)
we will add the generation of the unique stateid used for the copy.

Got it, yes, that did confuse me, dropping the comment and noting the coming change in the changelog would be great.--b.

Quoted 13 lines
Quoted 13 lines

--b.

fs/nfsd/xdr4.h struct nfsd4_seek { · line 554

Quoted 5 lines
fs/nfsd/xdr4.h +5 -0
+     u64                     cpn_sec;
+     u32                     cpn_nsec;
+     struct nl4_servers      cpn_src;
+};
+

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#50 lore raw

RFC v3 26/42 NFSD generalize nfsd4_compound_state flag names

In reply to Olga Kornievskaia

From: Andy Adamson <[email protected]>

Allow for sid_flag field non-stateid use.

---
 fs/nfsd/nfs4proc.c  | 8 ++++----
 fs/nfsd/nfs4state.c | 7 ++++---
 fs/nfsd/xdr4.h      | 6 +++---
 3 files changed, 11 insertions(+), 10 deletions(-)
fs/nfsd/nfs4proc.c +4 -4
index ed6b9f2..b49ff31 100644
@@ -522,9 +522,9 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 		return nfserr_restorefh;
 
 	fh_dup2(&cstate->current_fh, &cstate->save_fh);
-	if (HAS_STATE_ID(cstate, SAVED_STATE_ID_FLAG)) {
+	if (HAS_CSTATE_FLAG(cstate, SAVED_STATE_ID_FLAG)) {
 		memcpy(&cstate->current_stateid, &cstate->save_stateid, sizeof(stateid_t));
-		SET_STATE_ID(cstate, CURRENT_STATE_ID_FLAG);
+		SET_CSTATE_FLAG(cstate, CURRENT_STATE_ID_FLAG);
 	}
 	return nfs_ok;
 }
@@ -537,9 +537,9 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 		return nfserr_nofilehandle;
 
 	fh_dup2(&cstate->save_fh, &cstate->current_fh);
-	if (HAS_STATE_ID(cstate, CURRENT_STATE_ID_FLAG)) {
+	if (HAS_CSTATE_FLAG(cstate, CURRENT_STATE_ID_FLAG)) {
 		memcpy(&cstate->save_stateid, &cstate->current_stateid, sizeof(stateid_t));
-		SET_STATE_ID(cstate, SAVED_STATE_ID_FLAG);
+		SET_CSTATE_FLAG(cstate, SAVED_STATE_ID_FLAG);
 	}
 	return nfs_ok;
 }
fs/nfsd/nfs4state.c +4 -3
index 22002fb..e7e1b9f 100644
@@ -7080,7 +7080,8 @@ static int nfs4_state_create_net(struct net *net)
 static void
 get_stateid(struct nfsd4_compound_state *cstate, stateid_t *stateid)
 {
-	if (HAS_STATE_ID(cstate, CURRENT_STATE_ID_FLAG) && CURRENT_STATEID(stateid))
+	if (HAS_CSTATE_FLAG(cstate, CURRENT_STATE_ID_FLAG) &&
+	    CURRENT_STATEID(stateid))
 		memcpy(stateid, &cstate->current_stateid, sizeof(stateid_t));
 }
 
@@ -7089,14 +7090,14 @@ static int nfs4_state_create_net(struct net *net)
 {
 	if (cstate->minorversion) {
 		memcpy(&cstate->current_stateid, stateid, sizeof(stateid_t));
-		SET_STATE_ID(cstate, CURRENT_STATE_ID_FLAG);
+		SET_CSTATE_FLAG(cstate, CURRENT_STATE_ID_FLAG);
 	}
 }
 
 void
 clear_current_stateid(struct nfsd4_compound_state *cstate)
 {
-	CLEAR_STATE_ID(cstate, CURRENT_STATE_ID_FLAG);
+	CLEAR_CSTATE_FLAG(cstate, CURRENT_STATE_ID_FLAG);
 }
 
 /*
fs/nfsd/xdr4.h +3 -3
index 5db7cd8..38fcb4f 100644
@@ -46,9 +46,9 @@
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
 
-#define SET_STATE_ID(c, f) ((c)->sid_flags |= (f))
-#define HAS_STATE_ID(c, f) ((c)->sid_flags & (f))
-#define CLEAR_STATE_ID(c, f) ((c)->sid_flags &= ~(f))
+#define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
+#define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
+#define CLEAR_CSTATE_FLAG(c, f) ((c)->sid_flags &= ~(f))
 
 struct nfsd4_compound_state {
 	struct svc_fh		current_fh;
Signature · 1 line
1.8.3.1
#51 lore raw

RFC v3 27/42 NFSD: allow inter server COPY to have a STALE source server fh

In reply to Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +46 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	    struct nfsd4_putfh *putfh)
 {
+	__be32 ret;
+
 	fh_put(&cstate->current_fh);
 	cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
 	memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
 	       putfh->pf_fhlen);
-	return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+	ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+	if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+		CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+		SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+		ret = 0;
+	}
+	return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     void *arg)
 {
+	/**
+	* This is either an inter COPY (most likely) or an intra COPY with a
+	* stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+	* return nfserr_stale. No fh_dentry, just copy the file handle
+	* to use with the inter COPY READ.
+	*/
+	if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+		cstate->save_fh = cstate->current_fh;
+		return nfs_ok;
+	}
 	if (!cstate->current_fh.fh_dentry)
 		return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	if (status)
 		goto out;
 
+	/* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+	if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+		CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+		cstate->status = nfserr_copy_stalefh;
+		goto out_put;
+	}
+
 	bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
 			dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		status = nfs_ok;
 	}
 
+out_put:
 	fput(src);
 	fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 	struct nfsd4_compound_state *cstate = &resp->cstate;
 	struct svc_fh *current_fh = &cstate->current_fh;
 	struct svc_fh *save_fh = &cstate->save_fh;
+	int		i;
 	__be32		status;
 
 	svcxdr_init_encode(rqstp, resp);
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 		goto encode_op;
 	}
 
+	/* NFSv4.2 COPY source file handle may be from a different server */
+	for (i = 0; i < args->opcnt; i++) {
+		op = &args->ops[i];
+		if (op->opnum == OP_COPY)
+			SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+	}
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
 		opdesc = OPDESC(op);
 
+		if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+			goto call_op;
+
 		if (!current_fh->fh_dentry) {
 			if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
 				op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
 		if (opdesc->op_get_currentstateid)
 			opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
 		op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
 		/* Only from SEQUENCE */
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 			if (need_wrongsec_check(rqstp))
 				op->status = check_nfsd_access(current_fh->fh_export, rqstp);
 		}
+		/* Only from intra COPY */
+		if (cstate->status == nfserr_copy_stalefh) {
+			dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+				__func__);
+			status = nfserr_stale;
+			nfsd4_adjust_encode(resp);
+			goto out;
+		}
 encode_op:
 		if (op->status == nfserr_replay_me) {
 			op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
 	return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+	__be32 *p;
+
+	p = resp->cstate.putfh_errp;
+	*p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
 	struct xdr_stream *xdr = &resp->xdr;
 	struct nfs4_stateowner *so = resp->cstate.replay_owner;
+	struct nfsd4_compound_state *cstate = &resp->cstate;
 	struct svc_rqst *rqstp = resp->rqstp;
 	int post_err_offset;
 	nfsd4_enc encoder;
-	__be32 *p;
+	__be32 *p, *statp;
 
 	p = xdr_reserve_space(xdr, 8);
 	if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
 	}
 	*p++ = cpu_to_be32(op->opnum);
 	post_err_offset = xdr->buf->len;
+	statp = p;
 
 	if (op->opnum == OP_ILLEGAL)
 		goto status;
+
+	/** This is a COPY compound with a stale source server file handle.
+	 * If OP_COPY processing determines that this is an intra server to
+	 * server COPY, then this PUTFH should return nfserr_ stale so the
+	 * putfh_errp will be set to nfserr_stale. If this is an inter server
+	 * to server COPY, ignore the nfserr_stale.
+	 */
+	if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+		cstate->putfh_errp = statp;
+
 	BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
 	       !nfsd4_enc_ops[op->opnum]);
 	encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define	nfserr_replay_me	cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define	nfserr_replay_cache	cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh	cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)	(l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -0
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
 	size_t			iovlen;
 	u32			minorversion;
 	__be32			status;
+	__be32			*putfh_errp;
 	stateid_t	current_stateid;
 	stateid_t	save_stateid;
 	/* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
 		struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
Signature · 1 line
1.8.3.1
#52 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Some questions below, but really I think this approach can't work. Keep in mind this has to work not just for the one compound the current client uses, but for *any* compound containing a COPY, including those sent by future clients that may do something more complicated, or by malcious clients that may intentionally send weird compounds to make the server crash.

I think we want to flag the filehandle itself somehow, not the cstate.

And check very carefully to make sure that ops other than COPY can't get tripped up by one of these foreign filehandles.

Quoted 85 lines · Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +28 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	    struct nfsd4_putfh *putfh)
 {
+	__be32 ret;
+
 	fh_put(&cstate->current_fh);
 	cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
 	memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
 	       putfh->pf_fhlen);
-	return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+	ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+	if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+		CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+		SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+		ret = 0;
+	}
+	return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     void *arg)
 {
+	/**
+	* This is either an inter COPY (most likely) or an intra COPY with a
+	* stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+	* return nfserr_stale. No fh_dentry, just copy the file handle
+	* to use with the inter COPY READ.
+	*/
+	if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+		cstate->save_fh = cstate->current_fh;
+		return nfs_ok;
+	}
 	if (!cstate->current_fh.fh_dentry)
 		return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	if (status)
 		goto out;
 
+	/* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+	if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+		CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+		cstate->status = nfserr_copy_stalefh;
+		goto out_put;
+	}
+
 	bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
 			dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		status = nfs_ok;
 	}
 
+out_put:
 	fput(src);
 	fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 	struct nfsd4_compound_state *cstate = &resp->cstate;
 	struct svc_fh *current_fh = &cstate->current_fh;
 	struct svc_fh *save_fh = &cstate->save_fh;
+	int		i;
 	__be32		status;
 
 	svcxdr_init_encode(rqstp, resp);

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1826

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 		goto encode_op;
 	}
 
+	/* NFSv4.2 COPY source file handle may be from a different server */
+	for (i = 0; i < args->opcnt; i++) {
+		op = &args->ops[i];
+		if (op->opnum == OP_COPY)
+			SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+	}

So, if a compound has a COPY anywhere in it, then a PUTFH with a stale filehandle anywhere in the compound will temporarily succeed and result in IS_STALE_FH being set.

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1848

Quoted 10 lines
fs/nfsd/nfs4proc.c +3 -0
 	while (!status && resp->opcnt < args->opcnt) {
 		op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
 		opdesc = OPDESC(op);
 
+		if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+			goto call_op;
+

That means that once IS_STALE_FH is set, we will skip:

- nofilehandle checking
- moved checking
- fh_clear_wcc()
- resp_size checking
- current stateid capture

on every subsequent op. Can that really be right?

Quoted 12 lines
fs/nfsd/nfs4proc.c +1 -0
 		if (!current_fh->fh_dentry) {
 			if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
 				op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
 		if (opdesc->op_get_currentstateid)
 			opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
 		op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
 		/* Only from SEQUENCE */

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 			if (need_wrongsec_check(rqstp))
 				op->status = check_nfsd_access(current_fh->fh_export, rqstp);
 		}
+		/* Only from intra COPY */
+		if (cstate->status == nfserr_copy_stalefh) {
+			dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+				__func__);
+			status = nfserr_stale;
+			nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 109 lines
fs/nfsd/nfs4proc.c +2 -0
+			goto out;
+		}
 encode_op:
 		if (op->status == nfserr_replay_me) {
 			op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
 	return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+	__be32 *p;
+
+	p = resp->cstate.putfh_errp;
+	*p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
 	struct xdr_stream *xdr = &resp->xdr;
 	struct nfs4_stateowner *so = resp->cstate.replay_owner;
+	struct nfsd4_compound_state *cstate = &resp->cstate;
 	struct svc_rqst *rqstp = resp->rqstp;
 	int post_err_offset;
 	nfsd4_enc encoder;
-	__be32 *p;
+	__be32 *p, *statp;
 
 	p = xdr_reserve_space(xdr, 8);
 	if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
 	}
 	*p++ = cpu_to_be32(op->opnum);
 	post_err_offset = xdr->buf->len;
+	statp = p;
 
 	if (op->opnum == OP_ILLEGAL)
 		goto status;
+
+	/** This is a COPY compound with a stale source server file handle.
+	 * If OP_COPY processing determines that this is an intra server to
+	 * server COPY, then this PUTFH should return nfserr_ stale so the
+	 * putfh_errp will be set to nfserr_stale. If this is an inter server
+	 * to server COPY, ignore the nfserr_stale.
+	 */
+	if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+		cstate->putfh_errp = statp;
+
 	BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
 	       !nfsd4_enc_ops[op->opnum]);
 	encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define	nfserr_replay_me	cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define	nfserr_replay_cache	cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh	cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)	(l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -0
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
 	size_t			iovlen;
 	u32			minorversion;
 	__be32			status;
+	__be32			*putfh_errp;
 	stateid_t	current_stateid;
 	stateid_t	save_stateid;
 	/* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
 		struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#53 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

What if instead the client won't send the 2nd FH in case of the inter server to server COPY and it would avoid all this stale handle business. So for the "intra" COPY it'll still be PUTFH, SAVEFH, PUTFH, COPY but for the "inter" COPY it'll be PUTFH, COPY.

Quoted 271 lines · J. Bruce Fields

Some questions below, but really I think this approach can't work. Keep in mind this has to work not just for the one compound the current client uses, but for *any* compound containing a COPY, including those sent by future clients that may do something more complicated, or by malcious clients that may intentionally send weird compounds to make the server crash.

I think we want to flag the filehandle itself somehow, not the cstate.

And check very carefully to make sure that ops other than COPY can't get tripped up by one of these foreign filehandles.

Quoted 85 lines · Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +28 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
          struct nfsd4_putfh *putfh)
 {
+     __be32 ret;
+
      fh_put(&cstate->current_fh);
      cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
      memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
             putfh->pf_fhlen);
-     return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+             SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+             ret = 0;
+     }
+     return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
           void *arg)
 {
+     /**
+     * This is either an inter COPY (most likely) or an intra COPY with a
+     * stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+     * return nfserr_stale. No fh_dentry, just copy the file handle
+     * to use with the inter COPY READ.
+     */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             cstate->save_fh = cstate->current_fh;
+             return nfs_ok;
+     }
      if (!cstate->current_fh.fh_dentry)
              return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
      if (status)
              goto out;
 
+     /* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+             cstate->status = nfserr_copy_stalefh;
+             goto out_put;
+     }
+
      bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
                      dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
              status = nfs_ok;
      }
 
+out_put:
      fput(src);
      fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
      struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_fh *current_fh = &cstate->current_fh;
      struct svc_fh *save_fh = &cstate->save_fh;
+     int             i;
      __be32          status;
 
      svcxdr_init_encode(rqstp, resp);

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1826

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
              goto encode_op;
      }
 
+     /* NFSv4.2 COPY source file handle may be from a different server */
+     for (i = 0; i < args->opcnt; i++) {
+             op = &args->ops[i];
+             if (op->opnum == OP_COPY)
+                     SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+     }

So, if a compound has a COPY anywhere in it, then a PUTFH with a stale filehandle anywhere in the compound will temporarily succeed and result in IS_STALE_FH being set.

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1848

Quoted 10 lines
fs/nfsd/nfs4proc.c +3 -0
      while (!status && resp->opcnt < args->opcnt) {
              op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              opdesc = OPDESC(op);
 
+             if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+                     goto call_op;
+

That means that once IS_STALE_FH is set, we will skip:

- nofilehandle checking
- moved checking
- fh_clear_wcc()
- resp_size checking
- current stateid capture

on every subsequent op. Can that really be right?

Quoted 12 lines
fs/nfsd/nfs4proc.c +1 -0
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
              /* Only from SEQUENCE */

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 105 lines
fs/nfsd/nfs4proc.c +2 -0
+                     goto out;
+             }
 encode_op:
              if (op->status == nfserr_replay_me) {
                      op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+     __be32 *p;
+
+     p = resp->cstate.putfh_errp;
+     *p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
      struct xdr_stream *xdr = &resp->xdr;
      struct nfs4_stateowner *so = resp->cstate.replay_owner;
+     struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_rqst *rqstp = resp->rqstp;
      int post_err_offset;
      nfsd4_enc encoder;
-     __be32 *p;
+     __be32 *p, *statp;
 
      p = xdr_reserve_space(xdr, 8);
      if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      }
      *p++ = cpu_to_be32(op->opnum);
      post_err_offset = xdr->buf->len;
+     statp = p;
 
      if (op->opnum == OP_ILLEGAL)
              goto status;
+
+     /** This is a COPY compound with a stale source server file handle.
+      * If OP_COPY processing determines that this is an intra server to
+      * server COPY, then this PUTFH should return nfserr_ stale so the
+      * putfh_errp will be set to nfserr_stale. If this is an inter server
+      * to server COPY, ignore the nfserr_stale.
+      */
+     if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+             cstate->putfh_errp = statp;
+
      BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
             !nfsd4_enc_ops[op->opnum]);
      encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define      nfserr_replay_me        cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define      nfserr_replay_cache     cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh  cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)       (l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -1
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
      size_t                  iovlen;
      u32                     minorversion;
      __be32                  status;
+     __be32                  *putfh_errp;
      stateid_t       current_stateid;
      stateid_t       save_stateid;
      /* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
              struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#54 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 4 lines · Olga Kornievskaia

What if instead the client won't send the 2nd FH in case of the inter server to server COPY and it would avoid all this stale handle business. So for the "intra" COPY it'll still be PUTFH, SAVEFH, PUTFH, COPY but for the "inter" COPY it'll be PUTFH, COPY.

That might be easier to implement, but it'd require a change to the protocol, right? I may not understand what you're proposing.

--b.

Quoted 273 lines · J. Bruce Fields
Quoted 271 lines

Some questions below, but really I think this approach can't work. Keep in mind this has to work not just for the one compound the current client uses, but for *any* compound containing a COPY, including those sent by future clients that may do something more complicated, or by malcious clients that may intentionally send weird compounds to make the server crash.

I think we want to flag the filehandle itself somehow, not the cstate.

And check very carefully to make sure that ops other than COPY can't get tripped up by one of these foreign filehandles.

Quoted 85 lines · Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +28 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
          struct nfsd4_putfh *putfh)
 {
+     __be32 ret;
+
      fh_put(&cstate->current_fh);
      cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
      memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
             putfh->pf_fhlen);
-     return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+             SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+             ret = 0;
+     }
+     return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
           void *arg)
 {
+     /**
+     * This is either an inter COPY (most likely) or an intra COPY with a
+     * stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+     * return nfserr_stale. No fh_dentry, just copy the file handle
+     * to use with the inter COPY READ.
+     */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             cstate->save_fh = cstate->current_fh;
+             return nfs_ok;
+     }
      if (!cstate->current_fh.fh_dentry)
              return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
      if (status)
              goto out;
 
+     /* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+             cstate->status = nfserr_copy_stalefh;
+             goto out_put;
+     }
+
      bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
                      dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
              status = nfs_ok;
      }
 
+out_put:
      fput(src);
      fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
      struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_fh *current_fh = &cstate->current_fh;
      struct svc_fh *save_fh = &cstate->save_fh;
+     int             i;
      __be32          status;
 
      svcxdr_init_encode(rqstp, resp);

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1826

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
              goto encode_op;
      }
 
+     /* NFSv4.2 COPY source file handle may be from a different server */
+     for (i = 0; i < args->opcnt; i++) {
+             op = &args->ops[i];
+             if (op->opnum == OP_COPY)
+                     SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+     }

So, if a compound has a COPY anywhere in it, then a PUTFH with a stale filehandle anywhere in the compound will temporarily succeed and result in IS_STALE_FH being set.

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1848

Quoted 10 lines
fs/nfsd/nfs4proc.c +3 -0
      while (!status && resp->opcnt < args->opcnt) {
              op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              opdesc = OPDESC(op);
 
+             if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+                     goto call_op;
+

That means that once IS_STALE_FH is set, we will skip:

- nofilehandle checking
- moved checking
- fh_clear_wcc()
- resp_size checking
- current stateid capture

on every subsequent op. Can that really be right?

Quoted 12 lines
fs/nfsd/nfs4proc.c +1 -0
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
              /* Only from SEQUENCE */

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 105 lines
fs/nfsd/nfs4proc.c +2 -0
+                     goto out;
+             }
 encode_op:
              if (op->status == nfserr_replay_me) {
                      op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+     __be32 *p;
+
+     p = resp->cstate.putfh_errp;
+     *p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
      struct xdr_stream *xdr = &resp->xdr;
      struct nfs4_stateowner *so = resp->cstate.replay_owner;
+     struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_rqst *rqstp = resp->rqstp;
      int post_err_offset;
      nfsd4_enc encoder;
-     __be32 *p;
+     __be32 *p, *statp;
 
      p = xdr_reserve_space(xdr, 8);
      if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      }
      *p++ = cpu_to_be32(op->opnum);
      post_err_offset = xdr->buf->len;
+     statp = p;
 
      if (op->opnum == OP_ILLEGAL)
              goto status;
+
+     /** This is a COPY compound with a stale source server file handle.
+      * If OP_COPY processing determines that this is an intra server to
+      * server COPY, then this PUTFH should return nfserr_ stale so the
+      * putfh_errp will be set to nfserr_stale. If this is an inter server
+      * to server COPY, ignore the nfserr_stale.
+      */
+     if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+             cstate->putfh_errp = statp;
+
      BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
             !nfsd4_enc_ops[op->opnum]);
      encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define      nfserr_replay_me        cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define      nfserr_replay_cache     cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh  cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)       (l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -1
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
      size_t                  iovlen;
      u32                     minorversion;
      __be32                  status;
+     __be32                  *putfh_errp;
      stateid_t       current_stateid;
      stateid_t       save_stateid;
      /* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
              struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#55 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 8 lines · J. Bruce Fields
Quoted 4 lines · Olga Kornievskaia

What if instead the client won't send the 2nd FH in case of the inter server to server COPY and it would avoid all this stale handle business. So for the "intra" COPY it'll still be PUTFH, SAVEFH, PUTFH, COPY but for the "inter" COPY it'll be PUTFH, COPY.

That might be easier to implement, but it'd require a change to the protocol, right? I may not understand what you're proposing.

Sigh. You are right.

Quoted 276 lines

--b.

Quoted 273 lines · J. Bruce Fields
Quoted 271 lines

Some questions below, but really I think this approach can't work. Keep in mind this has to work not just for the one compound the current client uses, but for *any* compound containing a COPY, including those sent by future clients that may do something more complicated, or by malcious clients that may intentionally send weird compounds to make the server crash.

I think we want to flag the filehandle itself somehow, not the cstate.

And check very carefully to make sure that ops other than COPY can't get tripped up by one of these foreign filehandles.

Quoted 85 lines · Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +28 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
          struct nfsd4_putfh *putfh)
 {
+     __be32 ret;
+
      fh_put(&cstate->current_fh);
      cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
      memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
             putfh->pf_fhlen);
-     return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+             SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+             ret = 0;
+     }
+     return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
           void *arg)
 {
+     /**
+     * This is either an inter COPY (most likely) or an intra COPY with a
+     * stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+     * return nfserr_stale. No fh_dentry, just copy the file handle
+     * to use with the inter COPY READ.
+     */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             cstate->save_fh = cstate->current_fh;
+             return nfs_ok;
+     }
      if (!cstate->current_fh.fh_dentry)
              return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
      if (status)
              goto out;
 
+     /* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+             cstate->status = nfserr_copy_stalefh;
+             goto out_put;
+     }
+
      bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
                      dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
              status = nfs_ok;
      }
 
+out_put:
      fput(src);
      fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
      struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_fh *current_fh = &cstate->current_fh;
      struct svc_fh *save_fh = &cstate->save_fh;
+     int             i;
      __be32          status;
 
      svcxdr_init_encode(rqstp, resp);

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1826

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
              goto encode_op;
      }
 
+     /* NFSv4.2 COPY source file handle may be from a different server */
+     for (i = 0; i < args->opcnt; i++) {
+             op = &args->ops[i];
+             if (op->opnum == OP_COPY)
+                     SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+     }

So, if a compound has a COPY anywhere in it, then a PUTFH with a stale filehandle anywhere in the compound will temporarily succeed and result in IS_STALE_FH being set.

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1848

Quoted 10 lines
fs/nfsd/nfs4proc.c +3 -0
      while (!status && resp->opcnt < args->opcnt) {
              op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              opdesc = OPDESC(op);
 
+             if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+                     goto call_op;
+

That means that once IS_STALE_FH is set, we will skip:

- nofilehandle checking
- moved checking
- fh_clear_wcc()
- resp_size checking
- current stateid capture

on every subsequent op. Can that really be right?

Quoted 12 lines
fs/nfsd/nfs4proc.c +1 -0
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
              /* Only from SEQUENCE */

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 9 lines
fs/nfsd/nfs4proc.c +6 -0
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 105 lines
fs/nfsd/nfs4proc.c +2 -0
+                     goto out;
+             }
 encode_op:
              if (op->status == nfserr_replay_me) {
                      op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+     __be32 *p;
+
+     p = resp->cstate.putfh_errp;
+     *p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
      struct xdr_stream *xdr = &resp->xdr;
      struct nfs4_stateowner *so = resp->cstate.replay_owner;
+     struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_rqst *rqstp = resp->rqstp;
      int post_err_offset;
      nfsd4_enc encoder;
-     __be32 *p;
+     __be32 *p, *statp;
 
      p = xdr_reserve_space(xdr, 8);
      if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      }
      *p++ = cpu_to_be32(op->opnum);
      post_err_offset = xdr->buf->len;
+     statp = p;
 
      if (op->opnum == OP_ILLEGAL)
              goto status;
+
+     /** This is a COPY compound with a stale source server file handle.
+      * If OP_COPY processing determines that this is an intra server to
+      * server COPY, then this PUTFH should return nfserr_ stale so the
+      * putfh_errp will be set to nfserr_stale. If this is an inter server
+      * to server COPY, ignore the nfserr_stale.
+      */
+     if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+             cstate->putfh_errp = statp;
+
      BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
             !nfsd4_enc_ops[op->opnum]);
      encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define      nfserr_replay_me        cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define      nfserr_replay_cache     cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh  cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)       (l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -1
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
      size_t                  iovlen;
      u32                     minorversion;
      __be32                  status;
+     __be32                  *putfh_errp;
      stateid_t       current_stateid;
      stateid_t       save_stateid;
      /* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
              struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#56 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

In reply to J. Bruce Fields
Quoted 100 lines · J. Bruce Fields

Some questions below, but really I think this approach can't work. Keep in mind this has to work not just for the one compound the current client uses, but for *any* compound containing a COPY, including those sent by future clients that may do something more complicated, or by malcious clients that may intentionally send weird compounds to make the server crash.
I think we want to flag the filehandle itself somehow, not the state.

And check very carefully to make sure that ops other than COPY can't get tripped up by one of these foreign filehandles.

Quoted 88 lines · Olga Kornievskaia

The inter server to server COPY source server filehandle is guaranteed to be stale as the COPY is sent to the destination server.

---
 fs/nfsd/nfs4proc.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfsd/nfs4xdr.c  | 26 +++++++++++++++++++++++++-
 fs/nfsd/nfsd.h     |  2 ++
 fs/nfsd/xdr4.h     |  4 ++++
 4 files changed, 77 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +28 -1
index b49ff31..ceee852 100644
@@ -496,11 +496,19 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
          struct nfsd4_putfh *putfh)
 {
+     __be32 ret;
+
      fh_put(&cstate->current_fh);
      cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;
      memcpy(&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval,
             putfh->pf_fhlen);
-     return fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     ret = fh_verify(rqstp, &cstate->current_fh, 0, NFSD_MAY_BYPASS_GSS);
+     if (ret == nfserr_stale && HAS_CSTATE_FLAG(cstate, NO_VERIFY_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+             SET_CSTATE_FLAG(cstate, IS_STALE_FH);
+             ret = 0;
+     }
+     return ret;
 }
 
 static __be32
@@ -533,6 +541,16 @@ static __be32 nfsd4_open_omfg(struct svc_rqst *rqstp, struct nfsd4_compound_stat
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
           void *arg)
 {
+     /**
+     * This is either an inter COPY (most likely) or an intra COPY with a
+     * stale file handle. If the latter, nfsd4_copy will reset the PUTFH to
+     * return nfserr_stale. No fh_dentry, just copy the file handle
+     * to use with the inter COPY READ.
+     */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             cstate->save_fh = cstate->current_fh;
+             return nfs_ok;
+     }
      if (!cstate->current_fh.fh_dentry)
              return nfserr_nofilehandle;
 
@@ -1067,6 +1085,13 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
      if (status)
              goto out;
 
+     /* Intra copy source fh is stale. PUTFH will fail with ESTALE */
+     if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
+             CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+             cstate->status = nfserr_copy_stalefh;
+             goto out_put;
+     }
+
      bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
                      dst, copy->cp_dst_pos, copy->cp_count);
 
@@ -1081,6 +1106,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
              status = nfs_ok;
      }
 
+out_put:
      fput(src);
      fput(dst);
 out:
@@ -1759,6 +1785,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
      struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_fh *current_fh = &cstate->current_fh;
      struct svc_fh *save_fh = &cstate->save_fh;
+     int             i;
      __be32          status;
 
      svcxdr_init_encode(rqstp, resp);
@@ -1791,6 +1818,12 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
              goto encode_op;
      }
 
Quoted 10 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1826

Quoted 6 lines
fs/nfsd/nfs4proc.c +6 -0
+     /* NFSv4.2 COPY source file handle may be from a different server */
+     for (i = 0; i < args->opcnt; i++) {
+             op = &args->ops[i];
+             if (op->opnum == OP_COPY)
+                     SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
+     }

So, if a compound has a COPY anywhere in it, then a PUTFH with a stale filehandle anywhere in the compound will temporarily succeed and result in IS_STALE_FH being set.

How about changing it to be more restrictive by checking within the while loop that
if it's a PUTFH and it's followed by the SAVE_FH+PUTFH+COPY only then set the NO_VERIFY_FH.

        while (!status && resp->opcnt < args->opcnt) {
                op = &args->ops[resp->opcnt++];

                if (op->opnum == OP_PUTFH &&
                        args->ops[resp->opcnt].opnum == OP_SAVEFH &&
                        args->ops[resp->opcnt+1].opnum == OP_PUTFH &&
                        args->ops[resp->opcnt+2].opnum == OP_COPY) {
                        SET_CSTATE_FLAG(cstate, NO_VERIFY_FH);
                }

Would that address your concern or is this too restrictive?

I guess we can allow some other operations between the 2nd PUTFH and COPY because the 2nd filehandle will be validated and must be valid to continue processing the compound. It doesn't make sense to allow for operation after the 1st PUTFH as it can be a foreign handle.

Somewhere else you were talking about how a "foreign" file handle can mean something to the server. If that's the case and we do allow for operations between putfh, savefh, putfh then we'll get into trouble that I can't think we can
get out of.

If it's a inter copy and the source file handle means something to the server I can
think of the following scenario: the state won't be flagged IS_STALE then the filehandle would go thru the checks you listed below and can (unintentionally) result in an error (eg., err_moved?).

To really handle this, i can see doing something like, for a putfh with a copy elsewhere in the compound, skip ahead to the copy, find out if it's an inter copy,
then we know to mark the filehandle (as you suggested in the beginning) and check in all operations that they are not being done on a foreign handle. That seems very messy... Isn't restricting to putfh, savefh, putfh, copy a better way for the current situation?

Quoted 14 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1848

Quoted 11 lines
fs/nfsd/nfs4proc.c +3 -0
      while (!status && resp->opcnt < args->opcnt) {
              op = &args->ops[resp->opcnt++];
 
@@ -1810,6 +1843,9 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              opdesc = OPDESC(op);
 
+             if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+                     goto call_op;
+

That means that once IS_STALE_FH is set, we will skip:

Quoted 7 lines

- nofilehandle checking
- moved checking
- fh_clear_wcc()
- resp_size checking
- current stateid capture

on every subsequent op. Can that really be right?

This should be changed to
if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH) && op->opnum == OP_SAVEFH) then we need to skip the checks for savefh as it has no valid file handle.

Does that address your concern?

Quoted 11 lines
Quoted 11 lines
fs/nfsd/nfs4proc.c +1 -0
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
Quoted 14 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 11 lines
fs/nfsd/nfs4proc.c +6 -0
              /* Only from SEQUENCE */
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

If COPY comes in PUTFH,SAVEFH, PUTFH,COPY compound then I think it's safe?

Quoted 114 lines
Quoted 106 lines
fs/nfsd/nfs4proc.c +2 -0
+                     goto out;
+             }
 encode_op:
              if (op->status == nfserr_replay_me) {
                      op->replay = &cstate->replay_owner->so_replay;
fs/nfsd/nfs4xdr.c +25 -1
index 3e08c15..2896a11 100644
@@ -4624,15 +4624,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      return nfserr_rep_too_big;
 }
 
+/** Rewind the encoding to return nfserr_stale on the PUTFH
+ * in this failed Intra COPY compound
+ */
+void
+nfsd4_adjust_encode(struct nfsd4_compoundres *resp)
+{
+     __be32 *p;
+
+     p = resp->cstate.putfh_errp;
+     *p++ = nfserr_stale;
+}
+
 void
 nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op)
 {
      struct xdr_stream *xdr = &resp->xdr;
      struct nfs4_stateowner *so = resp->cstate.replay_owner;
+     struct nfsd4_compound_state *cstate = &resp->cstate;
      struct svc_rqst *rqstp = resp->rqstp;
      int post_err_offset;
      nfsd4_enc encoder;
-     __be32 *p;
+     __be32 *p, *statp;
 
      p = xdr_reserve_space(xdr, 8);
      if (!p) {
@@ -4641,9 +4654,20 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)
      }
      *p++ = cpu_to_be32(op->opnum);
      post_err_offset = xdr->buf->len;
+     statp = p;
 
      if (op->opnum == OP_ILLEGAL)
              goto status;
+
+     /** This is a COPY compound with a stale source server file handle.
+      * If OP_COPY processing determines that this is an intra server to
+      * server COPY, then this PUTFH should return nfserr_ stale so the
+      * putfh_errp will be set to nfserr_stale. If this is an inter server
+      * to server COPY, ignore the nfserr_stale.
+      */
+     if (op->opnum == OP_PUTFH && HAS_CSTATE_FLAG(cstate, IS_STALE_FH))
+             cstate->putfh_errp = statp;
+
      BUG_ON(op->opnum < 0 || op->opnum >= ARRAY_SIZE(nfsd4_enc_ops) ||
             !nfsd4_enc_ops[op->opnum]);
      encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
index d966068..8d6fb0f 100644
@@ -272,6 +272,8 @@ static inline bool nfsd4_spo_must_allow(struct svc_rqst *rqstp)
 #define      nfserr_replay_me        cpu_to_be32(11001)
 /* nfs41 replay detected */
 #define      nfserr_replay_cache     cpu_to_be32(11002)
+/* nfs42 intra copy failed with nfserr_stale */
+#define nfserr_copy_stalefh  cpu_to_be32(1103)
 
 /* Check for dir entries '.' and '..' */
 #define isdotent(n, l)       (l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -1
index 38fcb4f..aa94295 100644
@@ -45,6 +45,8 @@
 
 #define CURRENT_STATE_ID_FLAG (1<<0)
 #define SAVED_STATE_ID_FLAG (1<<1)
+#define NO_VERIFY_FH (1<<2)
+#define IS_STALE_FH  (1<<3)
 
 #define SET_CSTATE_FLAG(c, f) ((c)->sid_flags |= (f))
 #define HAS_CSTATE_FLAG(c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {
      size_t                  iovlen;
      u32                     minorversion;
      __be32                  status;
+     __be32                  *putfh_errp;
      stateid_t       current_stateid;
      stateid_t       save_stateid;
      /* to indicate current and saved state id presents */
@@ -705,6 +708,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,
 int nfs4svc_encode_compoundres(struct svc_rqst *, __be32 *,
              struct nfsd4_compoundres *);
 __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *, u32);
+void nfsd4_adjust_encode(struct nfsd4_compoundres *);
 void nfsd4_encode_operation(struct nfsd4_compoundres *, struct nfsd4_op *);
 void nfsd4_encode_replay(struct xdr_stream *xdr, struct nfsd4_op *op);
 __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
--

1.8.3.1

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#57 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 4 lines · Olga Kornievskaia

How about changing it to be more restrictive by checking within the while loop that
if it's a PUTFH and it's followed by the SAVE_FH+PUTFH+COPY only then set the NO_VERIFY_FH.

I agree that the only op that could reasonably follow the first PUTFH of a foreign filehandle is a SAVEFH. But once we've saved the foreign filehandle, the client could in theory do all sorts of stuff, as you say:

Quoted 3 lines

I guess we can allow some other operations between the 2nd PUTFH and COPY because the 2nd filehandle will be validated and must be valid to continue processing the compound.

PUTHF+SAVEFH+PUTFH+GETATTR+COPY might be useful to get pre-copy attributes of the target file, for example?

I'd rather not restrict this if we don't need to.

We could do something like:

- if this op is PUTFH
- if the following is not SAVEFH, stop and verify the filehandle.
- otherwise, skip to the next operation that uses a saved filehandle. The possibilities are:
- RENAME, LINK, RESTOREFH, CLONE: stop and verify the filehandle.
- COPY: if it's a local copy, stop and verify the filehandle. Otherwise, allow the PUTFH to succeed and delay verification.

Quoted 10 lines

Somewhere else you were talking about how a "foreign" file handle can mean something to the server. If that's the case and we do allow for operations between putfh, savefh, putfh then we'll get into trouble that I can't think we can get out of.

If it's a inter copy and the source file handle means something to the server I can think of the following scenario: the state won't be flagged IS_STALE then the filehandle would go thru the checks you listed below and can (unintentionally) result in an error (eg., err_moved?).

I don't see any problem with just leaving those checks in. If the current filehandle is not validated, then there's already a current_fh->fh_dentry check that skips the ABSENT_FH check, for example.

Quoted 5 lines

This should be changed to
if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH) && op->opnum == OP_SAVEFH) then we need to skip the checks for savefh as it has no valid file handle.

Does that address your concern?

I think you only need to skip the nofilehandle check in this case, not the other stuff.

I don't think the IS_STALE_FH flag is needed at all.

Quoted 25 lines
Quoted 10 lines
Quoted 10 lines
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
fs/nfsd/nfs4proc.c +1 -0
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
Quoted 14 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 11 lines
fs/nfsd/nfs4proc.c +6 -0
              /* Only from SEQUENCE */
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 2 lines

If COPY comes in PUTFH,SAVEFH, PUTFH,COPY compound then I think it's safe?

The spec says "If a server supports the inter-server copy feature, a PUTFH followed by a SAVEFH MUST NOT return NFS4ERR_STALE for either operation. These restrictions do not pose substantial difficulties for servers. CURRENT_FH and SAVED_FH may be validated in the context of the operation referencing them and an NFS4ERR_STALE error returned for an invalid filehandle at that point."

So we're supposed to return NFS4ERR_STALE on the COPY, not the PUTFH. So there's no need for this backtracking.

--b.

#58 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 22 lines · J. Bruce Fields
Quoted 4 lines · Olga Kornievskaia

How about changing it to be more restrictive by checking within the while loop that
if it's a PUTFH and it's followed by the SAVE_FH+PUTFH+COPY only then set the NO_VERIFY_FH.

I agree that the only op that could reasonably follow the first PUTFH of a foreign filehandle is a SAVEFH. But once we've saved the foreign filehandle, the client could in theory do all sorts of stuff, as you say:

Quoted 3 lines

I guess we can allow some other operations between the 2nd PUTFH and COPY because the 2nd filehandle will be validated and must be valid to continue processing the compound.

PUTHF+SAVEFH+PUTFH+GETATTR+COPY might be useful to get pre-copy attributes of the target file, for example?

I'd rather not restrict this if we don't need to.

We could do something like:

Quoted 10 lines

- if this op is PUTFH
- if the following is not SAVEFH, stop and verify the filehandle.
- otherwise, skip to the next operation that uses a saved filehandle. The possibilities are:
- RENAME, LINK, RESTOREFH, CLONE: stop and verify the filehandle.
- COPY: if it's a local copy, stop and verify the filehandle. Otherwise, allow the PUTFH to succeed and delay verification.

OK so I can peep into the upcoming copy and see if it's inter or intra and then based on that I can set NO_VERIFY_FH for processing the putfh.

Quoted 15 lines
Quoted 10 lines

Somewhere else you were talking about how a "foreign" file handle can mean something to the server. If that's the case and we do allow for operations between putfh, savefh, putfh then we'll get into trouble that I can't think we can get out of.

If it's a inter copy and the source file handle means something to the server I can think of the following scenario: the state won't be flagged IS_STALE then the filehandle would go thru the checks you listed below and can (unintentionally) result in an error (eg., err_moved?).

I don't see any problem with just leaving those checks in. If the current filehandle is not validated, then there's already a current_fh->fh_dentry check that skips the ABSENT_FH check, for example.

Quoted 8 lines
Quoted 5 lines

This should be changed to
if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH) && op->opnum == OP_SAVEFH) then we need to skip the checks for savefh as it has no valid file handle.

Does that address your concern?

I think you only need to skip the nofilehandle check in this case, not the other stuff.

As long as I can add that current_fh->fh_export is not null for the 2nd check otherwise it leads to an oops.

Quoted 1 line

I don't think the IS_STALE_FH flag is needed at all.

We still need it so that the savefh process can skip the check for the filehandle.
Or I can use something like checking that fh_dentry and fh_export are both null and use that to mean I need to check the filehandle check in savefh.

I think explicitly marking it makes the code either to understand instead of 'knowning/remembering' that lack of fh_dentry+fh_export means inter copy?

Quoted 35 lines
Quoted 25 lines
Quoted 10 lines
Quoted 10 lines
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
fs/nfsd/nfs4proc.c +1 -0
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
Quoted 14 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 11 lines
fs/nfsd/nfs4proc.c +6 -0
              /* Only from SEQUENCE */
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 2 lines

If COPY comes in PUTFH,SAVEFH, PUTFH,COPY compound then I think it's safe?

The spec says "If a server supports the inter-server copy feature, a PUTFH followed by a SAVEFH MUST NOT return NFS4ERR_STALE for either operation. These restrictions do not pose substantial difficulties for servers. CURRENT_FH and SAVED_FH may be validated in the context of the operation referencing them and an NFS4ERR_STALE error returned for an invalid filehandle at that point."

Quoted 2 lines

So we're supposed to return NFS4ERR_STALE on the COPY, not the PUTFH. So there's no need for this backtracking.

You are right.

#59 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 56 lines · Olga Kornievskaia
Quoted 22 lines · J. Bruce Fields
Quoted 4 lines · Olga Kornievskaia

How about changing it to be more restrictive by checking within the while loop that
if it's a PUTFH and it's followed by the SAVE_FH+PUTFH+COPY only then set the NO_VERIFY_FH.

I agree that the only op that could reasonably follow the first PUTFH of a foreign filehandle is a SAVEFH. But once we've saved the foreign filehandle, the client could in theory do all sorts of stuff, as you say:

Quoted 3 lines

I guess we can allow some other operations between the 2nd PUTFH and COPY because the 2nd filehandle will be validated and must be valid to continue processing the compound.

PUTHF+SAVEFH+PUTFH+GETATTR+COPY might be useful to get pre-copy attributes of the target file, for example?

I'd rather not restrict this if we don't need to.

We could do something like:

Quoted 10 lines

- if this op is PUTFH
- if the following is not SAVEFH, stop and verify the filehandle.
- otherwise, skip to the next operation that uses a saved filehandle. The possibilities are:
- RENAME, LINK, RESTOREFH, CLONE: stop and verify the filehandle.
- COPY: if it's a local copy, stop and verify the filehandle. Otherwise, allow the PUTFH to succeed and delay verification.

OK so I can peep into the upcoming copy and see if it's inter or intra and then based on that I can set NO_VERIFY_FH for processing the putfh.

Quoted 19 lines
Quoted 10 lines

Somewhere else you were talking about how a "foreign" file handle can mean something to the server. If that's the case and we do allow for operations between putfh, savefh, putfh then we'll get into trouble that I can't think we can get out of.

If it's a inter copy and the source file handle means something to the server I can think of the following scenario: the state won't be flagged IS_STALE then the filehandle would go thru the checks you listed below and can (unintentionally) result in an error (eg., err_moved?).

I don't see any problem with just leaving those checks in. If the current filehandle is not validated, then there's already a current_fh->fh_dentry check that skips the ABSENT_FH check, for example.

Quoted 4 lines

This should be changed to
if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH) && op->opnum == OP_SAVEFH) then we need to skip the checks for savefh as it has no valid file handle.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Does that address your concern?

I think you only need to skip the nofilehandle check in this case, not the other stuff.

As long as I can add that current_fh->fh_export is not null for the 2nd check otherwise it leads to an oops.

fh_export should be set whenever fh_dentry is.

Quoted 7 lines
Quoted 1 line

I don't think the IS_STALE_FH flag is needed at all.

We still need it so that the savefh process can skip the check for the filehandle.

Or I can use something like checking that fh_dentry and fh_export are both null and use that to mean I need to check the filehandle check in savefh.

Yes, or you can just check the fh_dentry.

Quoted 2 lines

I think explicitly marking it makes the code either to understand instead of 'knowning/remembering' that lack of fh_dentry+fh_export means inter copy?

If it turns out we still need a flag I think I'd rather it be set on the filehandle somehow rather than in the cstate. It's the filehandle that's unverified.

--b.

Quoted 40 lines
Quoted 36 lines
Quoted 26 lines
Quoted 11 lines
Quoted 11 lines
              if (!current_fh->fh_dentry) {
                      if (!(opdesc->op_flags & ALLOWED_WITHOUT_FH)) {
                              op->status = nfserr_nofilehandle;
fs/nfsd/nfs4proc.c +1 -0
@@ -1844,6 +1880,7 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
 
              if (opdesc->op_get_currentstateid)
                      opdesc->op_get_currentstateid(cstate, &op->u);
+call_op:
              op->status = opdesc->op_func(rqstp, cstate, &op->u);
 
Quoted 14 lines

fs/nfsd/nfs4proc.c static void svcxdr_init_encode(struct svc_rqst *rqstp, · line 1907

Quoted 11 lines
fs/nfsd/nfs4proc.c +6 -0
              /* Only from SEQUENCE */
@@ -1862,6 +1899,14 @@ static void svcxdr_init_encode(struct svc_rqst *rqstp,
                      if (need_wrongsec_check(rqstp))
                              op->status = check_nfsd_access(current_fh->fh_export, rqstp);
              }
+             /* Only from intra COPY */
+             if (cstate->status == nfserr_copy_stalefh) {
+                     dprintk("%s NFS4.2 intra COPY stale src filehandle\n",
+                             __func__);
+                     status = nfserr_stale;
+                     nfsd4_adjust_encode(resp);

Are you sure it's safe just to throw away any operations since that stale PUTFH? What if some of those operations had side effects?

Quoted 2 lines

If COPY comes in PUTFH,SAVEFH, PUTFH,COPY compound then I think it's safe?

The spec says "If a server supports the inter-server copy feature, a PUTFH followed by a SAVEFH MUST NOT return NFS4ERR_STALE for either operation. These restrictions do not pose substantial difficulties for servers. CURRENT_FH and SAVED_FH may be validated in the context of the operation referencing them and an NFS4ERR_STALE error returned for an invalid filehandle at that point."

Quoted 2 lines

So we're supposed to return NFS4ERR_STALE on the COPY, not the PUTFH. So there's no need for this backtracking.

You are right.

#60 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 3 lines · J. Bruce Fields
Quoted 2 lines · Olga Kornievskaia

We still need it so that the savefh process can skip the check for the filehandle.

By the way, I think the current check in nfsd4_savefh is redundant; we can probably apply something like the following. (Untested.)

--b.

commit c1bab7c64408

Author: J. Bruce Fields <[email protected]>
Date:   Fri Sep 15 16:02:52 2017 -0400

nfsd: remove unnecessary nofilehandle checks

These checks should have already be done centrally in nfsd4_proc_compound, the checks in each individual operation are unnecessary.

fs/nfsd/nfs4proc.c +0 -12
index 3c69db7d4905..07ebae912e0e 100644
@@ -485,9 +485,6 @@ static __be32
 nfsd4_getfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	    union nfsd4_op_u *u)
 {
-	if (!cstate->current_fh.fh_dentry)
-		return nfserr_nofilehandle;
-
 	u->getfh = &cstate->current_fh;
 	return nfs_ok;
 }
@@ -535,9 +532,6 @@ static __be32
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     union nfsd4_op_u *u)
 {
-	if (!cstate->current_fh.fh_dentry)
-		return nfserr_nofilehandle;
-
 	fh_dup2(&cstate->save_fh, &cstate->current_fh);
 	if (HAS_STATE_ID(cstate, CURRENT_STATE_ID_FLAG)) {
 		memcpy(&cstate->save_stateid, &cstate->current_stateid, sizeof(stateid_t));
@@ -703,10 +697,7 @@ nfsd4_link(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	   union nfsd4_op_u *u)
 {
 	struct nfsd4_link *link = &u->link;
-	__be32 status = nfserr_nofilehandle;
 
-	if (!cstate->save_fh.fh_dentry)
-		return status;
 	status = nfsd_link(rqstp, &cstate->current_fh,
 			   link->li_name, link->li_namelen, &cstate->save_fh);
 	if (!status)
@@ -850,10 +841,7 @@ nfsd4_rename(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     union nfsd4_op_u *u)
 {
 	struct nfsd4_rename *rename = &u->rename;
-	__be32 status = nfserr_nofilehandle;
 
-	if (!cstate->save_fh.fh_dentry)
-		return status;
 	if (opens_in_grace(SVC_NET(rqstp)) &&
 		!(cstate->save_fh.fh_export->ex_flags & NFSEXP_NOSUBTREECHECK))
 		return nfserr_grace;
#61 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

Quoted 7 lines · J. Bruce Fields
Quoted 3 lines · J. Bruce Fields
Quoted 2 lines · Olga Kornievskaia

We still need it so that the savefh process can skip the check for the filehandle.

By the way, I think the current check in nfsd4_savefh is redundant; we can probably apply something like the following. (Untested.)

If those checks are not necessary then IS_STALE_FH is not necessary.

Quoted 61 lines

--b.

commit c1bab7c64408

Author: J. Bruce Fields <[email protected]>
Date:   Fri Sep 15 16:02:52 2017 -0400

nfsd: remove unnecessary nofilehandle checks

These checks should have already be done centrally in nfsd4_proc_compound, the checks in each individual operation are unnecessary.

fs/nfsd/nfs4proc.c +0 -12
index 3c69db7d4905..07ebae912e0e 100644
@@ -485,9 +485,6 @@ static __be32
 nfsd4_getfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
            union nfsd4_op_u *u)
 {
-       if (!cstate->current_fh.fh_dentry)
-               return nfserr_nofilehandle;
-
        u->getfh = &cstate->current_fh;
        return nfs_ok;
 }
@@ -535,9 +532,6 @@ static __be32
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
             union nfsd4_op_u *u)
 {
-       if (!cstate->current_fh.fh_dentry)
-               return nfserr_nofilehandle;
-
        fh_dup2(&cstate->save_fh, &cstate->current_fh);
        if (HAS_STATE_ID(cstate, CURRENT_STATE_ID_FLAG)) {
                memcpy(&cstate->save_stateid, &cstate->current_stateid, sizeof(stateid_t));
@@ -703,10 +697,7 @@ nfsd4_link(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
           union nfsd4_op_u *u)
 {
        struct nfsd4_link *link = &u->link;
-       __be32 status = nfserr_nofilehandle;
 
-       if (!cstate->save_fh.fh_dentry)
-               return status;
        status = nfsd_link(rqstp, &cstate->current_fh,
                           link->li_name, link->li_namelen, &cstate->save_fh);
        if (!status)
@@ -850,10 +841,7 @@ nfsd4_rename(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
             union nfsd4_op_u *u)
 {
        struct nfsd4_rename *rename = &u->rename;
-       __be32 status = nfserr_nofilehandle;
 
-       if (!cstate->save_fh.fh_dentry)
-               return status;
        if (opens_in_grace(SVC_NET(rqstp)) &&
                !(cstate->save_fh.fh_export->ex_flags & NFSEXP_NOSUBTREECHECK))
                return nfserr_grace;
#62 lore raw

Re: [RFC v3 27/42] NFSD: allow inter server COPY to have a STALE source server fh

In reply to J. Bruce Fields
Quoted 2 lines · J. Bruce Fields

By the way, I think the current check in nfsd4_savefh is redundant; we can probably apply something like the following. (Untested.)

Fixed and tested version for 4.15.

--b.

commit 6eb90459308b

Author: J. Bruce Fields <[email protected]>
Date:   Fri Sep 15 16:02:52 2017 -0400

nfsd: remove unnecessary nofilehandle checks

These checks should have already be done centrally in nfsd4_proc_compound, the checks in each individual operation are unnecessary.

fs/nfsd/nfs4proc.c +2 -12
index 3c69db7d4905..7aff6d383d34 100644
@@ -485,9 +485,6 @@ static __be32
 nfsd4_getfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	    union nfsd4_op_u *u)
 {
-	if (!cstate->current_fh.fh_dentry)
-		return nfserr_nofilehandle;
-
 	u->getfh = &cstate->current_fh;
 	return nfs_ok;
 }
@@ -535,9 +532,6 @@ static __be32
 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     union nfsd4_op_u *u)
 {
-	if (!cstate->current_fh.fh_dentry)
-		return nfserr_nofilehandle;
-
 	fh_dup2(&cstate->save_fh, &cstate->current_fh);
 	if (HAS_STATE_ID(cstate, CURRENT_STATE_ID_FLAG)) {
 		memcpy(&cstate->save_stateid, &cstate->current_stateid, sizeof(stateid_t));
@@ -703,10 +697,8 @@ nfsd4_link(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	   union nfsd4_op_u *u)
 {
 	struct nfsd4_link *link = &u->link;
-	__be32 status = nfserr_nofilehandle;
+	__be32 status;
 
-	if (!cstate->save_fh.fh_dentry)
-		return status;
 	status = nfsd_link(rqstp, &cstate->current_fh,
 			   link->li_name, link->li_namelen, &cstate->save_fh);
 	if (!status)
@@ -850,10 +842,8 @@ nfsd4_rename(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     union nfsd4_op_u *u)
 {
 	struct nfsd4_rename *rename = &u->rename;
-	__be32 status = nfserr_nofilehandle;
+	__be32 status;
 
-	if (!cstate->save_fh.fh_dentry)
-		return status;
 	if (opens_in_grace(SVC_NET(rqstp)) &&
 		!(cstate->save_fh.fh_export->ex_flags & NFSEXP_NOSUBTREECHECK))
 		return nfserr_grace;
#63 lore raw

RFC v3 28/42 NFSD add nfs4 inter ssc to nfsd4_copy

In reply to Olga Kornievskaia

Given a universal address, mount the source server from the destination server. Use an internal mount. Call the NFS client nfs42_ssc_open to obtain the NFS struct file suitable for nfsd_copy_range.

Add Kconfig dependencies for inter server to server copy

---
 fs/nfsd/Kconfig      |  10 ++
 fs/nfsd/nfs4proc.c   | 262 +++++++++++++++++++++++++++++++++++++++++++++++++--
 include/linux/nfs4.h |   1 +
 3 files changed, 263 insertions(+), 10 deletions(-)
fs/nfsd/Kconfig +10 -0
index 20b1c17..37ff3d5 100644
@@ -131,6 +131,16 @@ config NFSD_FLEXFILELAYOUT
 
 	  If unsure, say N.
 
+config NFSD_V4_2_INTER_SSC
+	bool "NFSv4.2 inter server to server COPY"
+	depends on NFSD_V4 && NFS_V4_1 && NFS_V4_2
+	help
+	  This option enables support for NFSv4.2 inter server to
+	  server copy where the destination server calls the NFSv4.2
+	  client to read the data to copy from the source server.
+
+	  If unsure, say N.
+
 config NFSD_V4_SECURITY_LABEL
 	bool "Provide Security Label support for NFSv4 server"
 	depends on NFSD_V4 && SECURITY
fs/nfsd/nfs4proc.c +252 -10

284 lines not on this page · raw message

include/linux/nfs4.h +1 -0
index 5aa36ac..843443b 100644
@@ -16,6 +16,7 @@
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
 #include <linux/sunrpc/msg_prot.h>
+#include <linux/nfs.h>
 
 enum nfs4_acl_whotype {
 	NFS4_ACL_WHO_NAMED = 0,
Signature · 1 line
1.8.3.1
#64 lore raw

Re: [RFC v3 28/42] NFSD add nfs4 inter ssc to nfsd4_copy

Quoted 42 lines · Olga Kornievskaia

Given a universal address, mount the source server from the destination server. Use an internal mount. Call the NFS client nfs42_ssc_open to obtain the NFS struct file suitable for nfsd_copy_range.

Add Kconfig dependencies for inter server to server copy

---
 fs/nfsd/Kconfig      |  10 ++
 fs/nfsd/nfs4proc.c   | 262 +++++++++++++++++++++++++++++++++++++++++++++++++--
 include/linux/nfs4.h |   1 +
 3 files changed, 263 insertions(+), 10 deletions(-)
fs/nfsd/Kconfig +10 -0
index 20b1c17..37ff3d5 100644
@@ -131,6 +131,16 @@ config NFSD_FLEXFILELAYOUT
 
 	  If unsure, say N.
 
+config NFSD_V4_2_INTER_SSC
+	bool "NFSv4.2 inter server to server COPY"
+	depends on NFSD_V4 && NFS_V4_1 && NFS_V4_2
+	help
+	  This option enables support for NFSv4.2 inter server to
+	  server copy where the destination server calls the NFSv4.2
+	  client to read the data to copy from the source server.
+
+	  If unsure, say N.
+
 config NFSD_V4_SECURITY_LABEL
 	bool "Provide Security Label support for NFSv4 server"
 	depends on NFSD_V4 && SECURITY
fs/nfsd/nfs4proc.c +0 -0
index ceee852..b1095e9 100644
@@ -1072,16 +1072,227 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	return status;
 }
 

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1084

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+#ifdef CONFIG_NFSD_V4_2_INTER_SSC
+
+extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
+				   struct nfs_fh *src_fh,
+				   nfs4_stateid *stateid);
+extern struct file *nfs42_ssc_close(struct file *filep);
+
+extern void nfs_sb_deactive(struct super_block *sb);
+
+#define NFSD42_INTERSSC_RAWDATA "minorversion=2,vers=4,addr=%s,clientaddr=%s"

INTERSCC_RAWDATA isn't the most helpful name. Maybe MOUNTOPTS could be in there?

Quoted 12 lines
fs/nfsd/nfs4proc.c +12 -0
+
+/**
+ * Support one copy source server for now.
+ */
+static struct vfsmount *
+nfsd4_interssc_connect(struct nl4_servers *nss, struct svc_rqst *rqstp)
+{
+	struct file_system_type *type;
+	struct vfsmount *ss_mnt;
+	struct nfs42_netaddr *naddr;
+	struct sockaddr_storage tmp_addr;
+	size_t tmp_addrlen, match_netid_len = 3;

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1106

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	char *startsep = "", *endsep = "", *match_netid = "tcp";
+	char *ipaddr, *ipaddr2, *raw_data;
+	int len, raw_len, status = -EINVAL;
+
+	/* Currently support for one NL4_NETADDR source server */
+	if (nss->nl_svr->nl4_type != NL4_NETADDR) {
+		WARN(nss->nl_svr->nl4_type != NL4_NETADDR,
+			"nfsd4_copy src server not NL4_NETADDR\n");
+		goto out_err;
+	}

We already checked this at xdr decoding time, didn't we? Well, maybe i'ts still worth a WARN. Better, as mentioned before, maybe the data structures just should assume only 1 NL4_NETADDR source server.

Quoted 6 lines
fs/nfsd/nfs4proc.c +6 -0
+
+	naddr = &nss->nl_svr->u.nl4_addr;
+
+	tmp_addrlen = rpc_uaddr2sockaddr(SVC_NET(rqstp), naddr->na_uaddr,
+					naddr->na_uaddr_len,
+					(struct sockaddr *)&tmp_addr,

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1122

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+					sizeof(tmp_addr));
+	if (tmp_addrlen == 0)
+		goto out_err;
+
+	if (tmp_addr.ss_family == AF_INET6) {
+		startsep = "[";
+		endsep = "]";
+		match_netid = "tcp6";
+		match_netid_len = 4;
+	}

What about the tcp/ipv4 case? Oh, I see, it's handled by the initialization above. That's a little confusing.

Quoted 18 lines
fs/nfsd/nfs4proc.c +18 -0
+
+	if (naddr->na_netid_len != match_netid_len ||
+	    strncmp(naddr->na_netid, match_netid, naddr->na_netid_len))
+		goto out_err;
+
+	/* Construct the raw data for the vfs_kern_mount call */
+	len = RPC_MAX_ADDRBUFLEN + 1;
+	ipaddr = kzalloc(len, GFP_KERNEL);
+	if (!ipaddr)
+		goto out_err;
+
+	rpc_ntop((struct sockaddr *)&tmp_addr, ipaddr, len);
+
+	/* 2 for ipv6 endsep and startsep. 3 for ":/" and trailing '/0'*/
+	ipaddr2 = kzalloc(len + 5, GFP_KERNEL);
+	if (!ipaddr2)
+		goto out_free_ipaddr;
+

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1150

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	rpc_ntop((struct sockaddr *)&rqstp->rq_daddr, ipaddr2, len + 5);
+
+	raw_len = strlen(NFSD42_INTERSSC_RAWDATA) + strlen(ipaddr) +
+			strlen(ipaddr2);
+	raw_data = kzalloc(raw_len, GFP_KERNEL);
+	if (!raw_data)
+		goto out_free_ipaddr2;
+
+	snprintf(raw_data, raw_len, NFSD42_INTERSSC_RAWDATA, ipaddr,
+		 ipaddr2);

Could "raw_data" be named more helpfully?

Quoted 5 lines
fs/nfsd/nfs4proc.c +5 -0
+
+	status = -ENODEV;
+	type = get_fs_type("nfs");
+	if (!type)
+		goto out_free_rawdata;

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1165

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+
+	/* Set the server:<export> for the vfs_kerne_mount call */
+	memset(ipaddr2, 0, len + 5);
+	snprintf(ipaddr2, len + 5, "%s%s%s:/", startsep, ipaddr, endsep);
+
+	dprintk("%s  Raw mount data:  %s server:export %s\n", __func__,
+		raw_data, ipaddr2);
+
+	/* Use an 'internal' mount: MS_KERNMOUNT -> MNT_INTERNAL */
+	ss_mnt = vfs_kern_mount(type, MS_KERNMOUNT, ipaddr2, raw_data);

I wonder if creating a mount (even a kernel internal one) has any unexpected side effects. Is it a problem that it could share caches with any already-existing mounts of that export?

Quoted 17 lines
fs/nfsd/nfs4proc.c +17 -0
+	if (IS_ERR(ss_mnt)) {
+		status = PTR_ERR(ss_mnt);
+		goto out_free_rawdata;
+	}
+
+	kfree(raw_data);
+	kfree(ipaddr2);
+	kfree(ipaddr);
+
+	return ss_mnt;
+
+out_free_rawdata:
+	kfree(raw_data);
+out_free_ipaddr2:
+	kfree(ipaddr2);
+out_free_ipaddr:
+	kfree(ipaddr);

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1192

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+out_err:
+	dprintk("--> %s ERROR %d\n", __func__, status);
+	return ERR_PTR(status);
+}
+
+static void
+nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
+{
+	mntput(ss_mnt);
+	nfs_sb_deactive(ss_mnt->mnt_sb);

OK, I don't claim to understand that.

Are you sure it's still safe to dereference ss_mnt after calling mntput() on it?

Quoted 28 lines
fs/nfsd/nfs4proc.c +28 -0
+}
+
+/**
+ * nfsd4_setup_inter_ssc
+ *
+ * Verify COPY destination stateid.
+ * Connect to the source server with NFSv4.1.
+ * Create the source struct file for nfsd_copy_range.
+ * Called with COPY cstate:
+ *    SAVED_FH: source filehandle
+ *    CURRENT_FH: destination filehandle
+ *
+ * Returns errno (not nfserrxxx)
+ */
+static struct vfsmount *
+nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
+			struct nfsd4_compound_state *cstate,
+			struct nfsd4_copy *copy, struct file **src,
+			struct file **dst)
+{
+	struct svc_fh *s_fh = NULL;
+	stateid_t *s_stid = &copy->cp_src_stateid;
+	struct nfs_fh fh;
+	nfs4_stateid stateid;
+	struct file *filp;
+	struct vfsmount *ss_mnt;
+	__be32 status;
+

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1230

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	/* Verify the destination stateid and set dst struct file*/
+	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
+					&copy->cp_dst_stateid,
+					WR_STATE, dst, NULL);
+	if (status) {
+		ss_mnt = ERR_PTR(be32_to_cpu(status));
+		goto out;
+	}
+
+	/* Inter copy source fh is always stale */

Not necessarily.

Quoted 5 lines
fs/nfsd/nfs4proc.c +5 -0
+	CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
+
+	ss_mnt = nfsd4_interssc_connect(&copy->cp_src, rqstp);
+	if (IS_ERR(ss_mnt))
+		goto out;

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1245

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+
+	s_fh = &cstate->save_fh;
+
+	fh.size = s_fh->fh_handle.fh_size;
+	memcpy(fh.data, &s_fh->fh_handle.fh_base, fh.size);
+	stateid.seqid = s_stid->si_generation;
+	memcpy(stateid.other, (void *)&s_stid->si_opaque,
+		sizeof(stateid_opaque_t));
+
+	filp =  nfs42_ssc_open(ss_mnt, &fh, &stateid);

So that was defined in the client-side patches?

Quoted 80 lines
fs/nfsd/nfs4proc.c +62 -7
+	if (IS_ERR(filp)) {
+		nfsd4_interssc_disconnect(ss_mnt);
+		return ERR_CAST(filp);
+	}
+	*src = filp;
+out:
+	return ss_mnt;
+}
+
+static void
+nfsd4_cleanup_inter_ssc(struct vfsmount *ss_mnt, struct file *src,
+			struct file *dst)
+{
+	nfs42_ssc_close(src);
+	fput(src);
+	fput(dst);
+
+	nfsd4_interssc_disconnect(ss_mnt);
+
+}
+
+#else /* CONFIG_NFSD_V4_2_INTER_SSC */
+
+static struct vfsmount *
+nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
+			struct nfsd4_compound_state *cstate,
+			struct nfsd4_copy *copy, struct file **src,
+			struct file **dst)
+{
+	return ERR_PTR(-EINVAL);
+}
+
+static void
+nfsd4_cleanup_inter_ssc(struct vfsmount *ss_mnt, struct file *src,
+			struct file *dst)
+{
+}
+
+#endif /* CONFIG_NFSD_V4_2_INTER_SSC */
+
 static __be32
-nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
-		struct nfsd4_copy *copy)
+nfsd4_setup_intra_ssc(struct svc_rqst *rqstp,
+		      struct nfsd4_compound_state *cstate,
+		      struct nfsd4_copy *copy, struct file **src,
+		      struct file **dst)
 {
-	struct file *src, *dst;
 	__be32 status;
-	ssize_t bytes;
 
-	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid, &src,
-				   &copy->cp_dst_stateid, &dst);
+	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid, src,
+				   &copy->cp_dst_stateid, dst);
 	if (status)
 		goto out;
 
@@ -1089,7 +1300,37 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	if (HAS_CSTATE_FLAG(cstate, IS_STALE_FH)) {
 		CLEAR_CSTATE_FLAG(cstate, IS_STALE_FH);
 		cstate->status = nfserr_copy_stalefh;
-		goto out_put;
+	}
+out:
+	return status;
+}
+
+static void
+nfsd4_cleanup_intra_ssc(struct file *src, struct file *dst)
+{
+	fput(src);
+	fput(dst);
+}
+
+static __be32
+nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
+	struct nfsd4_copy *copy)
+{

fs/nfsd/nfs4proc.c static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write) · line 1328

Quoted 10 lines
fs/nfsd/nfs4proc.c +10 -0
+	struct vfsmount *ss_mnt = NULL;
+	struct file *src, *dst;
+	__be32 status;
+	ssize_t bytes;
+
+	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
+		ss_mnt = nfsd4_setup_inter_ssc(rqstp, cstate, copy, &src, &dst);
+		if (IS_ERR(ss_mnt)) {
+			status = nfserrno(PTR_ERR(ss_mnt));
+			goto out;

Touched on before, I think: this could use some checking to make sure that the error returned is useful to the client. (I'm not sure exactly what the most likely failures are, though.)

--b.

Quoted 41 lines
fs/nfsd/nfs4proc.c +9 -3
+		}
+	} else {
+		status = nfsd4_setup_intra_ssc(rqstp, cstate, copy, &src, &dst);
+		if (status)
+			goto out;
 	}
 
 	bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
@@ -1106,9 +1347,10 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		status = nfs_ok;
 	}
 
-out_put:
-	fput(src);
-	fput(dst);
+	if (copy->cp_src.nl_nsvr > 0)   /* Inter server SSC */
+		nfsd4_cleanup_inter_ssc(ss_mnt, src, dst);
+	else
+		nfsd4_cleanup_intra_ssc(src, dst);
 out:
 	return status;
 }
include/linux/nfs4.h +1 -0
index 5aa36ac..843443b 100644
@@ -16,6 +16,7 @@
 #include <linux/uidgid.h>
 #include <uapi/linux/nfs4.h>
 #include <linux/sunrpc/msg_prot.h>
+#include <linux/nfs.h>
 
 enum nfs4_acl_whotype {
 	NFS4_ACL_WHO_NAMED = 0,
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#65 lore raw

RFC v3 29/42 NFSD return nfs4_stid in nfs4_preprocess_stateid_op

In reply to Olga Kornievskaia

Needed for nfsd4_copy_notify to add nfs4_cp_state to the nfs4_stid.

---
 fs/nfsd/nfs4proc.c  | 22 +++++++++++++---------
 fs/nfsd/nfs4state.c |  8 ++++++--
 fs/nfsd/state.h     |  3 ++-
 3 files changed, 21 insertions(+), 12 deletions(-)
fs/nfsd/nfs4proc.c +13 -9
index b1095e9..c6edf01 100644
@@ -780,7 +780,8 @@ static __be32 nfsd4_do_lookupp(struct svc_rqst *rqstp, struct svc_fh *fh)
 	/* check stateid */
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&read->rd_stateid, RD_STATE,
-					&read->rd_filp, &read->rd_tmp_file);
+					&read->rd_filp, &read->rd_tmp_file,
+					NULL);
 	if (status) {
 		dprintk("NFSD: nfsd4_read: couldn't process stateid!\n");
 		goto out;
@@ -926,7 +927,7 @@ static __be32 nfsd4_do_lookupp(struct svc_rqst *rqstp, struct svc_fh *fh)
 	if (setattr->sa_iattr.ia_valid & ATTR_SIZE) {
 		status = nfs4_preprocess_stateid_op(rqstp, cstate,
 				&cstate->current_fh, &setattr->sa_stateid,
-				WR_STATE, NULL, NULL);
+				WR_STATE, NULL, NULL, NULL);
 		if (status) {
 			dprintk("NFSD: nfsd4_setattr: couldn't process stateid!\n");
 			return status;
@@ -991,7 +992,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		return nfserr_inval;
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
-						stateid, WR_STATE, &filp, NULL);
+					stateid, WR_STATE, &filp, NULL, NULL);
 	if (status) {
 		dprintk("NFSD: nfsd4_write: couldn't process stateid!\n");
 		return status;
@@ -1022,14 +1023,16 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	__be32 status;
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->save_fh,
-					    src_stateid, RD_STATE, src, NULL);
+					    src_stateid, RD_STATE, src, NULL,
+					    NULL);
 	if (status) {
 		dprintk("NFSD: %s: couldn't process src stateid!\n", __func__);
 		goto out;
 	}
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
-					    dst_stateid, WR_STATE, dst, NULL);
+					    dst_stateid, WR_STATE, dst, NULL,
+					    NULL);
 	if (status) {
 		dprintk("NFSD: %s: couldn't process dst stateid!\n", __func__);
 		goto out_put_src;
@@ -1221,7 +1224,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	/* Verify the destination stateid and set dst struct file*/
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&copy->cp_dst_stateid,
-					WR_STATE, dst, NULL);
+					WR_STATE, dst, NULL, NULL);
 	if (status) {
 		ss_mnt = ERR_PTR(be32_to_cpu(status));
 		goto out;
@@ -1405,10 +1408,11 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 {
 	__be32 status;
 	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+	struct nfs4_stid *stid;
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&cn->cpn_src_stateid, RD_STATE, NULL,
-					NULL);
+					NULL, &stid);
 	if (status)
 		return status;
 
@@ -1440,7 +1444,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					    &fallocate->falloc_stateid,
-					    WR_STATE, &file, NULL);
+					    WR_STATE, &file, NULL, NULL);
 	if (status != nfs_ok) {
 		dprintk("NFSD: nfsd4_fallocate: couldn't process stateid!\n");
 		return status;
@@ -1479,7 +1483,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					    &seek->seek_stateid,
-					    RD_STATE, &file, NULL);
+					    RD_STATE, &file, NULL, NULL);
 	if (status) {
 		dprintk("NFSD: nfsd4_seek: couldn't process stateid!\n");
 		return status;
fs/nfsd/nfs4state.c +6 -2
index e7e1b9f..8425735 100644
@@ -4932,7 +4932,8 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
 __be32
 nfs4_preprocess_stateid_op(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *cstate, struct svc_fh *fhp,
-		stateid_t *stateid, int flags, struct file **filpp, bool *tmp_file)
+		stateid_t *stateid, int flags, struct file **filpp,
+		bool *tmp_file, struct nfs4_stid **cstid)
 {
 	struct inode *ino = d_inode(fhp->fh_dentry);
 	struct net *net = SVC_NET(rqstp);
@@ -4983,8 +4984,11 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
 	if (!status && filpp)
 		status = nfs4_check_file(rqstp, fhp, s, filpp, tmp_file, flags);
 out:
-	if (s)
+	if (s) {
+		if (!status && cstid)
+			*cstid = s;
 		nfs4_put_stid(s);
+	}
 	return status;
 }
 
fs/nfsd/state.h +2 -1
index 005c911..f5ab89a 100644
@@ -599,7 +599,8 @@ struct nfsd4_blocked_lock {
 
 extern __be32 nfs4_preprocess_stateid_op(struct svc_rqst *rqstp,
 		struct nfsd4_compound_state *cstate, struct svc_fh *fhp,
-		stateid_t *stateid, int flags, struct file **filp, bool *tmp_file);
+		stateid_t *stateid, int flags, struct file **filp,
+		bool *tmp_file, struct nfs4_stid **cstid);
 __be32 nfsd4_lookup_stateid(struct nfsd4_compound_state *cstate,
 		     stateid_t *stateid, unsigned char typemask,
 		     struct nfs4_stid **s, struct nfsd_net *nn);
Signature · 1 line
1.8.3.1
#66 lore raw

RFC v3 30/42 NFSD Unique stateid_t for inter server to server COPY authentication

In reply to Olga Kornievskaia

A per COPY unique stateid_t is required to authenticate the READ from the destination server (acting as a client) on the source server.

Multiple concurrent inter server to server copies of the same source file are also supported.

---
 fs/nfsd/netns.h     |   8 ++++
 fs/nfsd/nfs4proc.c  |  25 ++++++++----
 fs/nfsd/nfs4state.c | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/nfs4xdr.c   |   2 +-
 fs/nfsd/nfsctl.c    |   2 +
 fs/nfsd/state.h     |  18 +++++++++
 fs/nfsd/xdr4.h      |   2 +-
 7 files changed, 156 insertions(+), 9 deletions(-)
fs/nfsd/netns.h +8 -0
index 3714231..2c88a95 100644
@@ -119,6 +119,14 @@ struct nfsd_net {
 	u32 clverifier_counter;
 
 	struct svc_serv *nfsd_serv;
+
+	/*
+	 * clientid and stateid data for construction of net unique COPY
+	 * stateids.
+	 */
+	u32		s2s_cp_cl_id;
+	struct idr	s2s_cp_stateids;
+	spinlock_t	s2s_cp_lock;
 };
 
 /* Simple check to find out if a given net was properly initialized */
fs/nfsd/nfs4proc.c +18 -7
index c6edf01..3d8ed3f 100644
@@ -1402,6 +1402,13 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	return 0;
 }
 
+/*
+ * Use a unique stateid_t as the cnr_stateid so that the source server
+ * can authenticate the inter server to server copy READ from the
+ * destination server.
+ *
+ * Set the cnr_leasetime to the nfsd4_lease.
+ */
 static __be32
 nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		  struct nfsd4_copy_notify *cn)
@@ -1409,6 +1416,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	__be32 status;
 	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
 	struct nfs4_stid *stid;
+	struct nfs4_cp_state *cps;
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&cn->cpn_src_stateid, RD_STATE, NULL,
@@ -1419,12 +1427,11 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	cn->cpn_sec = nn->nfsd4_lease;
 	cn->cpn_nsec = 0;
 
-
-	/** XXX Save cpn_src_statid, cpn_src, and any other returned source
-	 * server addresses on which the source server is williing to accept
-	 * connections from the destination e.g. what is returned in cpn_src,
-	 * to verify READ from dest server.
-	 */
+	status = nfserrno(-ENOMEM);
+	cps = nfs4_alloc_init_cp_state(nn, nn->nfsd4_lease, stid);
+	if (!cps)
+		return status;
+	memcpy(&cn->cpn_cnr_stateid, &cps->cp_stateid, sizeof(stateid_t));
 
 	/**
 	 * For now, only return one server address in cpn_src, the
@@ -1432,7 +1439,11 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	 */
 	cn->cpn_src.nl_nsvr = 1;
 
-	return nfsd4_set_src_nl4_netaddr(rqstp, &cn->cpn_src);
+	status = nfsd4_set_src_nl4_netaddr(rqstp, &cn->cpn_src);
+	if (status != 0)
+		nfs4_free_cp_state(cps);
+
+	return status;
 }
 
 static __be32
fs/nfsd/nfs4state.c +108 -0

138 lines not on this page · raw message

fs/nfsd/nfs4xdr.c +1 -1
index 2896a11..c4438ac 100644
@@ -4458,7 +4458,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 	*p++ = cpu_to_be32(cn->cpn_nsec);
 
 	/* cnr_stateid */
-	nfserr = nfsd4_encode_stateid(xdr, &cn->cpn_src_stateid);
+	nfserr = nfsd4_encode_stateid(xdr, &cn->cpn_cnr_stateid);
 	if (nfserr)
 		return nfserr;
 
fs/nfsd/nfsctl.c +2 -0
index 6493df6..6ef5790 100644
@@ -1241,6 +1241,8 @@ static __net_init int nfsd_init_net(struct net *net)
 	nn->nfsd4_grace = 90;
 	nn->clverifier_counter = prandom_u32();
 	nn->clientid_counter = prandom_u32();
+	nn->s2s_cp_cl_id = nn->clientid_counter++;
+	pr_info("%s s2s_cp_cl_id %d\n", __func__, nn->s2s_cp_cl_id);
 	return 0;
 
 out_idmap_error:
fs/nfsd/state.h +18 -0
index f5ab89a..c084c57 100644
@@ -93,6 +93,7 @@ struct nfs4_stid {
 #define NFS4_REVOKED_DELEG_STID 16
 #define NFS4_CLOSED_DELEG_STID 32
 #define NFS4_LAYOUT_STID 64
+	struct list_head	sc_cp_list;
 	unsigned char		sc_type;
 	stateid_t		sc_stateid;
 	spinlock_t		sc_lock;
@@ -102,6 +103,20 @@ struct nfs4_stid {
 };
 
 /*
+ * An inter server to server copy stateid that is unique per nfsd_net.
+ * cp_stateid is returned as the COPY_NOTIFY cnr_stateid and presented
+ * by the destination server to the source server as a COPY authenticator.
+ * Used to lookup the parent COPY_NOTIFY cna_src_stateid nfs4_stid.
+ */
+struct nfs4_cp_state {
+	stateid_t		cp_stateid;
+	struct list_head	cp_list;	/* per parent nfs4_stid */
+	struct nfs4_stid	*cp_p_stid;	/* pointer to parent */
+	bool			cp_active;	/* has the copy started */
+	unsigned long		cp_timeout;	/* copy timeout */
+};
+
+/*
  * Represents a delegation stateid. The nfs4_client holds references to these
  * and they are put when it is being destroyed or when the delegation is
  * returned by the client:
@@ -606,6 +621,9 @@ __be32 nfsd4_lookup_stateid(struct nfsd4_compound_state *cstate,
 		     struct nfs4_stid **s, struct nfsd_net *nn);
 struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl, struct kmem_cache *slab,
 				  void (*sc_free)(struct nfs4_stid *));
+struct nfs4_cp_state *nfs4_alloc_init_cp_state(struct nfsd_net *nn, u64 cpn_sec,
+					       struct nfs4_stid *p_stid);
+void nfs4_free_cp_state(struct nfs4_cp_state *cps);
 void nfs4_unhash_stid(struct nfs4_stid *s);
 void nfs4_put_stid(struct nfs4_stid *s);
 void nfs4_inc_and_copy_stateid(stateid_t *dst, struct nfs4_stid *stid);
fs/nfsd/xdr4.h +1 -1
index aa94295..5b38f0a 100644
@@ -549,7 +549,7 @@ struct nfsd4_copy_notify {
 	struct nl4_server	cpn_dst;
 
 	/* response */
-	/* Note: cpn_src_stateid is used for cnr_stateid */
+	stateid_t		cpn_cnr_stateid;
 	u64			cpn_sec;
 	u32			cpn_nsec;
 	struct nl4_servers	cpn_src;
Signature · 1 line
1.8.3.1
#67 lore raw

RFC v3 31/42 NFSD CB_OFFLOAD xdr

In reply to Olga Kornievskaia
---
 fs/nfsd/nfs4callback.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfsd/state.h        |  1 +
 fs/nfsd/xdr4.h         |  6 ++++
 fs/nfsd/xdr4cb.h       | 10 ++++++
 4 files changed, 112 insertions(+)
fs/nfsd/nfs4callback.c +95 -0
index 0274db6..d36fc05 100644
@@ -39,6 +39,7 @@
 #include "state.h"
 #include "netns.h"
 #include "xdr4cb.h"
+#include "xdr4.h"
 
 #define NFSDDBG_FACILITY                NFSDDBG_PROC
 
@@ -105,6 +106,7 @@ enum nfs_cb_opnum4 {
 	OP_CB_WANTS_CANCELLED		= 12,
 	OP_CB_NOTIFY_LOCK		= 13,
 	OP_CB_NOTIFY_DEVICEID		= 14,
+	OP_CB_OFFLOAD			= 15,
 	OP_CB_ILLEGAL			= 10044
 };
 
@@ -677,6 +679,98 @@ static int nfs4_xdr_dec_cb_notify_lock(struct rpc_rqst *rqstp,
 }
 
 /*
+ * struct write_response4 {
+ *	stateid4	wr_callback_id<1>;
+ *	length4		wr_count;
+ *	stable_how4	wr_committed;
+ *	verifier4	wr_writeverf;
+ * };
+ * union offload_info4 switch (nfsstat4 coa_status) {
+ *	case NFS4_OK:
+ *		write_response4	coa_resok4;
+ *	default:
+ *	length4		coa_bytes_copied;
+ * };
+ * struct CB_OFFLOAD4args {
+ *	nfs_fh4		coa_fh;
+ *	stateid4	coa_stateid;
+ *	offload_info4	coa_offload_info;
+ * };
+ */
+static void encode_offload_info4(struct xdr_stream *xdr,
+				 __be32 nfserr,
+				 const struct nfsd4_copy *cp)
+{
+	__be32 *p;
+
+	p = xdr_reserve_space(xdr, 4);
+	*p++ = nfserr;
+	if (!nfserr) {
+		p = xdr_reserve_space(xdr, 4 + 8 + 4 + NFS4_VERIFIER_SIZE);
+		p = xdr_encode_empty_array(p);
+		p = xdr_encode_hyper(p, cp->cp_res.wr_bytes_written);
+		*p++ = cpu_to_be32(cp->cp_res.wr_stable_how);
+		p = xdr_encode_opaque_fixed(p, cp->cp_res.wr_verifier.data,
+					    NFS4_VERIFIER_SIZE);
+	} else {
+		p = xdr_reserve_space(xdr, 8);
+		p = xdr_encode_hyper(p, cp->cp_res.wr_bytes_written);
+	}
+}
+
+static void encode_cb_offload4args(struct xdr_stream *xdr,
+				   __be32 nfserr,
+				   const struct knfsd_fh *fh,
+				   const struct nfsd4_copy *cp,
+				   struct nfs4_cb_compound_hdr *hdr)
+{
+	__be32 *p;
+
+	p = xdr_reserve_space(xdr, 4);
+	*p++ = cpu_to_be32(OP_CB_OFFLOAD);
+	encode_nfs_fh4(xdr, fh);
+	encode_stateid4(xdr, &cp->cp_res.cb_stateid);
+	encode_offload_info4(xdr, nfserr, cp);
+
+	hdr->nops++;
+}
+
+static void nfs4_xdr_enc_cb_offload(struct rpc_rqst *req,
+				    struct xdr_stream *xdr,
+				    const struct nfsd4_callback *cb)
+{
+	const struct nfsd4_copy *cp =
+		container_of(cb, struct nfsd4_copy, cp_cb);
+	struct nfs4_cb_compound_hdr hdr = {
+		.ident = 0,
+		.minorversion = cb->cb_clp->cl_minorversion,
+	};
+
+	encode_cb_compound4args(xdr, &hdr);
+	encode_cb_sequence4args(xdr, cb, &hdr);
+	encode_cb_offload4args(xdr, cp->nfserr, &cp->fh, cp, &hdr);
+	encode_cb_nops(&hdr);
+}
+
+static int nfs4_xdr_dec_cb_offload(struct rpc_rqst *rqstp,
+				   struct xdr_stream *xdr,
+				   struct nfsd4_callback *cb)
+{
+	struct nfs4_cb_compound_hdr hdr;
+	int status;
+
+	status = decode_cb_compound4res(xdr, &hdr);
+	if (unlikely(status))
+		return status;
+
+	if (cb) {
+		status = decode_cb_sequence4res(xdr, cb);
+		if (unlikely(status || cb->cb_seq_status))
+			return status;
+	}
+	return decode_cb_op_status(xdr, OP_CB_OFFLOAD, &cb->cb_status);
+}
+/*
  * RPC procedure tables
  */
 #define PROC(proc, call, argtype, restype)				\
@@ -697,6 +791,7 @@ static int nfs4_xdr_dec_cb_notify_lock(struct rpc_rqst *rqstp,
 	PROC(CB_LAYOUT,	COMPOUND,	cb_layout,	cb_layout),
 #endif
 	PROC(CB_NOTIFY_LOCK,	COMPOUND,	cb_notify_lock,	cb_notify_lock),
+	PROC(CB_OFFLOAD,	COMPOUND,	cb_offload,	cb_offload),
 };
 
 static struct rpc_version nfs_cb_version4 = {
fs/nfsd/state.h +1 -0
index c084c57..3b0da32 100644
@@ -585,6 +585,7 @@ enum nfsd4_cb_op {
 	NFSPROC4_CLNT_CB_NULL = 0,
 	NFSPROC4_CLNT_CB_RECALL,
 	NFSPROC4_CLNT_CB_LAYOUT,
+	NFSPROC4_CLNT_CB_OFFLOAD,
 	NFSPROC4_CLNT_CB_SEQUENCE,
 	NFSPROC4_CLNT_CB_NOTIFY_LOCK,
 };
fs/nfsd/xdr4.h +6 -0
index 5b38f0a..e6fa914 100644
@@ -510,6 +510,7 @@ struct nfsd42_write_res {
 	u64			wr_bytes_written;
 	u32			wr_stable_how;
 	nfs4_verifier		wr_verifier;
+	stateid_t		cb_stateid;
 };
 
 /*  support 1 source server for now */
@@ -530,6 +531,11 @@ struct nfsd4_copy {
 
 	/* response */
 	struct nfsd42_write_res	cp_res;
+
+	/* for cb_offload */
+	struct nfsd4_callback	cp_cb;
+	__be32			nfserr;
+	struct knfsd_fh		fh;
 };
 
 struct nfsd4_seek {
fs/nfsd/xdr4cb.h +10 -0
index 49b719d..7e39913 100644
@@ -37,3 +37,13 @@
 #define NFS4_dec_cb_notify_lock_sz	(cb_compound_dec_hdr_sz  +      \
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
+#define enc_cb_offload_info_sz		(1 + 1 + 2 + 1 +		\
+					XDR_QUADLEN(NFS4_VERIFIER_SIZE))
+#define NFS4_enc_cb_offload_sz		(cb_compound_enc_hdr_sz +       \
+					cb_sequence_enc_sz +            \
+					enc_nfs4_fh_sz +		\
+					enc_stateid_sz +		\
+					enc_cb_offload_info_sz)
+#define NFS4_dec_cb_offload_sz		(cb_compound_dec_hdr_sz  +      \
+					cb_sequence_dec_sz +            \
+					op_dec_sz)
Signature · 1 line
1.8.3.1
#68 lore raw

RFC v3 32/42 NFSD OFFLOAD_STATUS xdr

In reply to Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c | 20 ++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 30 ++++++++++++++++++++++++++++--
 fs/nfsd/xdr4.h     | 10 ++++++++++
 3 files changed, 58 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +20 -0
index 3d8ed3f..6d20ee1 100644
@@ -1468,6 +1468,13 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	fput(file);
 	return status;
 }
+static __be32
+nfsd4_offload_status(struct svc_rqst *rqstp,
+		     struct nfsd4_compound_state *cstate,
+		     struct nfsd4_offload_status *os)
+{
+	return nfserr_notsupp;
+}
 
 static __be32
 nfsd4_allocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
@@ -2436,6 +2443,14 @@ static inline u32 nfsd4_copy_notify_rsize(struct svc_rqst *rqstp,
 		* sizeof(__be32);
 }
 
+static inline u32 nfsd4_offload_status_rsize(struct svc_rqst *rqstp,
+					     struct nfsd4_op *op)
+{
+	return (op_encode_hdr_size +
+		2 /* osr_count */ +
+		1 /* osr_complete<1> optional 0 for now */) * sizeof(__be32);
+}
+
 #ifdef CONFIG_NFSD_PNFS
 static inline u32 nfsd4_getdeviceinfo_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 {
@@ -2846,6 +2861,11 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		.op_name = "OP_COPY_NOTIFY",
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize,
 	},
+	[OP_OFFLOAD_STATUS] = {
+		.op_func = (nfsd4op_func)nfsd4_offload_status,
+		.op_name = "OP_OFFLOAD_STATUS",
+		.op_rsize_bop = (nfsd4op_rsize)nfsd4_offload_status_rsize,
+	},
 };
 
 /**
fs/nfsd/nfs4xdr.c +28 -2
index c4438ac..f032178 100644
@@ -1836,6 +1836,13 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 }
 
 static __be32
+nfsd4_decode_offload_status(struct nfsd4_compoundargs *argp,
+			    struct nfsd4_offload_status *os)
+{
+	return nfsd4_decode_stateid(argp, &os->stateid);
+}
+
+static __be32
 nfsd4_decode_seek(struct nfsd4_compoundargs *argp, struct nfsd4_seek *seek)
 {
 	DECODE_HEAD;
@@ -1942,7 +1949,7 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 	[OP_LAYOUTERROR]	= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_LAYOUTSTATS]	= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_OFFLOAD_CANCEL]	= (nfsd4_dec)nfsd4_decode_notsupp,
-	[OP_OFFLOAD_STATUS]	= (nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OFFLOAD_STATUS]	= (nfsd4_dec)nfsd4_decode_offload_status,
 	[OP_READ_PLUS]		= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_SEEK]		= (nfsd4_dec)nfsd4_decode_seek,
 	[OP_WRITE_SAME]		= (nfsd4_dec)nfsd4_decode_notsupp,
@@ -4483,6 +4490,25 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd4_encode_offload_status(struct nfsd4_compoundres *resp, __be32 nfserr,
+			    struct nfsd4_offload_status *os)
+{
+	struct xdr_stream *xdr = &resp->xdr;
+	__be32 *p;
+
+	if (nfserr)
+		return nfserr;
+
+	p = xdr_reserve_space(xdr, 8 + 4);
+	if (!p)
+		return nfserr_resource;
+	p = xdr_encode_hyper(p, os->count);
+	*p++ = cpu_to_be32(0);
+
+	return nfserr;
+}
+
+static __be32
 nfsd4_encode_seek(struct nfsd4_compoundres *resp, __be32 nfserr,
 		  struct nfsd4_seek *seek)
 {
@@ -4588,7 +4614,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 	[OP_LAYOUTERROR]	= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_LAYOUTSTATS]	= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_OFFLOAD_CANCEL]	= (nfsd4_enc)nfsd4_encode_noop,
-	[OP_OFFLOAD_STATUS]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_OFFLOAD_STATUS]	= (nfsd4_enc)nfsd4_encode_offload_status,
 	[OP_READ_PLUS]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_SEEK]		= (nfsd4_enc)nfsd4_encode_seek,
 	[OP_WRITE_SAME]		= (nfsd4_enc)nfsd4_encode_noop,
fs/nfsd/xdr4.h +10 -0
index e6fa914..c3e6907 100644
@@ -561,6 +561,15 @@ struct nfsd4_copy_notify {
 	struct nl4_servers	cpn_src;
 };
 
+struct nfsd4_offload_status {
+	/* request */
+	stateid_t	stateid;
+
+	/* response */
+	u64		count;
+	u32		status;
+};
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -617,6 +626,7 @@ struct nfsd4_op {
 		struct nfsd4_clone		clone;
 		struct nfsd4_copy		copy;
 		struct nfsd4_copy_notify	copy_notify;
+		struct nfsd4_offload_status	offload_status;
 		struct nfsd4_seek		seek;
 	} u;
 	struct nfs4_replay *			replay;
Signature · 1 line
1.8.3.1
#69 lore raw

Re: [RFC v3 32/42] NFSD OFFLOAD_STATUS xdr

Hi Olga,

Quoted 12 lines · Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c | 20 ++++++++++++++++++++
 fs/nfsd/nfs4xdr.c  | 30 ++++++++++++++++++++++++++++--
 fs/nfsd/xdr4.h     | 10 ++++++++++
 3 files changed, 58 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +0 -0
index 3d8ed3f..6d20ee1 100644

fs/nfsd/nfs4proc.c extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt, · line 1477

Quoted 10 lines
fs/nfsd/nfs4proc.c +7 -0
@@ -1468,6 +1468,13 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	fput(file);
 	return status;
 }
+static __be32
+nfsd4_offload_status(struct svc_rqst *rqstp,
+		     struct nfsd4_compound_state *cstate,
+		     struct nfsd4_offload_status *os)
+{
+	return nfserr_notsupp;
+}

One small comment on the placement of this function: I think it should go above nfsd4_fallocate() to keep fallocate / allocate / deallocate together.

Thanks!
Anna

Quoted 121 lines
fs/nfsd/nfs4proc.c +13 -0
 static __be32
 nfsd4_allocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
@@ -2436,6 +2443,14 @@ static inline u32 nfsd4_copy_notify_rsize(struct svc_rqst *rqstp,
 		* sizeof(__be32);
 }
 
+static inline u32 nfsd4_offload_status_rsize(struct svc_rqst *rqstp,
+					     struct nfsd4_op *op)
+{
+	return (op_encode_hdr_size +
+		2 /* osr_count */ +
+		1 /* osr_complete<1> optional 0 for now */) * sizeof(__be32);
+}
+
 #ifdef CONFIG_NFSD_PNFS
 static inline u32 nfsd4_getdeviceinfo_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 {
@@ -2846,6 +2861,11 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		.op_name = "OP_COPY_NOTIFY",
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize,
 	},
+	[OP_OFFLOAD_STATUS] = {
+		.op_func = (nfsd4op_func)nfsd4_offload_status,
+		.op_name = "OP_OFFLOAD_STATUS",
+		.op_rsize_bop = (nfsd4op_rsize)nfsd4_offload_status_rsize,
+	},
 };
 
 /**
fs/nfsd/nfs4xdr.c +28 -2
index c4438ac..f032178 100644
@@ -1836,6 +1836,13 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 }
 
 static __be32
+nfsd4_decode_offload_status(struct nfsd4_compoundargs *argp,
+			    struct nfsd4_offload_status *os)
+{
+	return nfsd4_decode_stateid(argp, &os->stateid);
+}
+
+static __be32
 nfsd4_decode_seek(struct nfsd4_compoundargs *argp, struct nfsd4_seek *seek)
 {
 	DECODE_HEAD;
@@ -1942,7 +1949,7 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 	[OP_LAYOUTERROR]	= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_LAYOUTSTATS]	= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_OFFLOAD_CANCEL]	= (nfsd4_dec)nfsd4_decode_notsupp,
-	[OP_OFFLOAD_STATUS]	= (nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OFFLOAD_STATUS]	= (nfsd4_dec)nfsd4_decode_offload_status,
 	[OP_READ_PLUS]		= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_SEEK]		= (nfsd4_dec)nfsd4_decode_seek,
 	[OP_WRITE_SAME]		= (nfsd4_dec)nfsd4_decode_notsupp,
@@ -4483,6 +4490,25 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 }
 
 static __be32
+nfsd4_encode_offload_status(struct nfsd4_compoundres *resp, __be32 nfserr,
+			    struct nfsd4_offload_status *os)
+{
+	struct xdr_stream *xdr = &resp->xdr;
+	__be32 *p;
+
+	if (nfserr)
+		return nfserr;
+
+	p = xdr_reserve_space(xdr, 8 + 4);
+	if (!p)
+		return nfserr_resource;
+	p = xdr_encode_hyper(p, os->count);
+	*p++ = cpu_to_be32(0);
+
+	return nfserr;
+}
+
+static __be32
 nfsd4_encode_seek(struct nfsd4_compoundres *resp, __be32 nfserr,
 		  struct nfsd4_seek *seek)
 {
@@ -4588,7 +4614,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 	[OP_LAYOUTERROR]	= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_LAYOUTSTATS]	= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_OFFLOAD_CANCEL]	= (nfsd4_enc)nfsd4_encode_noop,
-	[OP_OFFLOAD_STATUS]	= (nfsd4_enc)nfsd4_encode_noop,
+	[OP_OFFLOAD_STATUS]	= (nfsd4_enc)nfsd4_encode_offload_status,
 	[OP_READ_PLUS]		= (nfsd4_enc)nfsd4_encode_noop,
 	[OP_SEEK]		= (nfsd4_enc)nfsd4_encode_seek,
 	[OP_WRITE_SAME]		= (nfsd4_enc)nfsd4_encode_noop,
fs/nfsd/xdr4.h +10 -0
index e6fa914..c3e6907 100644
@@ -561,6 +561,15 @@ struct nfsd4_copy_notify {
 	struct nl4_servers	cpn_src;
 };
 
+struct nfsd4_offload_status {
+	/* request */
+	stateid_t	stateid;
+
+	/* response */
+	u64		count;
+	u32		status;
+};
+
 struct nfsd4_op {
 	int					opnum;
 	__be32					status;
@@ -617,6 +626,7 @@ struct nfsd4_op {
 		struct nfsd4_clone		clone;
 		struct nfsd4_copy		copy;
 		struct nfsd4_copy_notify	copy_notify;
+		struct nfsd4_offload_status	offload_status;
 		struct nfsd4_seek		seek;
 	} u;
 	struct nfs4_replay *			replay;
#70 lore raw

RFC v3 33/42 NFSD OFFLOAD_CANCEL xdr

In reply to Olga Kornievskaia
---
 fs/nfsd/nfs4proc.c | 13 +++++++++++++
 fs/nfsd/nfs4xdr.c  |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)
fs/nfsd/nfs4proc.c +13 -0
index 6d20ee1..5dd8278 100644
@@ -1447,6 +1447,14 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 }
 
 static __be32
+nfsd4_offload_cancel(struct svc_rqst *rqstp,
+		     struct nfsd4_compound_state *cstate,
+		     struct nfsd4_offload_status *os)
+{
+	return 0;
+}
+
+static __be32
 nfsd4_fallocate(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		struct nfsd4_fallocate *fallocate, int flags)
 {
@@ -2866,6 +2874,11 @@ static inline u32 nfsd4_seek_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op)
 		.op_name = "OP_OFFLOAD_STATUS",
 		.op_rsize_bop = (nfsd4op_rsize)nfsd4_offload_status_rsize,
 	},
+	[OP_OFFLOAD_CANCEL] = {
+		.op_func = (nfsd4op_func)nfsd4_offload_cancel,
+		.op_name = "OP_OFFLOAD_CANCEL",
+		.op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize,
+	},
 };
 
 /**
fs/nfsd/nfs4xdr.c +1 -1
index f032178..745ad17 100644
@@ -1948,7 +1948,7 @@ static __be32 nfsd4_decode_nl4_server(struct nfsd4_compoundargs *argp,
 	[OP_IO_ADVISE]		= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_LAYOUTERROR]	= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_LAYOUTSTATS]	= (nfsd4_dec)nfsd4_decode_notsupp,
-	[OP_OFFLOAD_CANCEL]	= (nfsd4_dec)nfsd4_decode_notsupp,
+	[OP_OFFLOAD_CANCEL]	= (nfsd4_dec)nfsd4_decode_offload_status,
 	[OP_OFFLOAD_STATUS]	= (nfsd4_dec)nfsd4_decode_offload_status,
 	[OP_READ_PLUS]		= (nfsd4_dec)nfsd4_decode_notsupp,
 	[OP_SEEK]		= (nfsd4_dec)nfsd4_decode_seek,
Signature · 1 line
1.8.3.1
#71 lore raw

RFC v3 34/42 NFSD xdr callback stateid in async COPY reply

In reply to Olga Kornievskaia
---
 fs/nfsd/nfs4xdr.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)
fs/nfsd/nfs4xdr.c +17 -4
index 745ad17..3211838 100644
@@ -4361,15 +4361,27 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 #endif /* CONFIG_NFSD_PNFS */
 
 static __be32
-nfsd42_encode_write_res(struct nfsd4_compoundres *resp, struct nfsd42_write_res *write)
+nfsd42_encode_write_res(struct nfsd4_compoundres *resp,
+		struct nfsd42_write_res *write, bool sync)
 {
 	__be32 *p;
+	p = xdr_reserve_space(&resp->xdr, 4);
+	if (!p)
+		return nfserr_resource;
 
-	p = xdr_reserve_space(&resp->xdr, 4 + 8 + 4 + NFS4_VERIFIER_SIZE);
+	if (sync)
+		*p++ = cpu_to_be32(0);
+	else {
+		__be32 nfserr;
+		*p++ = cpu_to_be32(1);
+		nfserr = nfsd4_encode_stateid(&resp->xdr, &write->cb_stateid);
+		if (nfserr)
+			return nfserr;
+	}
+	p = xdr_reserve_space(&resp->xdr, 8 + 4 + NFS4_VERIFIER_SIZE);
 	if (!p)
 		return nfserr_resource;
 
-	*p++ = cpu_to_be32(0);
 	p = xdr_encode_hyper(p, write->wr_bytes_written);
 	*p++ = cpu_to_be32(write->wr_stable_how);
 	p = xdr_encode_opaque_fixed(p, write->wr_verifier.data,
@@ -4430,7 +4442,8 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp,
 	__be32 *p;
 
 	if (!nfserr) {
-		nfserr = nfsd42_encode_write_res(resp, &copy->cp_res);
+		nfserr = nfsd42_encode_write_res(resp, &copy->cp_res,
+				copy->cp_synchronous);
 		if (nfserr)
 			return nfserr;
 
Signature · 1 line
1.8.3.1
#72 lore raw

RFC v3 35/42 NFSD first draft of async copy

In reply to Olga Kornievskaia

Upong receiving copy request for the inter copy, nfsd will establish the mount to the source server.

Asynchronous copies are handled by a single threaded workqueue. If we get asynchronous request, make sure to copy the needed arguments/state from the stack before starting the copy. Then queue work and reply back to the client indicating copy is asynchronous.

nfsd_copy_file_range() will copy in 4MBchunk so do a loop over the total number of bytes need to copy. In case a failure happens in the middle, we can return an error as well as how much we copied so far. Once done creating a workitem for the callback workqueue and send CB_OFFLOAD with the results.

---
 fs/nfsd/nfs4proc.c  | 224 +++++++++++++++++++++++++++++++++++++++++-----------
 fs/nfsd/nfs4state.c |   9 ++-
 fs/nfsd/state.h     |   2 +
 fs/nfsd/xdr4.h      |  11 +++
 4 files changed, 197 insertions(+), 49 deletions(-)
fs/nfsd/nfs4proc.c +176 -48
index 5dd8278..a977142 100644
@@ -77,6 +77,21 @@
 { }
 #endif
 
+static struct workqueue_struct *copy_wq;
+
+int nfsd4_create_copy_queue(void)
+{
+	copy_wq = create_singlethread_workqueue("nfsd4_copy");
+	if (!copy_wq)
+		return -ENOMEM;
+	return 0;
+}
+
+void nfsd4_destroy_copy_queue(void)
+{
+	destroy_workqueue(copy_wq);
+}
+
 #define NFSDDBG_FACILITY		NFSDDBG_PROC
 
 static u32 nfsd_attrmask[] = {
@@ -1210,21 +1225,17 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static struct vfsmount *
 nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
 			struct nfsd4_compound_state *cstate,
-			struct nfsd4_copy *copy, struct file **src,
-			struct file **dst)
+			struct nfsd4_copy *copy)
 {
 	struct svc_fh *s_fh = NULL;
 	stateid_t *s_stid = &copy->cp_src_stateid;
-	struct nfs_fh fh;
-	nfs4_stateid stateid;
-	struct file *filp;
 	struct vfsmount *ss_mnt;
 	__be32 status;
 
 	/* Verify the destination stateid and set dst struct file*/
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&copy->cp_dst_stateid,
-					WR_STATE, dst, NULL, NULL);
+					WR_STATE, &copy->fh_dst, NULL, NULL);
 	if (status) {
 		ss_mnt = ERR_PTR(be32_to_cpu(status));
 		goto out;
@@ -1238,19 +1249,11 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 		goto out;
 
 	s_fh = &cstate->save_fh;
-
-	fh.size = s_fh->fh_handle.fh_size;
-	memcpy(fh.data, &s_fh->fh_handle.fh_base, fh.size);
-	stateid.seqid = s_stid->si_generation;
-	memcpy(stateid.other, (void *)&s_stid->si_opaque,
+	copy->c_fh.size = s_fh->fh_handle.fh_size;
+	memcpy(copy->c_fh.data, &s_fh->fh_handle.fh_base, copy->c_fh.size);
+	copy->stateid.seqid = s_stid->si_generation;
+	memcpy(copy->stateid.other, (void *)&s_stid->si_opaque,
 		sizeof(stateid_opaque_t));
-
-	filp =  nfs42_ssc_open(ss_mnt, &fh, &stateid);
-	if (IS_ERR(filp)) {
-		nfsd4_interssc_disconnect(ss_mnt);
-		return ERR_CAST(filp);
-	}
-	*src = filp;
 out:
 	return ss_mnt;
 }
@@ -1272,8 +1275,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static struct vfsmount *
 nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
 			struct nfsd4_compound_state *cstate,
-			struct nfsd4_copy *copy, struct file **src,
-			struct file **dst)
+			struct nfsd4_copy *copy)
 {
 	return ERR_PTR(-EINVAL);
 }
@@ -1289,13 +1291,13 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static __be32
 nfsd4_setup_intra_ssc(struct svc_rqst *rqstp,
 		      struct nfsd4_compound_state *cstate,
-		      struct nfsd4_copy *copy, struct file **src,
-		      struct file **dst)
+		      struct nfsd4_copy *copy)
 {
 	__be32 status;
 
-	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid, src,
-				   &copy->cp_dst_stateid, dst);
+	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid,
+				   &copy->fh_src, &copy->cp_dst_stateid,
+				   &copy->fh_dst);
 	if (status)
 		goto out;
 
@@ -1315,47 +1317,173 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	fput(dst);
 }
 
+static void nfsd4_cb_offload_release(struct nfsd4_callback *cb)
+{
+	struct nfsd4_copy *copy = container_of(cb, struct nfsd4_copy, cp_cb);
+
+	kfree(copy);
+}
+
+static int nfsd4_cb_offload_done(struct nfsd4_callback *cb,
+				 struct rpc_task *task)
+{
+	return 1;
+}
+
+static const struct nfsd4_callback_ops nfsd4_cb_offload_ops = {
+	.release = nfsd4_cb_offload_release,
+	.done = nfsd4_cb_offload_done
+};
+
+static int nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
+{
+	memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
+		sizeof(copy->cp_dst_stateid));
+	copy->cp_res.wr_stable_how = NFS_UNSTABLE;
+	copy->cp_consecutive = 1;
+	copy->cp_synchronous = sync;
+	gen_boot_verifier(&copy->cp_res.wr_verifier, copy->net);
+
+	return nfs_ok;
+}
+
+static int _nfsd_copy_file_range(struct nfsd4_copy *copy)
+{
+	ssize_t bytes_copied = 0;
+	size_t bytes_total = copy->cp_count;
+	size_t bytes_to_copy;
+	u64 src_pos = copy->cp_src_pos;
+	u64 dst_pos = copy->cp_dst_pos;
+
+	do {
+		bytes_to_copy = min_t(u64, bytes_total, MAX_RW_COUNT);
+		bytes_copied = nfsd_copy_file_range(copy->fh_src, src_pos,
+				copy->fh_dst, dst_pos, bytes_to_copy);
+		if (bytes_copied <= 0)
+			break;
+		bytes_total -= bytes_copied;
+		copy->cp_res.wr_bytes_written += bytes_copied;
+		src_pos += bytes_copied;
+		dst_pos += bytes_copied;
+	} while (bytes_total > 0 && !copy->cp_synchronous);
+	return bytes_copied;
+}
+
+static int nfsd4_do_copy(struct nfsd4_copy *copy, bool sync)
+{
+	__be32 status;
+	ssize_t bytes;
+
+	bytes = _nfsd_copy_file_range(copy);
+	if (bytes < 0)
+		status = nfserrno(bytes);
+	else
+		status = nfsd4_init_copy_res(copy, sync);
+
+	if (copy->cp_src.nl_nsvr > 0)   /* Inter server SSC */
+		nfsd4_cleanup_inter_ssc(copy->ss_mnt, copy->fh_src,
+				copy->fh_dst);
+	else
+		nfsd4_cleanup_intra_ssc(copy->fh_src, copy->fh_dst);
+
+	return status;
+}
+
+static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst)
+{
+	memcpy(&dst->cp_src_stateid, &src->cp_src_stateid, sizeof(stateid_t));
+	memcpy(&dst->cp_dst_stateid, &src->cp_dst_stateid, sizeof(stateid_t));
+	dst->cp_src_pos = src->cp_src_pos;
+	dst->cp_dst_pos = src->cp_dst_pos;
+	dst->cp_count = src->cp_count;
+	memcpy(&dst->cp_src, &src->cp_src, sizeof(struct nl4_servers));
+	dst->cp_consecutive = src->cp_consecutive;
+	dst->cp_synchronous = src->cp_synchronous;
+	memcpy(&dst->cp_res, &src->cp_res, sizeof(src->cp_res));
+	memcpy(&dst->stateid, &src->stateid, sizeof(src->stateid));
+	memcpy(&dst->c_fh, &src->c_fh, sizeof(src->c_fh));
+	/* skipping nfsd4_callback */
+	memcpy(&dst->fh, &src->fh, sizeof(src->fh));
+	dst->fh = src->fh;
+	dst->cp_clp = src->cp_clp;
+	dst->fh_src = src->fh_src;
+	dst->fh_dst = src->fh_dst;
+	dst->ss_mnt = src->ss_mnt;
+	dst->net = src->net;
+}
+
+static void nfsd4_do_async_copy(struct work_struct *work)
+{
+	struct nfsd4_copy *copy =
+		container_of(work, struct nfsd4_copy, cp_work);
+	struct nfsd4_copy *cb_copy;
+
+	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
+		copy->fh_src = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,
+						&copy->stateid);
+		if (IS_ERR(copy->fh_src)) {
+			nfsd4_interssc_disconnect(copy->ss_mnt);
+			return;
+		}
+	}
+	copy->nfserr = nfsd4_do_copy(copy, 0);
+	cb_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
+	if (!cb_copy)
+		goto out;
+	memcpy(&cb_copy->cp_res, &copy->cp_res, sizeof(copy->cp_res));
+	cb_copy->cp_clp = copy->cp_clp;
+	cb_copy->nfserr = copy->nfserr;
+	memcpy(&cb_copy->fh, &copy->fh, sizeof(copy->fh));
+	nfsd4_init_cb(&cb_copy->cp_cb, cb_copy->cp_clp,
+			&nfsd4_cb_offload_ops, NFSPROC4_CLNT_CB_OFFLOAD);
+	nfsd4_run_cb(&cb_copy->cp_cb);
+out:
+	kfree(copy);
+}
+
 static __be32
 nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	struct nfsd4_copy *copy)
 {
-	struct vfsmount *ss_mnt = NULL;
-	struct file *src, *dst;
 	__be32 status;
-	ssize_t bytes;
 
 	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
-		ss_mnt = nfsd4_setup_inter_ssc(rqstp, cstate, copy, &src, &dst);
-		if (IS_ERR(ss_mnt)) {
-			status = nfserrno(PTR_ERR(ss_mnt));
+		copy->ss_mnt = nfsd4_setup_inter_ssc(rqstp, cstate, copy);
+		if (IS_ERR(copy->ss_mnt)) {
+			status = nfserrno(PTR_ERR(copy->ss_mnt));
 			goto out;
 		}
 	} else {
-		status = nfsd4_setup_intra_ssc(rqstp, cstate, copy, &src, &dst);
+		status = nfsd4_setup_intra_ssc(rqstp, cstate, copy);
 		if (status)
 			goto out;
 	}
 
-	bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
-			dst, copy->cp_dst_pos, copy->cp_count);
-
-	if (bytes < 0)
-		status = nfserrno(bytes);
-	else {
-		copy->cp_res.wr_bytes_written = bytes;
-		copy->cp_res.wr_stable_how = NFS_UNSTABLE;
-		copy->cp_consecutive = 1;
-		copy->cp_synchronous = 1;
-		gen_boot_verifier(&copy->cp_res.wr_verifier, SVC_NET(rqstp));
-		status = nfs_ok;
+	copy->cp_clp = cstate->clp;
+	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
+		sizeof(struct knfsd_fh));
+	copy->net = SVC_NET(rqstp);
+	if (!copy->cp_synchronous) {
+		struct nfsd4_copy *async_copy;
+
+		status = nfsd4_init_copy_res(copy, 0);
+		async_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
+		if (!async_copy)
+			goto out_err;
+		dup_copy_fields(copy, async_copy);
+		memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
+			sizeof(copy->cp_dst_stateid));
+		INIT_WORK(&async_copy->cp_work, nfsd4_do_async_copy);
+		queue_work(copy_wq, &async_copy->cp_work);
+	} else {
+		status = nfsd4_do_copy(copy, 1);
 	}
-
-	if (copy->cp_src.nl_nsvr > 0)   /* Inter server SSC */
-		nfsd4_cleanup_inter_ssc(ss_mnt, src, dst);
-	else
-		nfsd4_cleanup_intra_ssc(src, dst);
 out:
 	return status;
+out_err:
+	nfsd4_interssc_disconnect(copy->ss_mnt);
+	status = nfserrno(-ENOMEM);
+	goto out;
 }
 
 static __be32
fs/nfsd/nfs4state.c +8 -1
index b019fc8..d391ce5 100644
@@ -7123,8 +7123,14 @@ static int nfs4_state_create_net(struct net *net)
 		goto out_free_laundry;
 
 	set_max_delegations();
-	return 0;
 
+	ret = nfsd4_create_copy_queue();
+	if (ret)
+		goto out_free_callback;
+
+	return 0;
+out_free_callback:
+	nfsd4_destroy_callback_queue();
 out_free_laundry:
 	destroy_workqueue(laundry_wq);
 out_cleanup_cred:
@@ -7187,6 +7193,7 @@ static int nfs4_state_create_net(struct net *net)
 	destroy_workqueue(laundry_wq);
 	nfsd4_destroy_callback_queue();
 	cleanup_callback_cred();
+	nfsd4_destroy_copy_queue();
 }
 
 static void
fs/nfsd/state.h +2 -0
index 3b0da32..2acea23 100644
@@ -649,6 +649,8 @@ extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,
 extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 							struct nfsd_net *nn);
 extern bool nfs4_has_reclaimed_state(const char *name, struct nfsd_net *nn);
+extern int nfsd4_create_copy_queue(void);
+extern void nfsd4_destroy_copy_queue(void);
 
 struct nfs4_file *find_file(struct knfsd_fh *fh);
 void put_nfs4_file(struct nfs4_file *fi);
fs/nfsd/xdr4.h +11 -0
index c3e6907..d75e530 100644
@@ -532,10 +532,21 @@ struct nfsd4_copy {
 	/* response */
 	struct nfsd42_write_res	cp_res;
 
+	struct nfs_fh           c_fh;
+	nfs4_stateid            stateid;
+
 	/* for cb_offload */
 	struct nfsd4_callback	cp_cb;
 	__be32			nfserr;
 	struct knfsd_fh		fh;
+
+	struct work_struct	cp_work;
+	struct nfs4_client	*cp_clp;
+
+	struct file		*fh_src;
+	struct file		*fh_dst;
+	struct vfsmount		*ss_mnt;
+	struct net		*net;
 };
 
 struct nfsd4_seek {
Signature · 1 line
1.8.3.1
#73 lore raw

Re: [RFC v3 35/42] NFSD first draft of async copy

Hi Olga,

Quoted 9 lines · Olga Kornievskaia

Upong receiving copy request for the inter copy, nfsd will establish the mount to the source server.

Asynchronous copies are handled by a single threaded workqueue. If we get asynchronous request, make sure to copy the needed arguments/state from the stack before starting the copy. Then queue work and reply back to the client indicating copy is asynchronous.

Quoted 5 lines

nfsd_copy_file_range() will copy in 4MBchunk so do a loop over the total number of bytes need to copy. In case a failure happens in the middle, we can return an error as well as how much we copied so far. Once done creating a workitem for the callback workqueue and send CB_OFFLOAD with the results.

This doesn't compile if I set CONFIG_NFSD_V4_2_INTER_SSC=n:

fs/nfsd/nfs4proc.c: In function 'nfsd4_do_async_copy':
fs/nfsd/nfs4proc.c:1439:18: error: implicit declaration of function 'nfs42_ssc_open'; did you mean 'nfsd4_open'? [-Werror=implicit-function-declaration]
copy->fh_src = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,

                  ^~~~~~~~~~~~~~                                 
                  nfsd4_open    
fs/nfsd/nfs4proc.c:1439:16: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]                  
   copy->fh_src = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,      
                ^               
fs/nfsd/nfs4proc.c:1442:4: error: implicit declaration of function 'nfsd4_interssc_disconnect'; did you mean 'nfsd4_insert_layout'? [-Werror=implicit-function-declaration]
    nfsd4_interssc_disconnect(copy->ss_mnt);                     
    ^~~~~~~~~~~~~~~~~~~~~~~~~   
    nfsd4_insert_layout    

Thanks,
Anna

Quoted 377 lines
---
 fs/nfsd/nfs4proc.c  | 224 +++++++++++++++++++++++++++++++++++++++++-----------
 fs/nfsd/nfs4state.c |   9 ++-
 fs/nfsd/state.h     |   2 +
 fs/nfsd/xdr4.h      |  11 +++
 4 files changed, 197 insertions(+), 49 deletions(-)
fs/nfsd/nfs4proc.c +176 -48
index 5dd8278..a977142 100644
@@ -77,6 +77,21 @@
 { }
 #endif
 
+static struct workqueue_struct *copy_wq;
+
+int nfsd4_create_copy_queue(void)
+{
+	copy_wq = create_singlethread_workqueue("nfsd4_copy");
+	if (!copy_wq)
+		return -ENOMEM;
+	return 0;
+}
+
+void nfsd4_destroy_copy_queue(void)
+{
+	destroy_workqueue(copy_wq);
+}
+
 #define NFSDDBG_FACILITY		NFSDDBG_PROC
 
 static u32 nfsd_attrmask[] = {
@@ -1210,21 +1225,17 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static struct vfsmount *
 nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
 			struct nfsd4_compound_state *cstate,
-			struct nfsd4_copy *copy, struct file **src,
-			struct file **dst)
+			struct nfsd4_copy *copy)
 {
 	struct svc_fh *s_fh = NULL;
 	stateid_t *s_stid = &copy->cp_src_stateid;
-	struct nfs_fh fh;
-	nfs4_stateid stateid;
-	struct file *filp;
 	struct vfsmount *ss_mnt;
 	__be32 status;
 
 	/* Verify the destination stateid and set dst struct file*/
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&copy->cp_dst_stateid,
-					WR_STATE, dst, NULL, NULL);
+					WR_STATE, &copy->fh_dst, NULL, NULL);
 	if (status) {
 		ss_mnt = ERR_PTR(be32_to_cpu(status));
 		goto out;
@@ -1238,19 +1249,11 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 		goto out;
 
 	s_fh = &cstate->save_fh;
-
-	fh.size = s_fh->fh_handle.fh_size;
-	memcpy(fh.data, &s_fh->fh_handle.fh_base, fh.size);
-	stateid.seqid = s_stid->si_generation;
-	memcpy(stateid.other, (void *)&s_stid->si_opaque,
+	copy->c_fh.size = s_fh->fh_handle.fh_size;
+	memcpy(copy->c_fh.data, &s_fh->fh_handle.fh_base, copy->c_fh.size);
+	copy->stateid.seqid = s_stid->si_generation;
+	memcpy(copy->stateid.other, (void *)&s_stid->si_opaque,
 		sizeof(stateid_opaque_t));
-
-	filp =  nfs42_ssc_open(ss_mnt, &fh, &stateid);
-	if (IS_ERR(filp)) {
-		nfsd4_interssc_disconnect(ss_mnt);
-		return ERR_CAST(filp);
-	}
-	*src = filp;
 out:
 	return ss_mnt;
 }
@@ -1272,8 +1275,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static struct vfsmount *
 nfsd4_setup_inter_ssc(struct svc_rqst *rqstp,
 			struct nfsd4_compound_state *cstate,
-			struct nfsd4_copy *copy, struct file **src,
-			struct file **dst)
+			struct nfsd4_copy *copy)
 {
 	return ERR_PTR(-EINVAL);
 }
@@ -1289,13 +1291,13 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 static __be32
 nfsd4_setup_intra_ssc(struct svc_rqst *rqstp,
 		      struct nfsd4_compound_state *cstate,
-		      struct nfsd4_copy *copy, struct file **src,
-		      struct file **dst)
+		      struct nfsd4_copy *copy)
 {
 	__be32 status;
 
-	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid, src,
-				   &copy->cp_dst_stateid, dst);
+	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid,
+				   &copy->fh_src, &copy->cp_dst_stateid,
+				   &copy->fh_dst);
 	if (status)
 		goto out;
 
@@ -1315,47 +1317,173 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	fput(dst);
 }
 
+static void nfsd4_cb_offload_release(struct nfsd4_callback *cb)
+{
+	struct nfsd4_copy *copy = container_of(cb, struct nfsd4_copy, cp_cb);
+
+	kfree(copy);
+}
+
+static int nfsd4_cb_offload_done(struct nfsd4_callback *cb,
+				 struct rpc_task *task)
+{
+	return 1;
+}
+
+static const struct nfsd4_callback_ops nfsd4_cb_offload_ops = {
+	.release = nfsd4_cb_offload_release,
+	.done = nfsd4_cb_offload_done
+};
+
+static int nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
+{
+	memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
+		sizeof(copy->cp_dst_stateid));
+	copy->cp_res.wr_stable_how = NFS_UNSTABLE;
+	copy->cp_consecutive = 1;
+	copy->cp_synchronous = sync;
+	gen_boot_verifier(&copy->cp_res.wr_verifier, copy->net);
+
+	return nfs_ok;
+}
+
+static int _nfsd_copy_file_range(struct nfsd4_copy *copy)
+{
+	ssize_t bytes_copied = 0;
+	size_t bytes_total = copy->cp_count;
+	size_t bytes_to_copy;
+	u64 src_pos = copy->cp_src_pos;
+	u64 dst_pos = copy->cp_dst_pos;
+
+	do {
+		bytes_to_copy = min_t(u64, bytes_total, MAX_RW_COUNT);
+		bytes_copied = nfsd_copy_file_range(copy->fh_src, src_pos,
+				copy->fh_dst, dst_pos, bytes_to_copy);
+		if (bytes_copied <= 0)
+			break;
+		bytes_total -= bytes_copied;
+		copy->cp_res.wr_bytes_written += bytes_copied;
+		src_pos += bytes_copied;
+		dst_pos += bytes_copied;
+	} while (bytes_total > 0 && !copy->cp_synchronous);
+	return bytes_copied;
+}
+
+static int nfsd4_do_copy(struct nfsd4_copy *copy, bool sync)
+{
+	__be32 status;
+	ssize_t bytes;
+
+	bytes = _nfsd_copy_file_range(copy);
+	if (bytes < 0)
+		status = nfserrno(bytes);
+	else
+		status = nfsd4_init_copy_res(copy, sync);
+
+	if (copy->cp_src.nl_nsvr > 0)   /* Inter server SSC */
+		nfsd4_cleanup_inter_ssc(copy->ss_mnt, copy->fh_src,
+				copy->fh_dst);
+	else
+		nfsd4_cleanup_intra_ssc(copy->fh_src, copy->fh_dst);
+
+	return status;
+}
+
+static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst)
+{
+	memcpy(&dst->cp_src_stateid, &src->cp_src_stateid, sizeof(stateid_t));
+	memcpy(&dst->cp_dst_stateid, &src->cp_dst_stateid, sizeof(stateid_t));
+	dst->cp_src_pos = src->cp_src_pos;
+	dst->cp_dst_pos = src->cp_dst_pos;
+	dst->cp_count = src->cp_count;
+	memcpy(&dst->cp_src, &src->cp_src, sizeof(struct nl4_servers));
+	dst->cp_consecutive = src->cp_consecutive;
+	dst->cp_synchronous = src->cp_synchronous;
+	memcpy(&dst->cp_res, &src->cp_res, sizeof(src->cp_res));
+	memcpy(&dst->stateid, &src->stateid, sizeof(src->stateid));
+	memcpy(&dst->c_fh, &src->c_fh, sizeof(src->c_fh));
+	/* skipping nfsd4_callback */
+	memcpy(&dst->fh, &src->fh, sizeof(src->fh));
+	dst->fh = src->fh;
+	dst->cp_clp = src->cp_clp;
+	dst->fh_src = src->fh_src;
+	dst->fh_dst = src->fh_dst;
+	dst->ss_mnt = src->ss_mnt;
+	dst->net = src->net;
+}
+
+static void nfsd4_do_async_copy(struct work_struct *work)
+{
+	struct nfsd4_copy *copy =
+		container_of(work, struct nfsd4_copy, cp_work);
+	struct nfsd4_copy *cb_copy;
+
+	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
+		copy->fh_src = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,
+						&copy->stateid);
+		if (IS_ERR(copy->fh_src)) {
+			nfsd4_interssc_disconnect(copy->ss_mnt);
+			return;
+		}
+	}
+	copy->nfserr = nfsd4_do_copy(copy, 0);
+	cb_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
+	if (!cb_copy)
+		goto out;
+	memcpy(&cb_copy->cp_res, &copy->cp_res, sizeof(copy->cp_res));
+	cb_copy->cp_clp = copy->cp_clp;
+	cb_copy->nfserr = copy->nfserr;
+	memcpy(&cb_copy->fh, &copy->fh, sizeof(copy->fh));
+	nfsd4_init_cb(&cb_copy->cp_cb, cb_copy->cp_clp,
+			&nfsd4_cb_offload_ops, NFSPROC4_CLNT_CB_OFFLOAD);
+	nfsd4_run_cb(&cb_copy->cp_cb);
+out:
+	kfree(copy);
+}
+
 static __be32
 nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	struct nfsd4_copy *copy)
 {
-	struct vfsmount *ss_mnt = NULL;
-	struct file *src, *dst;
 	__be32 status;
-	ssize_t bytes;
 
 	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
-		ss_mnt = nfsd4_setup_inter_ssc(rqstp, cstate, copy, &src, &dst);
-		if (IS_ERR(ss_mnt)) {
-			status = nfserrno(PTR_ERR(ss_mnt));
+		copy->ss_mnt = nfsd4_setup_inter_ssc(rqstp, cstate, copy);
+		if (IS_ERR(copy->ss_mnt)) {
+			status = nfserrno(PTR_ERR(copy->ss_mnt));
 			goto out;
 		}
 	} else {
-		status = nfsd4_setup_intra_ssc(rqstp, cstate, copy, &src, &dst);
+		status = nfsd4_setup_intra_ssc(rqstp, cstate, copy);
 		if (status)
 			goto out;
 	}
 
-	bytes = nfsd_copy_file_range(src, copy->cp_src_pos,
-			dst, copy->cp_dst_pos, copy->cp_count);
-
-	if (bytes < 0)
-		status = nfserrno(bytes);
-	else {
-		copy->cp_res.wr_bytes_written = bytes;
-		copy->cp_res.wr_stable_how = NFS_UNSTABLE;
-		copy->cp_consecutive = 1;
-		copy->cp_synchronous = 1;
-		gen_boot_verifier(&copy->cp_res.wr_verifier, SVC_NET(rqstp));
-		status = nfs_ok;
+	copy->cp_clp = cstate->clp;
+	memcpy(&copy->fh, &cstate->current_fh.fh_handle,
+		sizeof(struct knfsd_fh));
+	copy->net = SVC_NET(rqstp);
+	if (!copy->cp_synchronous) {
+		struct nfsd4_copy *async_copy;
+
+		status = nfsd4_init_copy_res(copy, 0);
+		async_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
+		if (!async_copy)
+			goto out_err;
+		dup_copy_fields(copy, async_copy);
+		memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
+			sizeof(copy->cp_dst_stateid));
+		INIT_WORK(&async_copy->cp_work, nfsd4_do_async_copy);
+		queue_work(copy_wq, &async_copy->cp_work);
+	} else {
+		status = nfsd4_do_copy(copy, 1);
 	}
-
-	if (copy->cp_src.nl_nsvr > 0)   /* Inter server SSC */
-		nfsd4_cleanup_inter_ssc(ss_mnt, src, dst);
-	else
-		nfsd4_cleanup_intra_ssc(src, dst);
 out:
 	return status;
+out_err:
+	nfsd4_interssc_disconnect(copy->ss_mnt);
+	status = nfserrno(-ENOMEM);
+	goto out;
 }
 
 static __be32
fs/nfsd/nfs4state.c +8 -1
index b019fc8..d391ce5 100644
@@ -7123,8 +7123,14 @@ static int nfs4_state_create_net(struct net *net)
 		goto out_free_laundry;
 
 	set_max_delegations();
-	return 0;
 
+	ret = nfsd4_create_copy_queue();
+	if (ret)
+		goto out_free_callback;
+
+	return 0;
+out_free_callback:
+	nfsd4_destroy_callback_queue();
 out_free_laundry:
 	destroy_workqueue(laundry_wq);
 out_cleanup_cred:
@@ -7187,6 +7193,7 @@ static int nfs4_state_create_net(struct net *net)
 	destroy_workqueue(laundry_wq);
 	nfsd4_destroy_callback_queue();
 	cleanup_callback_cred();
+	nfsd4_destroy_copy_queue();
 }
 
 static void
fs/nfsd/state.h +2 -0
index 3b0da32..2acea23 100644
@@ -649,6 +649,8 @@ extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,
 extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 							struct nfsd_net *nn);
 extern bool nfs4_has_reclaimed_state(const char *name, struct nfsd_net *nn);
+extern int nfsd4_create_copy_queue(void);
+extern void nfsd4_destroy_copy_queue(void);
 
 struct nfs4_file *find_file(struct knfsd_fh *fh);
 void put_nfs4_file(struct nfs4_file *fi);
fs/nfsd/xdr4.h +11 -0
index c3e6907..d75e530 100644
@@ -532,10 +532,21 @@ struct nfsd4_copy {
 	/* response */
 	struct nfsd42_write_res	cp_res;
 
+	struct nfs_fh           c_fh;
+	nfs4_stateid            stateid;
+
 	/* for cb_offload */
 	struct nfsd4_callback	cp_cb;
 	__be32			nfserr;
 	struct knfsd_fh		fh;
+
+	struct work_struct	cp_work;
+	struct nfs4_client	*cp_clp;
+
+	struct file		*fh_src;
+	struct file		*fh_dst;
+	struct vfsmount		*ss_mnt;
+	struct net		*net;
 };
 
 struct nfsd4_seek {
#74 lore raw

RFC v3 36/42 NFSD handle OFFLOAD_CANCEL op

In reply to Olga Kornievskaia

Upon receiving OFFLOAD_CANCEL search the list of copy stateids, if found remove it and it will lead to following READs failing with ERR_PARTNER_NO_AUTH.

---
 fs/nfsd/nfs4proc.c  | 11 +++++++++++
 fs/nfsd/nfs4state.c | 30 ++++++++++++++++++++----------
 fs/nfsd/state.h     |  2 ++
 3 files changed, 33 insertions(+), 10 deletions(-)
fs/nfsd/nfs4proc.c +11 -0
index a977142..7e7b416 100644
@@ -1579,6 +1579,17 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 		     struct nfsd4_compound_state *cstate,
 		     struct nfsd4_offload_status *os)
 {
+
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+	__be32 status;
+	struct nfs4_cp_state *state = NULL;
+
+	status = find_cp_state(nn, &os->stateid, &state);
+	if (!status) {
+		list_del(&state->cp_list);
+		nfs4_free_cp_state(state);
+	}
+
 	return 0;
 }
 
fs/nfsd/nfs4state.c +20 -10
index d391ce5..bcd87c9 100644
@@ -743,6 +743,22 @@ static void nfs4_free_cp_statelist(struct nfs4_stid *stid)
 	}
 }
 
+__be32 find_cp_state(struct nfsd_net *nn, stateid_t *st,
+			    struct nfs4_cp_state **cps)
+{
+	struct nfs4_cp_state *state = NULL;
+
+	if (st->si_opaque.so_clid.cl_id != nn->s2s_cp_cl_id)
+		return nfserr_bad_stateid;
+	spin_lock(&nn->s2s_cp_lock);
+	state = idr_find(&nn->s2s_cp_stateids, st->si_opaque.so_id);
+	spin_unlock(&nn->s2s_cp_lock);
+	if (!state)
+		return nfserr_bad_stateid;
+	*cps = state;
+	return 0;
+}
+
 /*
  * A READ from an inter server to server COPY will have a
  * copy stateid. Return the parent nfs4_stid.
@@ -750,18 +766,12 @@ static void nfs4_free_cp_statelist(struct nfs4_stid *stid)
 static __be32 find_cp_state_parent(struct nfsd_net *nn, stateid_t *st,
 				   struct nfs4_stid **stid)
 {
+	__be32 status;
 	struct nfs4_cp_state *cps = NULL;
 
-	if (st->si_opaque.so_clid.cl_id != nn->s2s_cp_cl_id)
-		return nfserr_bad_stateid;
-	spin_lock(&nn->s2s_cp_lock);
-	cps = idr_find(&nn->s2s_cp_stateids, st->si_opaque.so_id);
-	spin_unlock(&nn->s2s_cp_lock);
-	if (!cps) {
-		pr_info("NFSD: find_cp_state cl_id %d so_id %d NOT FOUND\n",
-			st->si_opaque.so_clid.cl_id, st->si_opaque.so_id);
-		return nfserr_bad_stateid;
-	}
+	status = find_cp_state(nn, st, &cps);
+	if (status)
+		return status;
 
 	/* Did the inter server to server copy start in time? */
 	if (cps->cp_active == false && !time_after(cps->cp_timeout, jiffies))
fs/nfsd/state.h +2 -0
index 2acea23..ebf968d 100644
@@ -651,6 +651,8 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern bool nfs4_has_reclaimed_state(const char *name, struct nfsd_net *nn);
 extern int nfsd4_create_copy_queue(void);
 extern void nfsd4_destroy_copy_queue(void);
+extern __be32 find_cp_state(struct nfsd_net *nn, stateid_t *st,
+			struct nfs4_cp_state **cps);
 
 struct nfs4_file *find_file(struct knfsd_fh *fh);
 void put_nfs4_file(struct nfs4_file *fi);
Signature · 1 line
1.8.3.1
#75 lore raw

RFC v3 37/42 NFSD stop queued async copies on client shutdown

In reply to Olga Kornievskaia

If client is shutting down and there are still async copies going on, then stop queued async copies.

---
 fs/nfsd/nfs4proc.c  | 21 ++++++++++++++++-----
 fs/nfsd/nfs4state.c |  1 +
 fs/nfsd/state.h     |  2 ++
 3 files changed, 19 insertions(+), 5 deletions(-)
fs/nfsd/nfs4proc.c +16 -5
index 7e7b416..b93713d 100644
@@ -92,6 +92,12 @@ void nfsd4_destroy_copy_queue(void)
 	destroy_workqueue(copy_wq);
 }
 
+void nfsd4_shutdown_copy(struct nfs4_client *clp)
+{
+	set_bit(NFSD4_CLIENT_COPY_KILL, &clp->cl_flags);
+	flush_workqueue(copy_wq);
+}
+
 #define NFSDDBG_FACILITY		NFSDDBG_PROC
 
 static u32 nfsd_attrmask[] = {
@@ -1416,15 +1422,16 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 {
 	struct nfsd4_copy *copy =
 		container_of(work, struct nfsd4_copy, cp_work);
-	struct nfsd4_copy *cb_copy;
+	struct nfsd4_copy *cb_copy = NULL;
+
+	if (test_bit(NFSD4_CLIENT_COPY_KILL, &copy->cp_clp->cl_flags))
+		goto out_err;
 
 	if (copy->cp_src.nl_nsvr > 0) { /* Inter server SSC */
 		copy->fh_src = nfs42_ssc_open(copy->ss_mnt, &copy->c_fh,
 						&copy->stateid);
-		if (IS_ERR(copy->fh_src)) {
-			nfsd4_interssc_disconnect(copy->ss_mnt);
-			return;
-		}
+		if (IS_ERR(copy->fh_src))
+			goto out_err;
 	}
 	copy->nfserr = nfsd4_do_copy(copy, 0);
 	cb_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
@@ -1439,6 +1446,10 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 	nfsd4_run_cb(&cb_copy->cp_cb);
 out:
 	kfree(copy);
+	return;
+out_err:
+	nfsd4_interssc_disconnect(copy->ss_mnt);
+	goto out;
 }
 
 static __be32
fs/nfsd/nfs4state.c +1 -0
index bcd87c9..7727ce53 100644
@@ -1995,6 +1995,7 @@ static __be32 mark_client_expired_locked(struct nfs4_client *clp)
 	}
 	nfsd4_return_all_client_layouts(clp);
 	nfsd4_shutdown_callback(clp);
+	nfsd4_shutdown_copy(clp);
 	if (clp->cl_cb_conn.cb_xprt)
 		svc_xprt_put(clp->cl_cb_conn.cb_xprt);
 	free_client(clp);
fs/nfsd/state.h +2 -0
index ebf968d..fa749763 100644
@@ -336,6 +336,7 @@ struct nfs4_client {
 #define NFSD4_CLIENT_RECLAIM_COMPLETE	(3)	/* reclaim_complete done */
 #define NFSD4_CLIENT_CONFIRMED		(4)	/* client is confirmed */
 #define NFSD4_CLIENT_UPCALL_LOCK	(5)	/* upcall serialization */
+#define NFSD4_CLIENT_COPY_KILL		(6)	/* stop copy workqueue */
 #define NFSD4_CLIENT_CB_FLAG_MASK	(1 << NFSD4_CLIENT_CB_UPDATE | \
 					 1 << NFSD4_CLIENT_CB_KILL)
 	unsigned long		cl_flags;
@@ -645,6 +646,7 @@ extern void nfsd4_init_cb(struct nfsd4_callback *cb, struct nfs4_client *clp,
 extern int nfsd4_create_callback_queue(void);
 extern void nfsd4_destroy_callback_queue(void);
 extern void nfsd4_shutdown_callback(struct nfs4_client *);
+extern void nfsd4_shutdown_copy(struct nfs4_client *clp);
 extern void nfsd4_prepare_cb_recall(struct nfs4_delegation *dp);
 extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 							struct nfsd_net *nn);
Signature · 1 line
1.8.3.1
#76 lore raw

RFC v3 38/42 NFSD create new stateid for async copy

In reply to Olga Kornievskaia

Previously dst copy stateid was used as reply to the asynchronous copy. Instead, generate a new stateid and the destination server will keep a list of the stateids. If it receives a cancel, it can decide to forego sending the CB_OFFLOAD.

---
 fs/nfsd/nfs4proc.c | 56 +++++++++++++++++++++++++++++++++++++++++++-----------
 fs/nfsd/state.h    |  3 +++
 fs/nfsd/xdr4.h     |  2 ++
 3 files changed, 50 insertions(+), 11 deletions(-)
fs/nfsd/nfs4proc.c +45 -11
index b93713d..c411887 100644
@@ -1039,7 +1039,8 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 static __be32
 nfsd4_verify_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 		  stateid_t *src_stateid, struct file **src,
-		  stateid_t *dst_stateid, struct file **dst)
+		  stateid_t *dst_stateid, struct file **dst,
+		  struct nfs4_stid **stid)
 {
 	__be32 status;
 
@@ -1053,7 +1054,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					    dst_stateid, WR_STATE, dst, NULL,
-					    NULL);
+					    stid);
 	if (status) {
 		dprintk("NFSD: %s: couldn't process dst stateid!\n", __func__);
 		goto out_put_src;
@@ -1083,7 +1084,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	__be32 status;
 
 	status = nfsd4_verify_copy(rqstp, cstate, &clone->cl_src_stateid, &src,
-				   &clone->cl_dst_stateid, &dst);
+				   &clone->cl_dst_stateid, &dst, NULL);
 	if (status)
 		goto out;
 
@@ -1241,7 +1242,8 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 	/* Verify the destination stateid and set dst struct file*/
 	status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
 					&copy->cp_dst_stateid,
-					WR_STATE, &copy->fh_dst, NULL, NULL);
+					WR_STATE, &copy->fh_dst, NULL,
+					&copy->stid);
 	if (status) {
 		ss_mnt = ERR_PTR(be32_to_cpu(status));
 		goto out;
@@ -1303,7 +1305,7 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 
 	status = nfsd4_verify_copy(rqstp, cstate, &copy->cp_src_stateid,
 				   &copy->fh_src, &copy->cp_dst_stateid,
-				   &copy->fh_dst);
+				   &copy->fh_dst, &copy->stid);
 	if (status)
 		goto out;
 
@@ -1343,8 +1345,6 @@ static int nfsd4_cb_offload_done(struct nfsd4_callback *cb,
 
 static int nfsd4_init_copy_res(struct nfsd4_copy *copy, bool sync)
 {
-	memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
-		sizeof(copy->cp_dst_stateid));
 	copy->cp_res.wr_stable_how = NFS_UNSTABLE;
 	copy->cp_consecutive = 1;
 	copy->cp_synchronous = sync;
@@ -1360,6 +1360,7 @@ static int _nfsd_copy_file_range(struct nfsd4_copy *copy)
 	size_t bytes_to_copy;
 	u64 src_pos = copy->cp_src_pos;
 	u64 dst_pos = copy->cp_dst_pos;
+	bool cancelled = false;
 
 	do {
 		bytes_to_copy = min_t(u64, bytes_total, MAX_RW_COUNT);
@@ -1371,7 +1372,12 @@ static int _nfsd_copy_file_range(struct nfsd4_copy *copy)
 		copy->cp_res.wr_bytes_written += bytes_copied;
 		src_pos += bytes_copied;
 		dst_pos += bytes_copied;
-	} while (bytes_total > 0 && !copy->cp_synchronous);
+		if (!copy->cp_synchronous) {
+			spin_lock(&copy->cps->cp_lock);
+			cancelled = copy->cps->cp_cancelled;
+			spin_unlock(&copy->cps->cp_lock);
+		}
+	} while (bytes_total > 0 && !copy->cp_synchronous && !cancelled);
 	return bytes_copied;
 }
 
@@ -1416,6 +1422,8 @@ static void dup_copy_fields(struct nfsd4_copy *src, struct nfsd4_copy *dst)
 	dst->fh_dst = src->fh_dst;
 	dst->ss_mnt = src->ss_mnt;
 	dst->net = src->net;
+	dst->stid = src->stid;
+	dst->cps = src->cps;
 }
 
 static void nfsd4_do_async_copy(struct work_struct *work)
@@ -1434,6 +1442,10 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 			goto out_err;
 	}
 	copy->nfserr = nfsd4_do_copy(copy, 0);
+
+	if (copy->cps->cp_cancelled)
+		goto out;
+
 	cb_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
 	if (!cb_copy)
 		goto out;
@@ -1445,6 +1457,7 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 			&nfsd4_cb_offload_ops, NFSPROC4_CLNT_CB_OFFLOAD);
 	nfsd4_run_cb(&cb_copy->cp_cb);
 out:
+	nfs4_put_stid(copy->stid);
 	kfree(copy);
 	return;
 out_err:
@@ -1475,15 +1488,26 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 		sizeof(struct knfsd_fh));
 	copy->net = SVC_NET(rqstp);
 	if (!copy->cp_synchronous) {
+		struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
 		struct nfsd4_copy *async_copy;
 
 		status = nfsd4_init_copy_res(copy, 0);
 		async_copy = kzalloc(sizeof(struct nfsd4_copy), GFP_KERNEL);
 		if (!async_copy)
 			goto out_err;
+		copy->cps = nfs4_alloc_init_cp_state(nn, nn->nfsd4_lease,
+				copy->stid);
+		if (!copy->cps)
+			goto out_err;
+		/* take a reference on the parent stateid so it's not
+		 * not freed by the copy compound
+		 */
+		atomic_inc(&copy->stid->sc_count);
+		copy->cps->cp_dst_async = true;
+		spin_lock_init(&copy->cps->cp_lock);
+		memcpy(&copy->cp_res.cb_stateid, &copy->cps->cp_stateid,
+			sizeof(copy->cps->cp_stateid));
 		dup_copy_fields(copy, async_copy);
-		memcpy(&copy->cp_res.cb_stateid, &copy->cp_dst_stateid,
-			sizeof(copy->cp_dst_stateid));
 		INIT_WORK(&async_copy->cp_work, nfsd4_do_async_copy);
 		queue_work(copy_wq, &async_copy->cp_work);
 	} else {
@@ -1596,7 +1620,17 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 	struct nfs4_cp_state *state = NULL;
 
 	status = find_cp_state(nn, &os->stateid, &state);
-	if (!status) {
+	/* on the source server, remove stateid from list of acceptable
+	 * stateid to force reads to fail. on the destination server,
+	 * callback offload stateids shouldn't be removed and instead
+	 * mark the offload copy state to be cancelled.
+	 */
+	if (state) {
+		spin_lock(&state->cp_lock);
+		state->cp_cancelled = true;
+		spin_unlock(&state->cp_lock);
+	}
+	if (!status && !state->cp_dst_async) {
 		list_del(&state->cp_list);
 		nfs4_free_cp_state(state);
 	}
fs/nfsd/state.h +3 -0
index fa749763..70ee3fe 100644
@@ -114,6 +114,9 @@ struct nfs4_cp_state {
 	struct nfs4_stid	*cp_p_stid;	/* pointer to parent */
 	bool			cp_active;	/* has the copy started */
 	unsigned long		cp_timeout;	/* copy timeout */
+	bool			cp_dst_async;	/* async copy on dst server */
+	bool			cp_cancelled;	/* copy cancelled */
+	spinlock_t		cp_lock;
 };
 
 /*
fs/nfsd/xdr4.h +2 -0
index d75e530..ae5c6e3 100644
@@ -547,6 +547,8 @@ struct nfsd4_copy {
 	struct file		*fh_dst;
 	struct vfsmount		*ss_mnt;
 	struct net		*net;
+	struct nfs4_stid	*stid;
+	struct nfs4_cp_state	*cps;
 };
 
 struct nfsd4_seek {
Signature · 1 line
1.8.3.1
#77 lore raw

RFC v3 39/42 NFSD define EBADF in nfserrno

In reply to Olga Kornievskaia

vfs_copy_file_range() can return EBADF which currently nfsd does not recognize.

---
 fs/nfsd/nfsproc.c | 1 +
 1 file changed, 1 insertion(+)
fs/nfsd/nfsproc.c +1 -0
index 03a7e9d..5924926 100644
@@ -787,6 +787,7 @@ struct svc_version	nfsd_version2 = {
 		{ nfserr_serverfault, -ENFILE },
 		{ nfserr_io, -EUCLEAN },
 		{ nfserr_perm, -ENOKEY },
+		{ nfserr_badhandle, -EBADF},
 	};
 	int	i;
 
Signature · 1 line
1.8.3.1
#78 lore raw

RFC v3 40/42 NFSD support OFFLOAD_STATUS

In reply to Olga Kornievskaia

Update number of bytes copied in the copy state and query that value under lock if OFFLOAD_STATUS operation received.

---
 fs/nfsd/nfs4proc.c | 15 ++++++++++++++-
 fs/nfsd/state.h    |  1 +
 2 files changed, 15 insertions(+), 1 deletion(-)
fs/nfsd/nfs4proc.c +14 -1
index c411887..1094286 100644
@@ -1375,6 +1375,8 @@ static int _nfsd_copy_file_range(struct nfsd4_copy *copy)
 		if (!copy->cp_synchronous) {
 			spin_lock(&copy->cps->cp_lock);
 			cancelled = copy->cps->cp_cancelled;
+			copy->cps->cp_bytes_copied =
+				copy->cp_res.wr_bytes_written;
 			spin_unlock(&copy->cps->cp_lock);
 		}
 	} while (bytes_total > 0 && !copy->cp_synchronous && !cancelled);
@@ -1665,7 +1667,18 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 		     struct nfsd4_compound_state *cstate,
 		     struct nfsd4_offload_status *os)
 {
-	return nfserr_notsupp;
+	struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
+	__be32 status;
+	struct nfs4_cp_state *state = NULL;
+
+	status = find_cp_state(nn, &os->stateid, &state);
+
+	if (state) {
+		spin_lock(&state->cp_lock);
+		os->count = state->cp_bytes_copied;
+		spin_unlock(&state->cp_lock);
+	}
+	return status;
 }
 
 static __be32
fs/nfsd/state.h +1 -0
index 70ee3fe..25c5d82 100644
@@ -117,6 +117,7 @@ struct nfs4_cp_state {
 	bool			cp_dst_async;	/* async copy on dst server */
 	bool			cp_cancelled;	/* copy cancelled */
 	spinlock_t		cp_lock;
+	ssize_t			cp_bytes_copied;/* copy progress */
 };
 
 /*
Signature · 1 line
1.8.3.1
#79 lore raw

RFC v3 41/42 NFSD remove copy stateid when vfs_copy_file_range completes

In reply to Olga Kornievskaia

Before this patch, the copy state information sticks around until destination file is closed. However, we can release the resources earlier when copy is done.

In this patch, I choose to remove the stateid after vfs copy is done but before CB_OFFLOAD is done. The reason is simple because I don't need to keep track of the nfsd_net structure then when doing the callback (alternatively, we also need to copy the nfsd_net structure for the callback). The drawback is that time copy state information is available for query by OFFLOAD_STATUS is slightly less.

---
 fs/nfsd/nfs4proc.c | 2 ++
 1 file changed, 2 insertions(+)
fs/nfsd/nfs4proc.c +2 -0
index 1094286..86c20fa 100644
@@ -1459,6 +1459,8 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 			&nfsd4_cb_offload_ops, NFSPROC4_CLNT_CB_OFFLOAD);
 	nfsd4_run_cb(&cb_copy->cp_cb);
 out:
+	list_del(&copy->cps->cp_list);
+	nfs4_free_cp_state(copy->cps);
 	nfs4_put_stid(copy->stid);
 	kfree(copy);
 	return;
Signature · 1 line
1.8.3.1
#80 lore raw

RFC v3 42/42 NFSD delay the umount after COPY

In reply to Olga Kornievskaia

After COPY is done, we always unmounted right away. However, if another copy arrives right after it (ie, large copy that's broken down into pieces), then we want to save the time of re-establishing the clientid/session with the source server.

---
 fs/nfsd/nfs4proc.c  | 41 +++++++++++++++++++++++++++++++++++++----
 fs/nfsd/nfs4state.c |  6 ++++++
 fs/nfsd/state.h     |  2 ++
 fs/nfsd/xdr4.h      |  5 +++++
 4 files changed, 50 insertions(+), 4 deletions(-)
fs/nfsd/nfs4proc.c +37 -4
index 86c20fa..7dbe8df 100644
@@ -78,6 +78,7 @@
 #endif
 
 static struct workqueue_struct *copy_wq;
+static struct workqueue_struct *umount_wq;
 
 int nfsd4_create_copy_queue(void)
 {
@@ -98,6 +99,19 @@ void nfsd4_shutdown_copy(struct nfs4_client *clp)
 	flush_workqueue(copy_wq);
 }
 
+int nfsd4_create_umount_queue(void)
+{
+	umount_wq = create_singlethread_workqueue("nfsd4_umount");
+	if (!umount_wq)
+		return -ENOMEM;
+	return 0;
+}
+
+void nfsd4_destroy_umount_queue(void)
+{
+	destroy_workqueue(umount_wq);
+}
+
 #define NFSDDBG_FACILITY		NFSDDBG_PROC
 
 static u32 nfsd_attrmask[] = {
@@ -1211,12 +1225,24 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 }
 
 static void
-nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
+_nfsd4_interssc_disconnect(struct vfsmount *ss_mnt)
 {
 	mntput(ss_mnt);
 	nfs_sb_deactive(ss_mnt->mnt_sb);
 }
 
+static void
+nfsd4_interssc_disconnect(struct work_struct *work)
+{
+	struct delayed_work *dwork = to_delayed_work(work);
+	struct nfsd4_delayed_umount *mount =
+		container_of(dwork, struct nfsd4_delayed_umount, work);
+
+	_nfsd4_interssc_disconnect(mount->ss_mnt);
+	kfree(mount);
+}
+
+
 /**
  * nfsd4_setup_inter_ssc
  *
@@ -1270,11 +1296,18 @@ extern struct file *nfs42_ssc_open(struct vfsmount *ss_mnt,
 nfsd4_cleanup_inter_ssc(struct vfsmount *ss_mnt, struct file *src,
 			struct file *dst)
 {
+	struct nfsd4_delayed_umount *umount;
 	nfs42_ssc_close(src);
 	fput(src);
 	fput(dst);
 
-	nfsd4_interssc_disconnect(ss_mnt);
+	umount = kzalloc(sizeof(struct nfsd4_delayed_umount), GFP_KERNEL);
+	if (!umount)
+		return;
+	umount->ss_mnt = ss_mnt;
+	mntget(ss_mnt);
+	INIT_DELAYED_WORK(&umount->work, nfsd4_interssc_disconnect);
+	queue_delayed_work(umount_wq, &umount->work, 90 * HZ);
 
 }
 
@@ -1465,7 +1498,7 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 	kfree(copy);
 	return;
 out_err:
-	nfsd4_interssc_disconnect(copy->ss_mnt);
+	_nfsd4_interssc_disconnect(copy->ss_mnt);
 	goto out;
 }
 
@@ -1520,7 +1553,7 @@ static void nfsd4_do_async_copy(struct work_struct *work)
 out:
 	return status;
 out_err:
-	nfsd4_interssc_disconnect(copy->ss_mnt);
+	_nfsd4_interssc_disconnect(copy->ss_mnt);
 	status = nfserrno(-ENOMEM);
 	goto out;
 }
fs/nfsd/nfs4state.c +6 -0
index 7727ce53..daee2cc 100644
@@ -7138,8 +7138,13 @@ static int nfs4_state_create_net(struct net *net)
 	ret = nfsd4_create_copy_queue();
 	if (ret)
 		goto out_free_callback;
+	ret = nfsd4_create_umount_queue();
+	if (ret)
+		goto out_free_copy_queue;
 
 	return 0;
+out_free_copy_queue:
+	nfsd4_destroy_copy_queue();
 out_free_callback:
 	nfsd4_destroy_callback_queue();
 out_free_laundry:
@@ -7205,6 +7210,7 @@ static int nfs4_state_create_net(struct net *net)
 	nfsd4_destroy_callback_queue();
 	cleanup_callback_cred();
 	nfsd4_destroy_copy_queue();
+	nfsd4_destroy_umount_queue();
 }
 
 static void
fs/nfsd/state.h +2 -0
index 25c5d82..69db34d 100644
@@ -657,6 +657,8 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern bool nfs4_has_reclaimed_state(const char *name, struct nfsd_net *nn);
 extern int nfsd4_create_copy_queue(void);
 extern void nfsd4_destroy_copy_queue(void);
+extern int nfsd4_create_umount_queue(void);
+extern void nfsd4_destroy_umount_queue(void);
 extern __be32 find_cp_state(struct nfsd_net *nn, stateid_t *st,
 			struct nfs4_cp_state **cps);
 
fs/nfsd/xdr4.h +5 -0
index ae5c6e3..3b2176f 100644
@@ -551,6 +551,11 @@ struct nfsd4_copy {
 	struct nfs4_cp_state	*cps;
 };
 
+struct nfsd4_delayed_umount {
+	struct vfsmount		*ss_mnt;
+	struct delayed_work	work;
+};
+
 struct nfsd4_seek {
 	/* request */
 	stateid_t	seek_stateid;
Signature · 1 line
1.8.3.1