NFSv4.2: Add support for inter-server to server COPY
Subsystems (4)
cover NFSv4.2: Add support for inter-server to server COPY
From: Andy Adamson <[email protected]>
Version 2:
Fixed xdr issues.
Fixed stateid issues
Added lease time to copy_notify return
Version 1:
These patches add client and server support for NFSv4.2 inter-server to server copy offload. These patches build upon Anna Schumaker's COPY Operation patches which are in turn built upon Zack Brown's VFS vfs_copy_file_range syscall. Additional functionality is added to the COPY operation past what Anna added. The COPY_NOTIFY operation is added.
The first three patches change Zach Brown's vfs_copy_file_range implementation. The first two patches relax the check for only performing copy offload between files with the same mount and super block, moving that check into a helper function that can be called (e.g. BTRFS) in a file systems copy_file_range function. The third patch changes vfs_copy_file_range to call the destination file (file_out) copy_file_range f_ops proceedure instead of the file_in version.
The remaining patches implement the COPY_NOTIFY operation and the inter-ssc changes to the COPY operation. One source server using NL4_NETADDR is currently supported. I used the test program from Anna's COPY operation patch set.
Andy Adamson (12):
VFS: Separate cross fs check from vfs_copy_file_range
BTRFS: Use VFS copy offload helper
VFS SQUASH use file_out instead of file_in for copy_file_range NFS add same file check and flush source and destination for COPY NFS add inter ssc functions to nfs42proc
NFS add COPY_NOTIFY operation
NFSD add ca_source_server<> to COPY
NFSD add COPY_NOTIFY operation
NFS add ca_source_server<> to COPY
NFSD generalize nfsd4_compound_state flag names
NFSD: allow inter server COPY to have a STALE source server fh NFSD add nfs4 inter ssc to nfsd4_copy
Olga Kornievskaia (4):
NFS COPY xdr changes
NFS in copy use stateid returned by copy_notify NFS always use openstateid in COPY_NOTIFY
NFSD: extra stateid checking in read for interserver copy
fs/btrfs/ioctl.c | 3 + fs/nfs/nfs42.h | 2 +- fs/nfs/nfs42proc.c | 238 +++++++++++++++++++++- fs/nfs/nfs42xdr.c | 213 +++++++++++++++++++- fs/nfs/nfs4_fs.h | 9 +- fs/nfs/nfs4file.c | 17 +- fs/nfs/nfs4proc.c | 8 +- fs/nfs/nfs4state.c | 2 +- fs/nfs/nfs4xdr.c | 1 + fs/nfsd/Kconfig | 10 + fs/nfsd/nfs4proc.c | 456 +++++++++++++++++++++++++++++++++++++++++- fs/nfsd/nfs4state.c | 37 +++- fs/nfsd/nfs4xdr.c | 203 ++++++++++++++++++- fs/nfsd/nfsd.h | 2 + fs/nfsd/state.h | 8 + fs/nfsd/xdr4.h | 29 ++- fs/read_write.c | 38 ++-- include/linux/fs.h | 1 + include/linux/nfs4.h | 7 + include/linux/nfs4intercopy.h | 39 ++++ include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 28 +++ 22 files changed, 1289 insertions(+), 63 deletions(-) create mode 100644 include/linux/nfs4intercopy.h
Signature · 1 line
1.8.3.1
PATCH 1/16 VFS: Separate cross fs check from vfs_copy_file_range
From: Andy Adamson <[email protected]>
Move the high level vfs_copy_file_range entrypoint restriction that limits copy offloading to files on the same mount and super to
a helper function to be called by copy_file_range methods.
--- fs/read_write.c | 34 +++++++++++++++++++++++----------- include/linux/fs.h | 1 + 2 files changed, 24 insertions(+), 11 deletions(-)
fs/read_write.c +23 -11
@@ -1330,6 +1330,29 @@ COMPAT_SYSCALL_DEFINE4(sendfile64, int, out_fd, int, in_fd,#endif /* +* copy_offload_same_fs(). Test for file system copy_file_range methods +* that support only copy offloading to files on the same mount point +* and super (and not to the same file). +*/ +ssize_t copy_offload_same_fs (struct file *file_in,struct file *file_out) +{ +struct inode *inode_in =file_inode (file_in); +struct inode *inode_out =file_inode (file_out); + +/* forbid copy not on the same mount point and super */ +if (inode_in->i_sb != inode_out->i_sb || + file_in->f_path.mnt != file_out->f_path.mnt) +return -EXDEV ; + +/* forbid ranges in the same file */ +if (inode_in == inode_out) +return -EINVAL ; + +return 0 ; +} +EXPORT_SYMBOL (copy_offload_same_fs); + +/* * copy_file_range() differs from regular file read and write in that it * specifically allows return partial success. When it does so is up to * the copy_file_range method.
@@ -1339,7 +1362,6 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in, size_t len, int flags) {struct inode *inode_in; -struct inode *inode_out;ssize_t ret;if (flags)
@@ -1362,22 +1384,12 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,return -EINVAL ; inode_in =file_inode (file_in); - inode_out =file_inode (file_out);/* make sure offsets don't wrap and the input is inside i_size */ if (pos_in + len < pos_in || pos_out + len < pos_out || pos_in + len >i_size_read (inode_in))return -EINVAL ; -/* this could be relaxed once a method supports cross-fs copies */ -if (inode_in->i_sb != inode_out->i_sb || - file_in->f_path.mnt != file_out->f_path.mnt) -return -EXDEV ; - -/* forbid ranges in the same file */ -if (inode_in == inode_out) -return -EINVAL ; - ret =mnt_want_write_file (file_out);if (ret)return ret;
include/linux/fs.h +1 -0
@@ -1685,6 +1685,7 @@ extern ssize_t vfs_readv(struct file *, const struct iovec __user *, unsigned long, loff_t *);extern ssize_t vfs_writev (struct file *,const struct iovec __user *,unsigned long ,loff_t *); +extern ssize_t copy_offload_same_fs (struct file *,struct file *);extern ssize_t vfs_copy_file_range (struct file *,loff_t ,struct file *,loff_t ,size_t ,int );
Signature · 1 line
1.8.3.1
PATCH 2/16 BTRFS: Use VFS copy offload helper
In reply to [email protected]From: Andy Adamson <[email protected]>
--- fs/btrfs/ioctl.c | 3 +++ 1 file changed, 3 insertions(+)
fs/btrfs/ioctl.c +3 -0
@@ -3856,6 +3856,9 @@ ssize_t btrfs_copy_file_range(struct file *file_in, loff_t pos_in, {ssize_t ret; + ret =copy_offload_same_fs (file_in, file_out); +if (ret !=0 ) +return ret; ret =btrfs_clone_files (file_out, file_in, pos_in, len, pos_out);if (ret ==0 ) ret = len;
Signature · 1 line
1.8.3.1
PATCH 3/16 VFS SQUASH use file_out instead of file_in for copy_file_range
In reply to [email protected]From: Andy Adamson <[email protected]>
Calling file_in->copy_file_range file operations function will not work for NFS inter-server to server COPY. In the intra-ssc case, when the destination server calls vfs_copy_file_range (from nfsd_copy_range) to perform the copy offload, it uses an NFS client file_in struct file setup to let the destination server (acting as an NFS client) NFS READ the data to be copied from the source server. Using the NFS file_in thus results in calling the NFS client copy_file_range f-ops, nfs4_copy_file_range, which results in the destination server trying to start a new COPY (calling COPY_NOTIFY etc) causing a circular call mess.
Instead, vfs_copy_file_range should use the file_out f_ops->copy_file_range proceedure if it has one. The vfs_copy_file_range is a destination based operation as the source is simply being read, while the destination is being written and can therefore utilize a copy_file_range call if provided.
--- fs/read_write.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
fs/read_write.c +2 -2
@@ -1395,8 +1395,8 @@ ssize_t vfs_copy_file_range(struct file *file_in, loff_t pos_in,return ret; ret = -ENOTSUPP ; -if (file_in->f_op->copy_file_range) - ret = file_in->f_op->copy_file_range (file_in, pos_in, file_out, +if (file_out->f_op->copy_file_range) + ret = file_out->f_op->copy_file_range (file_in, pos_in, file_out, pos_out, len, flags);if (ret == -ENOTSUPP ) ret =do_splice_direct (file_in, &pos_in, file_out, &pos_out, len, flags);
Signature · 1 line
1.8.3.1
PATCH 4/16 NFS COPY xdr changes
In reply to [email protected]From: Olga Kornievskaia <[email protected]>
add support for decoding NFS4ERR_OFFLOAD_NO_REQS status and corresponding copy_requirements structure.
--- fs/nfs/nfs42xdr.c | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)
fs/nfs/nfs42xdr.c +22 -13
@@ -290,19 +290,9 @@ out_overflow:print_overflow_msg (__func__, xdr);return -EIO ; } - -static int decode_copy (struct xdr_stream *xdr,struct nfs42_copy_res *res) -{ +static int decode_copy_requirements (struct xdr_stream *xdr, +struct nfs42_copy_res *res) {__be32 *p; -int status; - - status =decode_op_hdr (xdr,OP_COPY ); -if (status) -return status; - - status =decode_write_response (xdr, &res->write_res); -if (status) -return status; p =xdr_inline_decode (xdr,4 +4 );if (unlikely (!p))
@@ -311,12 +301,31 @@ static int decode_copy(struct xdr_stream *xdr, struct nfs42_copy_res *res) res->consecutive =be32_to_cpup (p++); res->synchronous =be32_to_cpup (p++);return 0 ; - out_overflow:print_overflow_msg (__func__, xdr);return -EIO ; } +static int decode_copy (struct xdr_stream *xdr,struct nfs42_copy_res *res) +{ +int status; + + status =decode_op_hdr (xdr,OP_COPY ); +if (status ==NFS4ERR_OFFLOAD_NO_REQS ) { + status =decode_copy_requirements (xdr, res); +if (status) +return status; +return NFS4ERR_OFFLOAD_NO_REQS ; + }else if (status) +return status; + + status =decode_write_response (xdr, &res->write_res); +if (status) +return status; + +return decode_copy_requirements (xdr, res); +} +static int decode_deallocate (struct xdr_stream *xdr,struct nfs42_falloc_res *res) {return decode_op_hdr (xdr,OP_DEALLOCATE );
Signature · 1 line
1.8.3.1
PATCH 5/16 NFS add same file check and flush source and destination for COPY
In reply to [email protected]From: Andy Adamson <[email protected]>
--- fs/nfs/nfs4file.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
fs/nfs/nfs4file.c +15 -0
@@ -136,6 +136,21 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, size_t count, int flags) { + struct inode *in_inode = file_inode(file_in); +struct inode *out_inode =file_inode (file_out); +int ret; + +if (in_inode == out_inode) +return -EINVAL ; + +/* flush any pending writes */ + ret =nfs_sync_inode (in_inode); +if (ret) +return ret; + ret =nfs_sync_inode (out_inode); +if (ret) +return ret; +return nfs42_proc_copy (file_in, pos_in, file_out, pos_out, count); }
Signature · 1 line
1.8.3.1
PATCH 6/16 NFS add inter ssc functions to nfs42proc
In reply to [email protected]From: Andy Adamson <[email protected]>
Given an NFSv4 stateid and filehandle from the COPY operaion, provide the destination server with an NFS client function to create a struct file suitable for the destiniation server to READ the data to be copied.
nfs4intercopy.h holds symbols shared by nfs and nfsd.
--- fs/nfs/nfs42proc.c | 106 ++++++++++++++++++++++++++++++++++++++++++ fs/nfs/nfs4_fs.h | 7 +++ fs/nfs/nfs4proc.c | 7 +-- include/linux/nfs4intercopy.h | 28 +++++++++++ include/linux/nfs_xdr.h | 3 ++ 5 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 include/linux/nfs4intercopy.h
fs/nfs/nfs42proc.c +106 -0
@@ -8,6 +8,7 @@#include <linux/nfs4.h> #include <linux/nfs_xdr.h> #include <linux/nfs_fs.h> +#include <linux/file.h> #include "nfs4_fs.h" #include "nfs42.h" #include "iostat.h"
@@ -309,3 +310,108 @@ int nfs42_proc_layoutstats_generic(struct nfs_server *server,return PTR_ERR (task);return 0 ; } + +static int read_name_gen =1 ; +#define SSC_READ_NAME_BODY "ssc_read_%d" + +struct file * +nfs42_ssc_open (struct nfs42_inter_ssc *ssc,struct nfs_fh *src_fh, +nfs4_stateid *stateid) +{ +struct nfs_fattr fattr; +struct path path = { + .dentry =NULL , + }; +struct file *filep, *res; +struct nfs_server *server; +struct inode *r_ino =NULL ; +struct nfs_open_context *ctx; +struct nfs4_state_owner *sp; +char *read_name; +int len, status =0 ; + +/* vfsmount is bad for some reason */ +if (IS_ERR (ssc->sc_root_mnt)) { +dprintk ("%s MOUNT ERROR %ld\n" , __func__, +PTR_ERR (ssc->sc_root_mnt)); + res =ERR_CAST (ssc->sc_root_mnt); + goto out; + } + server =NFS_SERVER (ssc->sc_mnt_dentry->d_inode); + +nfs_fattr_init (&fattr); + + status =nfs4_proc_getattr (server, src_fh, &fattr,NULL ); +if (status <0 ) { + res =ERR_PTR (status); + goto out; + } + + res =ERR_PTR (-ENOMEM ); + len =strlen (SSC_READ_NAME_BODY ) +16 ; + read_name =kzalloc (len,GFP_NOFS ); +if (read_name ==NULL ) + goto out; +snprintf (read_name, len,SSC_READ_NAME_BODY , read_name_gen++); +dprintk ("%s read_name %s\n" , __func__, read_name); + +/* Just put the file under the mount point */ + path.dentry =d_alloc_name (ssc->sc_mnt_dentry, read_name); +kfree (read_name); +if (path.dentry ==NULL ) + goto out; + + path.mnt = ssc->sc_root_mnt; + + r_ino =nfs_fhget (ssc->sc_mnt_dentry->d_inode->i_sb, src_fh, &fattr, +NULL ); +if (IS_ERR (r_ino)) { + res =ERR_CAST (r_ino); + goto out_path; + } + +d_add_unique (path.dentry, r_ino); + + filep =alloc_file (&path,FMODE_READ , r_ino->i_fop); +if (IS_ERR (filep)) { + res =ERR_CAST (filep); + goto out_path; + } + + ctx =alloc_nfs_open_context (filep->f_path.dentry, filep->f_mode); +if (IS_ERR (ctx)) { + res =ERR_CAST (ctx); + goto out_filep; + } + + res =ERR_PTR (-EINVAL ); + sp =nfs4_get_state_owner (server, ctx->cred,GFP_KERNEL ); +if (sp ==NULL ) + goto out_ctx; + + ctx->state =nfs4_get_open_state (r_ino, sp); +if (ctx->state ==NULL ) + goto out_stateowner; + +__update_open_stateid (ctx->state, stateid,NULL , filep->f_mode); + +nfs_file_set_open_context (filep, ctx); +put_nfs_open_context (ctx);/* nfs_open does this.. :) */ + + res = filep; +out: +dprintk ("<-- %s error %ld filep %p r_ino %p\n" , + __func__,IS_ERR (res) ?PTR_ERR (res) :0 , res, r_ino); + +return res; +out_stateowner: +nfs4_put_state_owner (sp); +out_ctx: +put_nfs_open_context (ctx); +out_filep: +fput (filep); +out_path: +path_put (&path);/* dput dentry and mntput mnt */ +goto out; +} +EXPORT_SYMBOL_GPL (nfs42_ssc_open);
fs/nfs/nfs4_fs.h +7 -0
@@ -257,6 +257,13 @@ extern int nfs4_set_rw_stateid(nfs4_stateid *stateid,const struct nfs_open_context *ctx, const struct nfs_lock_context *l_ctx, fmode_t fmode); +extern int nfs4_proc_getattr (struct nfs_server *server,struct nfs_fh *fhandle, +struct nfs_fattr *fattr, +struct nfs4_label *label); +extern void __update_open_stateid (struct nfs4_state *state, +nfs4_stateid *open_stateid, +const nfs4_stateid *deleg_stateid, +fmode_t fmode);#if defined(CONFIG_NFS_V4_1 )static inline struct nfs4_session *nfs4_get_session (const struct nfs_server *server)
fs/nfs/nfs4proc.c +4 -3
@@ -80,7 +80,6 @@ static int _nfs4_recover_proc_open(struct nfs4_opendata *data);static int nfs4_do_fsinfo (struct nfs_server *,struct nfs_fh *,struct nfs_fsinfo *);static int nfs4_async_handle_error (struct rpc_task *,const struct nfs_server *,struct nfs4_state *,long *);static void nfs_fixup_referral_attributes (struct nfs_fattr *fattr); -static int nfs4_proc_getattr (struct nfs_server *,struct nfs_fh *,struct nfs_fattr *,struct nfs4_label *label);static int _nfs4_proc_getattr (struct nfs_server *server,struct nfs_fh *fhandle,struct nfs_fattr *fattr,struct nfs4_label *label); static int nfs4_do_setattr(struct inode *inode, struct rpc_cred *cred, struct nfs_fattr *fattr, struct iattr *sattr,
@@ -1276,7 +1275,7 @@ static void nfs_set_open_stateid_locked(struct nfs4_state *state, nfs4_stateid *nfs4_stateid_copy (&state->open_stateid, stateid); } -static void __update_open_stateid (struct nfs4_state *state,nfs4_stateid *open_stateid,const nfs4_stateid *deleg_stateid,fmode_t fmode) +void __update_open_stateid (struct nfs4_state *state,nfs4_stateid *open_stateid,const nfs4_stateid *deleg_stateid,fmode_t fmode) {/* * Protect the call to nfs4_state_set_mode_locked and
@@ -1294,6 +1293,7 @@ static void __update_open_stateid(struct nfs4_state *state, nfs4_stateid *open_supdate_open_stateflags (state, fmode);spin_unlock (&state->owner->so_lock); } +EXPORT_SYMBOL_GPL (__update_open_stateid);static int update_open_stateid (struct nfs4_state *state,nfs4_stateid *open_stateid,nfs4_stateid *delegation,fmode_t fmode) {
@@ -3233,7 +3233,7 @@ static int _nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle,return nfs4_call_sync (server->client, server, &msg, &args.seq_args, &res.seq_res,0 ); } -static int nfs4_proc_getattr (struct nfs_server *server,struct nfs_fh *fhandle, +int nfs4_proc_getattr (struct nfs_server *server,struct nfs_fh *fhandle,struct nfs_fattr *fattr,struct nfs4_label *label) {struct nfs4_exception exception = { };
@@ -3246,6 +3246,7 @@ static int nfs4_proc_getattr(struct nfs_server *server, struct nfs_fh *fhandle, }while (exception.retry);return err; } +EXPORT_SYMBOL_GPL (nfs4_proc_getattr);/* * The file is not closed if it is opened due to the a request to change
include/linux/nfs4intercopy.h new file +28 -0
@@ -0,0 +1,28 @@ +/* +* linux/fs/nfs/nfs4intercopy.h +* +* Copyright (C) 2014 Andy Adamson <[email protected]> +* +* nfs inter-server server-side copy READ implementation +* +*/ + +#include <linux/socket.h> +#include <linux/sunrpc/msg_prot.h> +#include <linux/nfs.h> +#include <linux/nfs4.h> + +struct nfs42_netaddr { +unsigned int na_netid_len; +char na_netid[RPCBIND_MAXNETIDLEN +1 ]; +unsigned int na_uaddr_len; +char na_uaddr[RPCBIND_MAXUADDRLEN +1 ]; +}; + +struct nfs42_inter_ssc { +struct vfsmount *sc_root_mnt; +struct dentry *sc_mnt_dentry; +}; + +extern struct file *nfs42_ssc_open (struct nfs42_inter_ssc *ssc, +struct nfs_fh *fh,nfs4_stateid *stateid);
include/linux/nfs_xdr.h +3 -0
@@ -1303,6 +1303,9 @@ nfs_free_pnfs_ds_cinfo(struct pnfs_ds_commit_info *cinfo)#endif /* CONFIG_NFS_V4_1 */ #ifdef CONFIG_NFS_V4_2 + +#include <linux/nfs4intercopy.h> + struct nfs42_falloc_args { struct nfs4_sequence_args seq_args;
Signature · 1 line
1.8.3.1
PATCH 7/16 NFS add COPY_NOTIFY operation
In reply to [email protected]From: Andy Adamson <[email protected]>
Only NL4_NETATTR, No support for NL4_NAME and NL4_URL. Allow only one source server address to be returned for now.
--- fs/nfs/nfs42.h | 2 +- fs/nfs/nfs42proc.c | 98 ++++++++++++++++++++++++- fs/nfs/nfs42xdr.c | 167 ++++++++++++++++++++++++++++++++++++++++++ fs/nfs/nfs4file.c | 2 +- fs/nfs/nfs4proc.c | 1 + fs/nfs/nfs4xdr.c | 1 + include/linux/nfs4.h | 7 ++ include/linux/nfs4intercopy.h | 11 +++ include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 22 ++++++ 10 files changed, 307 insertions(+), 5 deletions(-)
fs/nfs/nfs42.h +1 -1
@@ -13,7 +13,7 @@/* nfs4.2proc.c */ int nfs42_proc_allocate (struct file *,loff_t ,loff_t ); -ssize_t nfs42_proc_copy (struct file *,loff_t ,struct file *,loff_t ,size_t ); +ssize_t nfs42_do_copy (struct file *,loff_t ,struct file *,loff_t ,size_t );int nfs42_proc_deallocate (struct file *,loff_t ,loff_t );loff_t nfs42_proc_llseek (struct file *,loff_t ,int ); int nfs42_proc_layoutstats_generic(struct nfs_server *,
fs/nfs/nfs42proc.c +95 -3
122 lines not on this page · raw message
fs/nfs/nfs42xdr.c +167 -0
203 lines not on this page · raw message
fs/nfs/nfs4file.c +1 -1
@@ -151,7 +151,7 @@ static ssize_t nfs4_copy_file_range(struct file *file_in, loff_t pos_in,if (ret)return ret; -return nfs42_proc_copy (file_in, pos_in, file_out, pos_out, count); +return nfs42_do_copy (file_in, pos_in, file_out, pos_out, count); }static loff_t nfs4_file_llseek (struct file *filep,loff_t offset,int whence)
fs/nfs/nfs4proc.c +1 -0
@@ -8650,6 +8650,7 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = { | NFS_CAP_ATOMIC_OPEN_V1 |NFS_CAP_ALLOCATE |NFS_CAP_COPY + |NFS_CAP_COPY_NOTIFY |NFS_CAP_DEALLOCATE |NFS_CAP_SEEK | NFS_CAP_LAYOUTSTATS,
fs/nfs/nfs4xdr.c +1 -0
@@ -7433,6 +7433,7 @@ struct rpc_procinfo nfs4_procedures[] = {PROC (DEALLOCATE , enc_deallocate, dec_deallocate),PROC (LAYOUTSTATS , enc_layoutstats, dec_layoutstats),PROC (COPY , enc_copy, dec_copy), + PROC(COPY_NOTIFY, enc_copy_notify, dec_copy_notify), #endif /* CONFIG_NFS_V4_2 */ };
include/linux/nfs4.h +7 -0
@@ -502,6 +502,7 @@ enum {NFSPROC4_CLNT_DEALLOCATE ,NFSPROC4_CLNT_LAYOUTSTATS ,NFSPROC4_CLNT_COPY , + NFSPROC4_CLNT_COPY_NOTIFY, };/* nfs41 types */
@@ -572,4 +573,10 @@ enum data_content4 { NFS4_CONTENT_HOLE = 1, }; +enum netloc_type4 { +NL4_NAME =1 , +NL4_URL =2 , +NL4_NETADDR =3 , +}; +#endif
include/linux/nfs4intercopy.h +11 -0
@@ -19,6 +19,17 @@ struct nfs42_netaddr {char na_uaddr[RPCBIND_MAXUADDRLEN +1 ]; }; +struct nl4_server { +enum netloc_type4 nl4_type; +union { +struct {/* NL4_NAME, NL4_URL */ +int nl4_str_sz; +char nl4_str[NFS4_OPAQUE_LIMIT +1 ]; + }; +struct nfs42_netaddr nl4_addr;/* NL4_NETADDR */ + } u; +}; +struct nfs42_inter_ssc {struct vfsmount *sc_root_mnt;struct dentry *sc_mnt_dentry;
include/linux/nfs_fs_sb.h +1 -0
@@ -239,5 +239,6 @@ struct nfs_server {#define NFS_CAP_DEALLOCATE (1U << 21)#define NFS_CAP_LAYOUTSTATS (1U << 22)#define NFS_CAP_COPY (1U << 23) +#define NFS_CAP_COPY_NOTIFY (1U << 24)#endif
include/linux/nfs_xdr.h +22 -0
@@ -1324,6 +1324,9 @@ struct nfs42_falloc_res {const struct nfs_server *falloc_server; }; +/* support 1 source server for now */ +#define NFS42_MAX_SSC_SRC 1 +struct nfs42_copy_args {struct nfs4_sequence_args seq_args;
@@ -1351,6 +1354,25 @@ struct nfs42_copy_res {bool synchronous; }; +struct nfs42_copy_notify_args { +struct nfs4_sequence_args cna_seq_args; + +struct nfs_fh *cna_src_fh; +nfs4_stateid cna_src_stateid; +/* cna_destiniation_server */ +struct nl4_server cna_dst; +}; + +struct nfs42_copy_notify_res { +struct nfs4_sequence_res cnr_seq_res; + +struct nfstime4 cnr_lease_time; +nfs4_stateid cnr_stateid; +/* cnr_source_server */ +int cnr_nsrc;/* for now, always 1 */ +struct nl4_server cnr_src[NFS42_MAX_SSC_SRC ]; +}; +struct nfs42_seek_args {struct nfs4_sequence_args seq_args;
Signature · 1 line
1.8.3.1
PATCH 8/16 NFSD add ca_source_server<> to COPY
In reply to [email protected]From: Andy Adamson <[email protected]>
--- fs/nfsd/nfs4proc.c | 14 +++++++++++++- fs/nfsd/nfs4xdr.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/xdr4.h | 6 ++++++ 3 files changed, 66 insertions(+), 3 deletions(-)
fs/nfsd/nfs4proc.c +13 -1
@@ -2014,6 +2014,18 @@ static inline u32 nfsd4_layoutreturn_rsize(struct svc_rqst *rqstp, struct nfsd4_ }#endif /* CONFIG_NFSD_PNFS */ +static inline u32 nfsd4_copy_rsize (struct svc_rqst *rqstp,struct nfsd4_op *op) +{ +return (op_encode_hdr_size + +1 /* wr_callback */ + + op_encode_stateid_maxsz/* wr_callback */ + +2 /* wr_count */ + +1 /* wr_committed */ + + op_encode_verifier_maxsz + +1 /* cr_consecutive */ + +1 /* cr_synchronous */ ) *sizeof (__be32); +} +static struct nfsd4_operation nfsd4_ops[] = { [OP_ACCESS ] = { .op_func = (nfsd4op_func )nfsd4_access,
@@ -2344,7 +2356,7 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_func = (nfsd4op_func)nfsd4_copy, .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, .op_name = "OP_COPY", - .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_rsize, }, [OP_SEEK] = { .op_func = (nfsd4op_func)nfsd4_seek,
fs/nfsd/nfs4xdr.c +47 -2
68 lines not on this page · raw message
fs/nfsd/xdr4.h +6 -0
@@ -37,6 +37,7 @@#ifndef _LINUX_NFSD_XDR4_H#define _LINUX_NFSD_XDR4_H +#include <linux/nfs4intercopy.h> #include "state.h" #include "nfsd.h"
@@ -497,6 +498,9 @@ struct nfsd42_write_res {nfs4_verifier wr_verifier; }; +/* support 1 source server for now */ +#define NFSD4_MAX_SSC_SRC 1 +struct nfsd4_copy {/* request */ stateid_t cp_src_stateid;
@@ -504,6 +508,8 @@ struct nfsd4_copy {u64 cp_src_pos;u64 cp_dst_pos;u64 cp_count; +int cp_nsrc; +struct nl4_server cp_src[NFSD4_MAX_SSC_SRC ];/* both */ bool cp_consecutive;
Signature · 1 line
1.8.3.1
PATCH 9/16 NFSD add COPY_NOTIFY operation
In reply to [email protected]From: Andy Adamson <[email protected]>
--- fs/nfsd/nfs4proc.c | 110 +++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++++- fs/nfsd/xdr4.h | 13 ++++++ 3 files changed, 249 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +110 -0
@@ -35,6 +35,7 @@#include <linux/file.h> #include <linux/falloc.h> #include <linux/slab.h> +#include <linux/sunrpc/addr.h> #include "idmap.h" #include "cache.h"
@@ -1070,6 +1071,94 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static int +nfsd4_set_src_nl4_netaddr (struct svc_rqst *rqstp,struct nfs42_netaddr *naddr) +{ +const struct sockaddr *addr = (struct sockaddr *)&rqstp->rq_daddr; +const struct sockaddr_in *sin = (const struct sockaddr_in *)addr; +const struct sockaddr_in6 *sin6 = (const struct sockaddr_in6 *)addr; +int uaddr_len = rqstp->rq_daddrlen +4 +1 ;/* port (4) and '\0' (1) */ +size_t ret; +unsigned short port; + +switch (addr->sa_family) { +case AF_INET : + port =ntohs (sin->sin_port); + ret =rpc_ntop (addr, naddr->na_uaddr,sizeof (naddr->na_uaddr)); +snprintf (naddr->na_uaddr + ret, uaddr_len,".%u.%u" , + port >>8 , port &255 ); + naddr->na_uaddr_len =strlen (naddr->na_uaddr); + +snprintf (naddr->na_netid,4 ,"%s" ,"tcp" ); + naddr->na_netid_len =3 ; +break ; +case AF_INET6 : + port =ntohs (sin6->sin6_port); + ret =rpc_ntop (addr, naddr->na_uaddr,sizeof (naddr->na_uaddr)); +snprintf (naddr->na_uaddr + ret, uaddr_len,".%u.%u" , + port >>8 , port &255 ); + naddr->na_uaddr_len =strlen (naddr->na_uaddr); + +snprintf (naddr->na_netid,5 ,"%s" ,"tcp6" ); + naddr->na_netid_len =4 ; +break ; +default : +dprintk ("NFSD nfsd4_set_notify_src: unknown address type: %d" , + addr->sa_family); +return -EINVAL ; + } +return 0 ; +} + +static __be32 +nfsd4_copy_notify (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_copy_notify *cp_notify) +{ +__be32 status; +struct file *src =NULL ; +struct nfs42_netaddr *naddr; +struct nfsd_net *nn =net_generic (SVC_NET (rqstp), nfsd_net_id); + + status =nfs4_preprocess_stateid_op (rqstp, cstate, + &cstate->current_fh, + &cp_notify->cpn_src_stateid, +RD_STATE , &src,NULL ); +if (status) +return status; + + +/** XXX Save cpn_src_statid, cpn_src, and any other returned source +* server addresses on which the source server is williing to accept +* connections from the destination e.g. what is returned in cpn_src, +* to verify READ from dest server. +*/ + +/** +* For now, only return one source server address, the address used +* by the client in the static cpn_src. +*/ + cp_notify->cpn_nsrc =1 ; + cp_notify->cpn_src[0 ].nl4_type =NL4_NETADDR ; + naddr = &cp_notify->cpn_src[0 ].u.nl4_addr; + + status =nfsd4_set_src_nl4_netaddr (rqstp, naddr); +if (status !=0 ) + goto out; + + cp_notify->cpn_nsrc =1 ; + cp_notify->cpn_sec = nn->nfsd4_lease; + cp_notify->cpn_nsec =0 ; + +dprintk ("<-- %s cpn_dst %s:%s cpn_nsrc %d cpn_src %s:%s\n" , __func__, + cp_notify->cpn_dst.u.nl4_addr.na_netid, + cp_notify->cpn_dst.u.nl4_addr.na_uaddr, + cp_notify->cpn_nsrc, + cp_notify->cpn_src[0 ].u.nl4_addr.na_netid, + cp_notify->cpn_src[0 ].u.nl4_addr.na_uaddr); +out: +return status; +} +static __be32 nfsd4_fallocate (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate,struct nfsd4_fallocate *fallocate,int flags)
@@ -2026,6 +2115,21 @@ static inline u32 nfsd4_copy_rsize(struct svc_rqst *rqstp, struct nfsd4_op *op) 1 /* cr_synchronous */) * sizeof(__be32); } +static inline u32 nfsd4_copy_notify_rsize (struct svc_rqst *rqstp, +struct nfsd4_op *op) +{ +return (op_encode_hdr_size + +3 /* cnr_lease_time */ + +1 /* We support one cnr_source_server */ + +1 /* cnr_stateid seq */ + + op_encode_stateid_maxsz/* cnr_stateid */ + +1 /* num cnr_source_server*/ + +1 /* nl4_type */ + +1 /* nl4 size */ + +XDR_QUADLEN (NFS4_OPAQUE_LIMIT )/*nl4_loc + nl4_loc_sz */ ) + *sizeof (__be32); +} +static struct nfsd4_operation nfsd4_ops[] = { [OP_ACCESS ] = { .op_func = (nfsd4op_func )nfsd4_access,
@@ -2362,6 +2466,12 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_func = (nfsd4op_func)nfsd4_seek, .op_name = "OP_SEEK", }, + [OP_COPY_NOTIFY] = { + .op_func = (nfsd4op_func)nfsd4_copy_notify, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_COPY_NOTIFY", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_copy_notify_rsize, + }, };int nfsd4_max_reply (struct svc_rqst *rqstp,struct nfsd4_op *op)
fs/nfsd/nfs4xdr.c +126 -2
152 lines not on this page · raw message
fs/nfsd/xdr4.h +13 -0
@@ -530,6 +530,19 @@ struct nfsd4_seek {loff_t seek_pos; }; +struct nfsd4_copy_notify { +/* request */ +stateid_t cpn_src_stateid; +struct nl4_server cpn_dst; + +/* response */ +/* Note: cpn_src_stateid is used for cnr_stateid */ +u64 cpn_sec; +u32 cpn_nsec; +u32 cpn_nsrc; +struct nl4_server cpn_src[NFSD4_MAX_SSC_SRC ]; +}; +struct nfsd4_op {int opnum;__be32 status;
Signature · 1 line
1.8.3.1
PATCH 10/16 NFS add ca_source_server<> to COPY
In reply to [email protected]From: Andy Adamson <[email protected]>
Support only one source server address: the same address that the client and source server use.
--- fs/nfs/nfs42proc.c | 22 ++++++++++++++++++++-- fs/nfs/nfs42xdr.c | 27 +++++++++++++++++++++++++-- include/linux/nfs_xdr.h | 3 +++ 3 files changed, 48 insertions(+), 4 deletions(-)
fs/nfs/nfs42proc.c +20 -2
@@ -157,8 +157,12 @@ int nfs42_proc_deallocate(struct file *filep, loff_t offset, loff_t len)return err; } +/** +* Support one source address for now. +*/ static ssize_t nfs42_proc_copy (struct file *src,loff_t pos_src, -struct file *dst,loff_t pos_dst,size_t count) +struct file *dst,loff_t pos_dst,size_t count, +struct nfs42_copy_notify_res *cn_res) {struct nfs42_copy_args args = { .src_fh =NFS_FH (file_inode (src)),
@@ -176,6 +180,19 @@ static ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src,struct nfs_server *server =NFS_SERVER (file_inode (dst));int status; +if (cn_res) { + args.cp_nsrc =1 ;/* support for one NL4_NETADDR for now */ + args.cp_src[0 ].nl4_type = cn_res->cnr_src[0 ].nl4_type; + args.cp_src[0 ].u.nl4_addr = cn_res->cnr_src[0 ].u.nl4_addr; + +dprintk ("--> %s nl4_type %d cp_addr netid %d:%s uaddr %d:%s\n" , + __func__, args.cp_src[0 ].nl4_type, + args.cp_src[0 ].u.nl4_addr.na_netid_len, + args.cp_src[0 ].u.nl4_addr.na_netid, + args.cp_src[0 ].u.nl4_addr.na_uaddr_len, + args.cp_src[0 ].u.nl4_addr.na_uaddr); + } +if (!(server->caps &NFS_CAP_COPY ))return -ENOTSUPP ;
@@ -297,7 +314,8 @@ ssize_t nfs42_do_copy(struct file *src, loff_t pos_src, struct file *dst,if (ret) goto out_err; } - ret =nfs42_proc_copy (src, pos_src, dst, pos_dst, count); +/* NULL nfs42_copy_notify_res pointer signals intra ssc */ + ret =nfs42_proc_copy (src, pos_src, dst, pos_dst, count, cn_resp); out_err:kfree (cn_resp);
fs/nfs/nfs42xdr.c +25 -2
@@ -20,7 +20,10 @@#define encode_copy_maxsz (op_encode_hdr_maxsz + \ XDR_QUADLEN(NFS4_STATEID_SIZE) + \ XDR_QUADLEN(NFS4_STATEID_SIZE) + \ - 2 + 2 + 2 + 1 + 1 + 1) + 2 + 2 + 2 + 1 + 1 + 1 +\ + 1 +/* One cnr_source_server */ \ + 1 + /* nl4_type */ \ + 1 + XDR_QUADLEN(NFS4_OPAQUE_LIMIT)) #define decode_copy_maxsz (op_decode_hdr_maxsz + \ NFS42_WRITE_RES_SIZE + \ 1 /* cr_consecutive */ + \
@@ -136,7 +139,27 @@ static void encode_copy(struct xdr_stream *xdr,encode_uint32 (xdr,1 );/* consecutive = true */ encode_uint32 (xdr,1 );/* synchronous = true */ -encode_uint32 (xdr,0 );/* src server list */ +encode_uint32 (xdr, args->cp_nsrc);/* set to 1 for inter-ssc*/ +if (args->cp_nsrc ==0 )/* intra-ssc */ +return ; + +encode_uint32 (xdr, args->cp_src[0 ].nl4_type); +/* Support one src server in list for now */ +switch (args->cp_src[0 ].nl4_type) { +case NL4_NAME : +case NL4_URL : +encode_string (xdr, args->cp_src[0 ].u.nl4_str_sz, + args->cp_src[0 ].u.nl4_str); +break ; +case NL4_NETADDR : +encode_string (xdr, args->cp_src[0 ].u.nl4_addr.na_netid_len, + args->cp_src[0 ].u.nl4_addr.na_netid); +encode_string (xdr, args->cp_src[0 ].u.nl4_addr.na_uaddr_len, + args->cp_src[0 ].u.nl4_addr.na_uaddr); +break ; +default : +WARN_ON_ONCE (1 ); + } } static void encode_copy_notify(struct xdr_stream *xdr,
include/linux/nfs_xdr.h +3 -0
@@ -1339,6 +1339,9 @@ struct nfs42_copy_args {u64 dst_pos;u64 count; +/* Support one source server */ +int cp_nsrc; +struct nl4_server cp_src[NFS42_MAX_SSC_SRC ]; };struct nfs42_write_res {
Signature · 1 line
1.8.3.1
PATCH 11/16 NFSD generalize nfsd4_compound_state flag names
In reply to [email protected]From: Andy Adamson <[email protected]>
Allow for sid_flag field non-stateid use.
--- fs/nfsd/nfs4proc.c | 8 ++++---- fs/nfsd/nfs4state.c | 7 ++++--- fs/nfsd/xdr4.h | 6 +++--- 3 files changed, 11 insertions(+), 10 deletions(-)
fs/nfsd/nfs4proc.c +4 -4
@@ -538,9 +538,9 @@ nfsd4_restorefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return nfserr_restorefh;fh_dup2 (&cstate->current_fh, &cstate->save_fh); -if (HAS_STATE_ID (cstate,SAVED_STATE_ID_FLAG )) { +if (HAS_CSTATE_FLAG (cstate,SAVED_STATE_ID_FLAG )) {memcpy (&cstate->current_stateid, &cstate->save_stateid,sizeof (stateid_t)); -SET_STATE_ID (cstate,CURRENT_STATE_ID_FLAG ); +SET_CSTATE_FLAG (cstate,CURRENT_STATE_ID_FLAG ); }return nfs_ok; }
@@ -553,9 +553,9 @@ nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return nfserr_nofilehandle;fh_dup2 (&cstate->save_fh, &cstate->current_fh); -if (HAS_STATE_ID (cstate,CURRENT_STATE_ID_FLAG )) { +if (HAS_CSTATE_FLAG (cstate,CURRENT_STATE_ID_FLAG )) {memcpy (&cstate->save_stateid, &cstate->current_stateid,sizeof (stateid_t)); -SET_STATE_ID (cstate,SAVED_STATE_ID_FLAG ); +SET_CSTATE_FLAG (cstate,SAVED_STATE_ID_FLAG ); }return nfs_ok; }
fs/nfsd/nfs4state.c +4 -3
@@ -6660,7 +6660,8 @@ nfs4_state_shutdown(void)static void get_stateid (struct nfsd4_compound_state *cstate,stateid_t *stateid) { -if (HAS_STATE_ID (cstate,CURRENT_STATE_ID_FLAG ) &&CURRENT_STATEID (stateid)) +if (HAS_CSTATE_FLAG (cstate,CURRENT_STATE_ID_FLAG ) && +CURRENT_STATEID (stateid))memcpy (stateid, &cstate->current_stateid,sizeof (stateid_t)); }
@@ -6669,14 +6670,14 @@ put_stateid(struct nfsd4_compound_state *cstate, stateid_t *stateid) {if (cstate->minorversion) {memcpy (&cstate->current_stateid, stateid,sizeof (stateid_t)); -SET_STATE_ID (cstate,CURRENT_STATE_ID_FLAG ); +SET_CSTATE_FLAG (cstate,CURRENT_STATE_ID_FLAG ); } }void clear_current_stateid (struct nfsd4_compound_state *cstate) { -CLEAR_STATE_ID (cstate,CURRENT_STATE_ID_FLAG ); +CLEAR_CSTATE_FLAG (cstate,CURRENT_STATE_ID_FLAG ); }/*
fs/nfsd/xdr4.h +3 -3
@@ -47,9 +47,9 @@#define CURRENT_STATE_ID_FLAG (1<<0)#define SAVED_STATE_ID_FLAG (1<<1) -#define SET_STATE_ID (c, f) ((c)->sid_flags |= (f)) -#define HAS_STATE_ID (c, f) ((c)->sid_flags & (f)) -#define CLEAR_STATE_ID (c, f) ((c)->sid_flags &= ~(f)) +#define SET_CSTATE_FLAG (c, f) ((c)->sid_flags |= (f)) +#define HAS_CSTATE_FLAG (c, f) ((c)->sid_flags & (f)) +#define CLEAR_CSTATE_FLAG (c, f) ((c)->sid_flags &= ~(f))struct nfsd4_compound_state {struct svc_fh current_fh;
Signature · 1 line
1.8.3.1
PATCH 12/16 NFSD: allow inter server COPY to have a STALE source server fh
In reply to [email protected]From: Andy Adamson <[email protected]>
--- fs/nfsd/nfs4proc.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++- fs/nfsd/nfs4xdr.c | 26 +++++++++++++++++++++++++- fs/nfsd/nfsd.h | 2 ++ fs/nfsd/xdr4.h | 4 ++++ 4 files changed, 81 insertions(+), 2 deletions(-)
fs/nfsd/nfs4proc.c +50 -1
@@ -36,6 +36,7 @@#include <linux/falloc.h> #include <linux/slab.h> #include <linux/sunrpc/addr.h> +#include <linux/module.h> #include "idmap.h" #include "cache.h"
@@ -512,11 +513,20 @@ static __be32 nfsd4_putfh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, struct nfsd4_putfh *putfh) { + __be32 ret; +fh_put (&cstate->current_fh); cstate->current_fh.fh_handle.fh_size = putfh->pf_fhlen;memcpy (&cstate->current_fh.fh_handle.fh_base, putfh->pf_fhval, putfh->pf_fhlen); -return fh_verify (rqstp, &cstate->current_fh,0 ,NFSD_MAY_BYPASS_GSS ); + ret =fh_verify (rqstp, &cstate->current_fh,0 ,NFSD_MAY_BYPASS_GSS ); +if (ret == nfserr_stale &&HAS_CSTATE_FLAG (cstate,NO_VERIFY_FH )) { +dprintk ("%s SET IS_STALE_FH\n" , __func__); +CLEAR_CSTATE_FLAG (cstate,NO_VERIFY_FH ); +SET_CSTATE_FLAG (cstate,IS_STALE_FH ); + ret =0 ; + } +return ret; } static __be32
@@ -549,6 +559,18 @@ static __be32 nfsd4_savefh(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, void *arg) { +/** +* This is either an inter COPY (most likely) or an intra COPY with a +* stale file handle. If the latter, nfsd4_copy will reset the PUTFH to +* return * nfserr_stale. No fh_dentry, just copy the file handle +* to use with the inter COPY READ. +* XXX For debug only: +*/ +if (HAS_CSTATE_FLAG (cstate,IS_STALE_FH )) { +dprintk ("%s IS_STALE_FH\n" , __func__); + cstate->save_fh = cstate->current_fh; +return nfs_ok; + }if (!cstate->current_fh.fh_dentry)return nfserr_nofilehandle;
@@ -1051,6 +1073,13 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,if (status)return status; +/* Intra copy source fh is stale */ +if (HAS_CSTATE_FLAG (cstate,IS_STALE_FH )) { +CLEAR_CSTATE_FLAG (cstate,IS_STALE_FH ); + cstate->status = nfserr_copy_stalefh; + goto out; + } + bytes =nfsd_copy_range (src, copy->cp_src_pos, dst, copy->cp_dst_pos, copy->cp_count);
@@ -1068,6 +1097,7 @@ nfsd4_copy(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,fput (src);fput (dst); +out:return status; }
@@ -1752,6 +1782,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate = &resp->cstate;struct svc_fh *current_fh = &cstate->current_fh;struct svc_fh *save_fh = &cstate->save_fh; +int i;__be32 status;svcxdr_init_encode (rqstp, resp);
@@ -1784,6 +1815,12 @@ nfsd4_proc_compound(struct svc_rqst *rqstp, goto encode_op; } +/* NFSv4.2 COPY source file handle may be from a different server */ +for (i =0 ; i < args->opcnt; i++) { + op = &args->ops[i]; +if (op->opnum ==OP_COPY ) +SET_CSTATE_FLAG (cstate,NO_VERIFY_FH ); + }while (!status && resp->opcnt < args->opcnt) { op = &args->ops[resp->opcnt++];
@@ -1803,6 +1840,9 @@ nfsd4_proc_compound(struct svc_rqst *rqstp, opdesc =OPDESC (op); +if (HAS_CSTATE_FLAG (cstate,IS_STALE_FH )) + goto call_op; +if (!current_fh->fh_dentry) {if (!(opdesc->op_flags &ALLOWED_WITHOUT_FH )) { op->status = nfserr_nofilehandle;
@@ -1837,6 +1877,7 @@ nfsd4_proc_compound(struct svc_rqst *rqstp,if (opdesc->op_get_currentstateid) opdesc->op_get_currentstateid (cstate, &op->u); +call_op: op->status = opdesc->op_func (rqstp, cstate, &op->u);if (!op->status) {
@@ -1857,6 +1898,14 @@ encode_op: status = op->status; goto out; } +/* Only from intra COPY */ +if (cstate->status == nfserr_copy_stalefh) { +dprintk ("%s NFS4.2 intra COPY stale src filehandle\n" , + __func__); + status = nfserr_stale; +nfsd4_adjust_encode (resp); + goto out; + }if (op->status == nfserr_replay_me) { op->replay = &cstate->replay_owner->so_replay;nfsd4_encode_replay (&resp->xdr, op);
fs/nfsd/nfs4xdr.c +25 -1
@@ -4507,15 +4507,28 @@ __be32 nfsd4_check_resp_size(struct nfsd4_compoundres *resp, u32 respsize)return nfserr_rep_too_big; } +/** Rewind the encoding to return nfserr_stale on the PUTFH +* in this failed Intra COPY compound +*/ +void +nfsd4_adjust_encode (struct nfsd4_compoundres *resp) +{ +__be32 *p; + + p = resp->cstate.putfh_errp; + *p++ = nfserr_stale; +} +void nfsd4_encode_operation (struct nfsd4_compoundres *resp,struct nfsd4_op *op) {struct xdr_stream *xdr = &resp->xdr;struct nfs4_stateowner *so = resp->cstate.replay_owner; +struct nfsd4_compound_state *cstate = &resp->cstate;struct svc_rqst *rqstp = resp->rqstp;int post_err_offset;nfsd4_enc encoder; -__be32 *p; +__be32 *p, *statp; p =xdr_reserve_space (xdr,8 );if (!p) {
@@ -4524,9 +4537,20 @@ nfsd4_encode_operation(struct nfsd4_compoundres *resp, struct nfsd4_op *op) } *p++ =cpu_to_be32 (op->opnum); post_err_offset = xdr->buf->len; + statp = p;if (op->opnum ==OP_ILLEGAL ) goto status; + +/** This is a COPY compound with a stale source server file handle. +* If OP_COPY processing determines that this is an intra server to +* server COPY, then this PUTFH should return nfserr_ stale so the +* putfh_errp will be set to nfserr_stale. If this is an inter server +* to server COPY, ignore the nfserr_stale. +*/ +if (op->opnum ==OP_PUTFH &&HAS_CSTATE_FLAG (cstate,IS_STALE_FH )) + cstate->putfh_errp = statp; +BUG_ON (op->opnum <0 || op->opnum >=ARRAY_SIZE (nfsd4_enc_ops) || !nfsd4_enc_ops[op->opnum]); encoder = nfsd4_enc_ops[op->opnum];
fs/nfsd/nfsd.h +2 -0
@@ -267,6 +267,8 @@ void nfsd_lockd_shutdown(void);#define nfserr_replay_me cpu_to_be32(11001)/* nfs41 replay detected */ #define nfserr_replay_cache cpu_to_be32(11002) +/* nfs42 intra copy failed with nfserr_stale */ +#define nfserr_copy_stalefh cpu_to_be32(1103)/* Check for dir entries '.' and '..' */ #define isdotent (n, l) (l < 3 && n[0] == '.' && (l == 1 || n[1] == '.'))
fs/nfsd/xdr4.h +4 -0
@@ -46,6 +46,8 @@#define CURRENT_STATE_ID_FLAG (1<<0)#define SAVED_STATE_ID_FLAG (1<<1) +#define NO_VERIFY_FH (1<<2) +#define IS_STALE_FH (1<<3)#define SET_CSTATE_FLAG (c, f) ((c)->sid_flags |= (f))#define HAS_CSTATE_FLAG (c, f) ((c)->sid_flags & (f))
@@ -63,6 +65,7 @@ struct nfsd4_compound_state {size_t iovlen;u32 minorversion;__be32 status; +__be32 *putfh_errp;stateid_t current_stateid;stateid_t save_stateid;/* to indicate current and saved state id presents */
@@ -692,6 +695,7 @@ int nfs4svc_decode_compoundargs(struct svc_rqst *, __be32 *,int nfs4svc_encode_compoundres (struct svc_rqst *,__be32 *,struct nfsd4_compoundres *);__be32 nfsd4_check_resp_size (struct nfsd4_compoundres *,u32 ); +void nfsd4_adjust_encode (struct nfsd4_compoundres *);void nfsd4_encode_operation (struct nfsd4_compoundres *,struct nfsd4_op *);void nfsd4_encode_replay (struct xdr_stream *xdr,struct nfsd4_op *op); __be32 nfsd4_encode_fattr_to_buf(__be32 **p, int words,
Signature · 1 line
1.8.3.1
PATCH 13/16 NFSD add nfs4 inter ssc to nfsd4_copy
In reply to [email protected]From: Andy Adamson <[email protected]>
Given a universal address, mount the source server from the destination server. Use an internal mount. Call the NFS client nfs42_ssc_open to obtain the NFS struct file suitable for nfsd_copy_range.
Add Kconfig dependencies for inter server to server copy
--- fs/nfsd/Kconfig | 10 ++ fs/nfsd/nfs4proc.c | 268 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 269 insertions(+), 9 deletions(-)
fs/nfsd/Kconfig +10 -0
@@ -93,6 +93,16 @@ config NFSD_PNFS If unsure, say N. +config NFSD_V4_2_INTER_SSC + bool "NFSv4.2 inter server to server COPY" + depends on NFSD_V4 && NFS_V4_1 && NFS_V4_2 + help + This option enables support for NFSv4.2 inter server to + server copy where the destination server calls the NFSv4.2 + client to read the data to copy from the source server. + + If unsure, say N. + config NFSD_V4_SECURITY_LABEL bool "Provide Security Label support for NFSv4 server" depends on NFSD_V4 && SECURITY
fs/nfsd/nfs4proc.c +259 -9
290 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 14/16 NFS in copy use stateid returned by copy_notify
In reply to [email protected]From: Olga Kornievskaia <[email protected]>
--- fs/nfs/nfs42proc.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
fs/nfs/nfs42proc.c +6 -4
@@ -191,15 +191,17 @@ static ssize_t nfs42_proc_copy(struct file *src, loff_t pos_src, args.cp_src[0 ].u.nl4_addr.na_netid, args.cp_src[0 ].u.nl4_addr.na_uaddr_len, args.cp_src[0].u.nl4_addr.na_uaddr); + +nfs4_stateid_copy (&args.src_stateid, &cn_res->cnr_stateid); + }else { + status =nfs42_set_rw_stateid (&args.src_stateid, src,FMODE_READ ); +if (status) +return status; }if (!(server->caps &NFS_CAP_COPY ))return -ENOTSUPP ; - status =nfs42_set_rw_stateid (&args.src_stateid, src,FMODE_READ ); -if (status) -return status; - status =nfs42_set_rw_stateid (&args.dst_stateid, dst,FMODE_WRITE );if (status)return status;
Signature · 1 line
1.8.3.1
PATCH 15/16 NFS always use openstateid in COPY_NOTIFY
In reply to [email protected]From: Olga Kornievskaia <[email protected]>
--- fs/nfs/nfs42proc.c | 14 ++++++++++---- fs/nfs/nfs4_fs.h | 2 +- fs/nfs/nfs4state.c | 2 +- 3 files changed, 12 insertions(+), 6 deletions(-)
fs/nfs/nfs42proc.c +10 -4
@@ -39,6 +39,15 @@ static int nfs42_set_rw_stateid(nfs4_stateid *dst, struct file *file,return ret; } +static void nfs42_set_open_stateid (nfs4_stateid *dst,struct file *file) +{ +struct nfs_open_context *open; + + open =get_nfs_open_context (nfs_file_open_context (file)); +nfs4_copy_open_stateid (dst, open->state); +put_nfs_open_context (open); +} +static void nfs42_set_netaddr (struct file *file_out,struct nfs42_netaddr *naddr) {
@@ -272,10 +281,7 @@ static int nfs42_proc_copy_notify(struct file *src, struct file *dst, args->cna_src_fh =NFS_FH (file_inode (src)), args->cna_dst.nl4_type =NL4_NETADDR ;nfs42_set_netaddr (src, &args->cna_dst.u.nl4_addr); - - status =nfs42_set_rw_stateid (&args->cna_src_stateid, src,FMODE_READ ); -if (status) -return status; +nfs42_set_open_stateid (&args->cna_src_stateid, src); msg.rpc_argp = args; status = nfs4_call_sync(src_server->client, src_server, &msg,
fs/nfs/nfs4_fs.h +1 -1
@@ -447,7 +447,7 @@ extern void nfs4_put_lock_state(struct nfs4_lock_state *lsp);extern int nfs4_set_lock_state (struct nfs4_state *state,struct file_lock *fl);extern int nfs4_select_rw_stateid (nfs4_stateid *,struct nfs4_state *,fmode_t ,const struct nfs_lockowner *); - +extern void nfs4_copy_open_stateid (nfs4_stateid *,struct nfs4_state *);extern struct nfs_seqid *nfs_alloc_seqid (struct nfs_seqid_counter *counter,gfp_t gfp_mask);extern int nfs_wait_on_sequence (struct nfs_seqid *seqid,struct rpc_task *task);extern void nfs_increment_open_seqid (int status,struct nfs_seqid *seqid);
fs/nfs/nfs4state.c +1 -1
@@ -967,7 +967,7 @@ out:return ret; } -static void nfs4_copy_open_stateid (nfs4_stateid *dst,struct nfs4_state *state) +void nfs4_copy_open_stateid (nfs4_stateid *dst,struct nfs4_state *state) {const nfs4_stateid *src;int seq;
Signature · 1 line
1.8.3.1
PATCH 16/16 NFSD: extra stateid checking in read for interserver copy
In reply to [email protected]From: Olga Kornievskaia <[email protected]>
Expand stateid structure to store a mark signifying this stateid is a COPY stateid and special to pass stateid check on READ operation coming from a different clientid.
In COPY_NOTIFY, mark the stateid stored under the clientid as 'special' by setting boolean is_copy field to true.
If we received a READ and check for the stateid fails, we need to look for the stateid under all other clientid structures and their corresponding stateid lists. Make sure the stateid was previously marked for use READ from a COPY operation.
--- fs/nfsd/nfs4proc.c | 15 ++++++++++++++- fs/nfsd/nfs4state.c | 30 +++++++++++++++++++++++------- fs/nfsd/state.h | 8 ++++++++ 3 files changed, 45 insertions(+), 8 deletions(-)
fs/nfsd/nfs4proc.c +14 -1
@@ -1398,6 +1398,7 @@ nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,struct file *src =NULL ;struct nfs42_netaddr *naddr;struct nfsd_net *nn =net_generic (SVC_NET (rqstp), nfsd_net_id); +struct nfs4_stid *stid =NULL ; status = nfs4_preprocess_stateid_op(rqstp, cstate, &cstate->current_fh,
@@ -1412,7 +1413,19 @@ nfsd4_copy_notify(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,* connections from the destination e.g. what is returned in cpn_src, * to verify READ from dest server. */ - +/* mark the original open stateid special */ + status = nfserr_bad_stateid; + stid =find_stateid_by_type (cstate->session->se_client, + &cp_notify->cpn_src_stateid,NFS4_DELEG_STID |NFS4_OPEN_STID | +NFS4_LOCK_STID ); +if (stid) { +/* cnp_src_stateid is used for reply cnr_stateid */ + stid->is_copy =1 ; +nfs4_put_stid (stid); + }else { +dprintk ("NFSD: %s can't find cpn_src_stateid\n" , __func__); + goto out; + }/** * For now, only return one source server address, the address used * by the client in the static cpn_src.
fs/nfsd/nfs4state.c +23 -7
@@ -1809,12 +1809,6 @@ same_verf(nfs4_verifier *v1, nfs4_verifier *v2)return 0 ==memcmp (v1->data, v2->data,sizeof (v1->data)); } -static int -same_clid (clientid_t *cl1,clientid_t *cl2) -{ -return (cl1->cl_boot == cl2->cl_boot) && (cl1->cl_id == cl2->cl_id); -} -static bool groups_equal (struct group_info *g1,struct group_info *g2) {int i;
@@ -1916,7 +1910,7 @@ find_stateid_locked(struct nfs4_client *cl, stateid_t *t)return ret; } -static struct nfs4_stid * +struct nfs4_stid *find_stateid_by_type (struct nfs4_client *cl,stateid_t *t,char typemask) {struct nfs4_stid *s;
@@ -4670,6 +4664,28 @@ nfs4_preprocess_stateid_op(struct svc_rqst *rqstp, status =nfsd4_lookup_stateid (cstate, stateid,NFS4_DELEG_STID |NFS4_OPEN_STID |NFS4_LOCK_STID , &s, nn); +if (status == nfserr_bad_stateid) { +/** if clientid in rd_stateid is different from current, +* find corresponding clientid and look for stateid there +*/ +struct nfs4_client *clp =NULL ; + +spin_lock (&nn->client_lock); +list_for_each_entry (clp, &nn->client_lru, cl_lru) { +if (same_clid (&clp->cl_clientid, + &stateid->si_opaque.so_clid)) { + + s =find_stateid_by_type (clp, stateid, +NFS4_DELEG_STID |NFS4_OPEN_STID | +NFS4_LOCK_STID ); +if (s && s->is_copy) +dprintk ("%s COPY stateid\n" , __func__); + status = nfs_ok; +break ; + } + } +spin_unlock (&nn->client_lock); + }if (status)return status; status = check_stateid_generation(stateid, &s->sc_stateid,
fs/nfsd/state.h +8 -0
@@ -94,6 +94,7 @@ struct nfs4_stid {#define NFS4_REVOKED_DELEG_STID 16#define NFS4_CLOSED_DELEG_STID 32#define NFS4_LAYOUT_STID 64 +bool is_copy;unsigned char sc_type;stateid_t sc_stateid;struct nfs4_client *sc_client;
@@ -583,6 +584,13 @@ enum nfsd4_cb_op {struct nfsd4_compound_state ;struct nfsd_net ; +static inline int same_clid (clientid_t *cl1,clientid_t *cl2) +{ +return (cl1->cl_boot == cl2->cl_boot) && (cl1->cl_id == cl2->cl_id); +} + +extern struct nfs4_stid *find_stateid_by_type (struct nfs4_client *cl, +stateid_t *t,char typemask);extern __be32 nfs4_preprocess_stateid_op (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate,struct svc_fh *fhp,stateid_t *stateid,int flags,struct file **filp,bool *tmp_file);
Signature · 1 line
1.8.3.1
Apologies for the slow response. I'm waiting to review this carefully until the simpler (single-server) case is done.
Eventually I'd be curious for any performance results, specifically:
- when copying a large file, is the copy able to use all the available server<->server bandwidth?
- when copying a small file, how does COPY compare to a read and a write?
--b.
Quoted 77 lines · On Fri, Sep 04, 2015 at 01:29:22PM -0400, [email protected] wrote:
From: Andy Adamson <[email protected]>
Version 2:
Fixed xdr issues.
Fixed stateid issues
Added lease time tocopy_notifyreturnVersion 1:
These patches add client and server support for NFSv4.2 inter-server to server copy offload. These patches build upon Anna Schumaker's COPY Operation patches which are in turn built upon Zack Brown's VFSvfs_copy_file_rangesyscall. Additional functionality is added to the COPY operation past what Anna added. TheCOPY_NOTIFYoperation is added.The first three patches change Zach Brown's
vfs_copy_file_rangeimplementation. The first two patches relax the check for only performing copy offload between files with the same mount and super block, moving that check into a helper function that can be called (e.g. BTRFS) in a file systemscopy_file_rangefunction. The third patch changesvfs_copy_file_rangeto call the destination file (file_out)copy_file_rangef_opsproceedure instead of thefile_inversion.The remaining patches implement the
COPY_NOTIFYoperation and the inter-ssc changes to the COPY operation. One source server usingNL4_NETADDRis currently supported. I used the test program from Anna's COPY operation patch set.Andy Adamson (12):
VFS: Separate cross fs check fromvfs_copy_file_range
BTRFS: Use VFS copy offload helper
VFS SQUASH usefile_outinstead offile_inforcopy_file_rangeNFS add same file check and flush source and destination for COPY NFS add inter ssc functions to nfs42proc
NFS addCOPY_NOTIFYoperation
NFSD add ca_source_server<> to COPY
NFSD addCOPY_NOTIFYoperation
NFS add ca_source_server<> to COPY
NFSD generalizenfsd4_compound_stateflag names
NFSD: allow inter server COPY to have a STALE source server fh NFSD add nfs4 inter ssc tonfsd4_copyOlga Kornievskaia (4):
NFS COPY xdr changes
NFS in copy use stateid returned bycopy_notifyNFS always use openstateid inCOPY_NOTIFY
NFSD: extra stateid checking in read for interserver copyfs/btrfs/ioctl.c | 3 + fs/nfs/nfs42.h | 2 +- fs/nfs/nfs42proc.c | 238 +++++++++++++++++++++- fs/nfs/nfs42xdr.c | 213 +++++++++++++++++++- fs/nfs/nfs4_fs.h | 9 +- fs/nfs/nfs4file.c | 17 +- fs/nfs/nfs4proc.c | 8 +- fs/nfs/nfs4state.c | 2 +- fs/nfs/nfs4xdr.c | 1 + fs/nfsd/Kconfig | 10 + fs/nfsd/nfs4proc.c | 456 +++++++++++++++++++++++++++++++++++++++++- fs/nfsd/nfs4state.c | 37 +++- fs/nfsd/nfs4xdr.c | 203 ++++++++++++++++++- fs/nfsd/nfsd.h | 2 + fs/nfsd/state.h | 8 + fs/nfsd/xdr4.h | 29 ++- fs/read_write.c | 38 ++-- include/linux/fs.h | 1 + include/linux/nfs4.h | 7 + include/linux/nfs4intercopy.h | 39 ++++ include/linux/nfs_fs_sb.h | 1 + include/linux/nfs_xdr.h | 28 +++ 22 files changed, 1289 insertions(+), 63 deletions(-) create mode 100644 include/linux/nfs4intercopy.hSignature · 1 line
1.8.3.1