NFSD: Add support for WRITE_PLUS and SEEK
Subsystems (3)
cover NFSD: Add support for WRITE_PLUS and SEEK
This patch series adds server support for the NFS v4.2 operations WRITE_PLUS and SEEK. The first few patches fix up error codes to match the current draft of the spec and then add in the NFS v4.2 decode operations array so later patches can focus on only adding in the new operations.
Patch 3 (Add WRITE_PLUS support for hole punches) exports the do_fallocate() function from fs/open.c. Should this be its own patch submitted somewhere else? Is there a better way to get into the VFS fallocate routines?
The final patch is not meant to be applied, and is only included as an example of what I did to add offload handling to the client.
Questions, comments and thoughts are appreciated!
Anna
Anna Schumaker (4):
NFSD: Update error codes
NFSD: Create nfs v4.2 decode ops
NFSD: Add WRITE_PLUS support for hole punches
NFSD: Implement SEEK
fs/nfsd/nfs4proc.c | 89 +++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++- fs/nfsd/nfsd.h | 2 +- fs/nfsd/vfs.c | 14 ++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 39 +++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 21 +++++- 8 files changed, 342 insertions(+), 4 deletions(-)
Signature · 1 line
1.8.4.1
PATCH 1/4 NFSD: Update error codes
Recent NFS v4.2 drafts have removed NFS4ERR_METADATA_NOTSUPP and reassigned the error code to NFS4ERR_UNION_NOTSUPP.
--- fs/nfsd/nfsd.h | 2 +- include/linux/nfs4.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/nfsd/nfsd.h +1 -1
@@ -244,7 +244,7 @@ void nfsd_lockd_shutdown(void);#define nfserr_deleg_revoked cpu_to_be32(NFS4ERR_DELEG_REVOKED)#define nfserr_partner_notsupp cpu_to_be32(NFS4ERR_PARTNER_NOTSUPP)#define nfserr_partner_no_auth cpu_to_be32(NFS4ERR_PARTNER_NO_AUTH) -#define nfserr_metadata_notsupp cpu_to_be32(NFS4ERR_METADATA_NOTSUPP) +#define nfserr_union_notsupp cpu_to_be32(NFS4ERR_UNION_NOTSUPP)#define nfserr_offload_denied cpu_to_be32(NFS4ERR_OFFLOAD_DENIED)#define nfserr_wrong_lfs cpu_to_be32(NFS4ERR_WRONG_LFS)#define nfserr_badlabel cpu_to_be32(NFS4ERR_BADLABEL)
include/linux/nfs4.h +1 -1
@@ -232,7 +232,7 @@ enum nfsstat4 {/* nfs42 */ NFS4ERR_PARTNER_NOTSUPP =10088 ,NFS4ERR_PARTNER_NO_AUTH =10089 , -NFS4ERR_METADATA_NOTSUPP =10090 , +NFS4ERR_UNION_NOTSUPP =10090 ,NFS4ERR_OFFLOAD_DENIED =10091 ,NFS4ERR_WRONG_LFS =10092 , NFS4ERR_BADLABEL = 10093,
Signature · 1 line
1.8.4.1
PATCH 2/4 NFSD: Create nfs v4.2 decode ops
In reply to Anna SchumakerI'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
--- fs/nfsd/nfs4xdr.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++- include/linux/nfs4.h | 11 +++++++ 2 files changed, 95 insertions(+), 1 deletion(-)
fs/nfsd/nfs4xdr.c +84 -1
@@ -1599,6 +1599,78 @@ static nfsd4_dec nfsd41_dec_ops[] = { [OP_RECLAIM_COMPLETE] = (nfsd4_dec)nfsd4_decode_reclaim_complete, }; +static nfsd4_dec nfsd42_dec_ops[] = { + [OP_ACCESS ] = (nfsd4_dec )nfsd4_decode_access, + [OP_CLOSE ] = (nfsd4_dec )nfsd4_decode_close, + [OP_COMMIT ] = (nfsd4_dec )nfsd4_decode_commit, + [OP_CREATE ] = (nfsd4_dec )nfsd4_decode_create, + [OP_DELEGPURGE ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_DELEGRETURN ] = (nfsd4_dec )nfsd4_decode_delegreturn, + [OP_GETATTR ] = (nfsd4_dec )nfsd4_decode_getattr, + [OP_GETFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_LINK ] = (nfsd4_dec )nfsd4_decode_link, + [OP_LOCK ] = (nfsd4_dec )nfsd4_decode_lock, + [OP_LOCKT ] = (nfsd4_dec )nfsd4_decode_lockt, + [OP_LOCKU ] = (nfsd4_dec )nfsd4_decode_locku, + [OP_LOOKUP ] = (nfsd4_dec )nfsd4_decode_lookup, + [OP_LOOKUPP ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_NVERIFY ] = (nfsd4_dec )nfsd4_decode_verify, + [OP_OPEN ] = (nfsd4_dec )nfsd4_decode_open, + [OP_OPENATTR ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OPEN_CONFIRM ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OPEN_DOWNGRADE ] = (nfsd4_dec )nfsd4_decode_open_downgrade, + [OP_PUTFH ] = (nfsd4_dec )nfsd4_decode_putfh, + [OP_PUTPUBFH ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_PUTROOTFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_READ ] = (nfsd4_dec )nfsd4_decode_read, + [OP_READDIR ] = (nfsd4_dec )nfsd4_decode_readdir, + [OP_READLINK ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_REMOVE ] = (nfsd4_dec )nfsd4_decode_remove, + [OP_RENAME ] = (nfsd4_dec )nfsd4_decode_rename, + [OP_RENEW ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_RESTOREFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_SAVEFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_SECINFO ] = (nfsd4_dec )nfsd4_decode_secinfo, + [OP_SETATTR ] = (nfsd4_dec )nfsd4_decode_setattr, + [OP_SETCLIENTID ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SETCLIENTID_CONFIRM ]= (nfsd4_dec )nfsd4_decode_notsupp, + [OP_VERIFY ] = (nfsd4_dec )nfsd4_decode_verify, + [OP_WRITE ] = (nfsd4_dec )nfsd4_decode_write, + [OP_RELEASE_LOCKOWNER ] = (nfsd4_dec )nfsd4_decode_notsupp, + +/* new operations for NFSv4.1 */ + [OP_BACKCHANNEL_CTL ] = (nfsd4_dec )nfsd4_decode_backchannel_ctl, + [OP_BIND_CONN_TO_SESSION ]= (nfsd4_dec )nfsd4_decode_bind_conn_to_session, + [OP_EXCHANGE_ID ] = (nfsd4_dec )nfsd4_decode_exchange_id, + [OP_CREATE_SESSION ] = (nfsd4_dec )nfsd4_decode_create_session, + [OP_DESTROY_SESSION ] = (nfsd4_dec )nfsd4_decode_destroy_session, + [OP_FREE_STATEID ] = (nfsd4_dec )nfsd4_decode_free_stateid, + [OP_GET_DIR_DELEGATION ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_GETDEVICEINFO ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_GETDEVICELIST ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTCOMMIT ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTGET ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTRETURN ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SECINFO_NO_NAME ] = (nfsd4_dec )nfsd4_decode_secinfo_no_name, + [OP_SEQUENCE ] = (nfsd4_dec )nfsd4_decode_sequence, + [OP_SET_SSV ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_TEST_STATEID ] = (nfsd4_dec )nfsd4_decode_test_stateid, + [OP_WANT_DELEGATION ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_DESTROY_CLIENTID ] = (nfsd4_dec )nfsd4_decode_destroy_clientid, + [OP_RECLAIM_COMPLETE ] = (nfsd4_dec )nfsd4_decode_reclaim_complete, + +/* new operations for NFSv4.2 */ + [OP_COPY ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_ABORT ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_COPY_NOTIFY ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_REVOKE ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_STATUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_WRITE_PLUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_READ_PLUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SEEK ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_IO_ADVISE ] = (nfsd4_dec )nfsd4_decode_notsupp, +}; +struct nfsd4_minorversion_ops {nfsd4_dec *decoders;int nops;
@@ -1607,7 +1679,7 @@ struct nfsd4_minorversion_ops {static struct nfsd4_minorversion_ops nfsd4_minorversion[] = { [0 ] = { nfsd4_dec_ops,ARRAY_SIZE (nfsd4_dec_ops) }, [1 ] = { nfsd41_dec_ops,ARRAY_SIZE (nfsd41_dec_ops) }, - [2 ] = { nfsd41_dec_ops,ARRAY_SIZE (nfsd41_dec_ops) }, + [2 ] = { nfsd42_dec_ops,ARRAY_SIZE (nfsd42_dec_ops) }, }; static __be32
@@ -3591,6 +3663,17 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_WANT_DELEGATION] = (nfsd4_enc)nfsd4_encode_noop, [OP_DESTROY_CLIENTID] = (nfsd4_enc)nfsd4_encode_noop, [OP_RECLAIM_COMPLETE] = (nfsd4_enc)nfsd4_encode_noop, + +/* NFSv4.2 operations */ + [OP_COPY] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_ABORT] = (nfsd4_enc)nfsd4_encode_noop, + [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, + [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop, };/*
include/linux/nfs4.h +11 -0
@@ -110,6 +110,17 @@ enum nfs_opnum4 {OP_DESTROY_CLIENTID =57 ,OP_RECLAIM_COMPLETE =58 , +/* nfs42 */ +OP_COPY =59 , +OP_OFFLOAD_ABORT =60 , +OP_COPY_NOTIFY =61 , +OP_OFFLOAD_REVOKE =62 , +OP_OFFLOAD_STATUS =63 , +OP_WRITE_PLUS =64 , +OP_READ_PLUS =65 , +OP_SEEK =66 , +OP_IO_ADVISE =67 , + OP_ILLEGAL = 10044, };
Signature · 1 line
1.8.4.1
Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode ops
Quoted 2 lines · Anna Schumaker
I'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
Makes sense.
But: now we're duplicating the list of 4.0 op decoders 3 times and the 4.1 ops twice. We'll never need different decoders for different minorversions (worst case we can test for the minorversion in the decoder if necessary).
I wonder if there's a better way to organize this.... Maybe something more like a single array with
[OP_SETCLIENTID] = {
.op_decode = (nfsd4_dec)nfsd4_decode_access,
.op_unsupported_since_version = 1,
}
...
[OP_EXCHANGE_ID] = {
.op_decode = (nfsd4_dec)nfsd4_decode_exchange_id,
.op_first_supported_in_version = 1,
}?
--b.
Quoted 142 lines
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4xdr.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++- include/linux/nfs4.h | 11 +++++++ 2 files changed, 95 insertions(+), 1 deletion(-)fs/nfsd/nfs4xdr.c +84
-1@@ -1599,6 +1599,78 @@ static nfsd4_dec nfsd41_dec_ops[] = { [OP_RECLAIM_COMPLETE] = (nfsd4_dec)nfsd4_decode_reclaim_complete, }; +static nfsd4_dec nfsd42_dec_ops[] = { + [OP_ACCESS ] = (nfsd4_dec )nfsd4_decode_access, + [OP_CLOSE ] = (nfsd4_dec )nfsd4_decode_close, + [OP_COMMIT ] = (nfsd4_dec )nfsd4_decode_commit, + [OP_CREATE ] = (nfsd4_dec )nfsd4_decode_create, + [OP_DELEGPURGE ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_DELEGRETURN ] = (nfsd4_dec )nfsd4_decode_delegreturn, + [OP_GETATTR ] = (nfsd4_dec )nfsd4_decode_getattr, + [OP_GETFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_LINK ] = (nfsd4_dec )nfsd4_decode_link, + [OP_LOCK ] = (nfsd4_dec )nfsd4_decode_lock, + [OP_LOCKT ] = (nfsd4_dec )nfsd4_decode_lockt, + [OP_LOCKU ] = (nfsd4_dec )nfsd4_decode_locku, + [OP_LOOKUP ] = (nfsd4_dec )nfsd4_decode_lookup, + [OP_LOOKUPP ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_NVERIFY ] = (nfsd4_dec )nfsd4_decode_verify, + [OP_OPEN ] = (nfsd4_dec )nfsd4_decode_open, + [OP_OPENATTR ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OPEN_CONFIRM ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OPEN_DOWNGRADE ] = (nfsd4_dec )nfsd4_decode_open_downgrade, + [OP_PUTFH ] = (nfsd4_dec )nfsd4_decode_putfh, + [OP_PUTPUBFH ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_PUTROOTFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_READ ] = (nfsd4_dec )nfsd4_decode_read, + [OP_READDIR ] = (nfsd4_dec )nfsd4_decode_readdir, + [OP_READLINK ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_REMOVE ] = (nfsd4_dec )nfsd4_decode_remove, + [OP_RENAME ] = (nfsd4_dec )nfsd4_decode_rename, + [OP_RENEW ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_RESTOREFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_SAVEFH ] = (nfsd4_dec )nfsd4_decode_noop, + [OP_SECINFO ] = (nfsd4_dec )nfsd4_decode_secinfo, + [OP_SETATTR ] = (nfsd4_dec )nfsd4_decode_setattr, + [OP_SETCLIENTID ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SETCLIENTID_CONFIRM ]= (nfsd4_dec )nfsd4_decode_notsupp, + [OP_VERIFY ] = (nfsd4_dec )nfsd4_decode_verify, + [OP_WRITE ] = (nfsd4_dec )nfsd4_decode_write, + [OP_RELEASE_LOCKOWNER ] = (nfsd4_dec )nfsd4_decode_notsupp, + +/* new operations for NFSv4.1 */ + [OP_BACKCHANNEL_CTL ] = (nfsd4_dec )nfsd4_decode_backchannel_ctl, + [OP_BIND_CONN_TO_SESSION ]= (nfsd4_dec )nfsd4_decode_bind_conn_to_session, + [OP_EXCHANGE_ID ] = (nfsd4_dec )nfsd4_decode_exchange_id, + [OP_CREATE_SESSION ] = (nfsd4_dec )nfsd4_decode_create_session, + [OP_DESTROY_SESSION ] = (nfsd4_dec )nfsd4_decode_destroy_session, + [OP_FREE_STATEID ] = (nfsd4_dec )nfsd4_decode_free_stateid, + [OP_GET_DIR_DELEGATION ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_GETDEVICEINFO ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_GETDEVICELIST ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTCOMMIT ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTGET ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_LAYOUTRETURN ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SECINFO_NO_NAME ] = (nfsd4_dec )nfsd4_decode_secinfo_no_name, + [OP_SEQUENCE ] = (nfsd4_dec )nfsd4_decode_sequence, + [OP_SET_SSV ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_TEST_STATEID ] = (nfsd4_dec )nfsd4_decode_test_stateid, + [OP_WANT_DELEGATION ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_DESTROY_CLIENTID ] = (nfsd4_dec )nfsd4_decode_destroy_clientid, + [OP_RECLAIM_COMPLETE ] = (nfsd4_dec )nfsd4_decode_reclaim_complete, + +/* new operations for NFSv4.2 */ + [OP_COPY ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_ABORT ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_COPY_NOTIFY ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_REVOKE ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_OFFLOAD_STATUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_WRITE_PLUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_READ_PLUS ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_SEEK ] = (nfsd4_dec )nfsd4_decode_notsupp, + [OP_IO_ADVISE ] = (nfsd4_dec )nfsd4_decode_notsupp, +}; +struct nfsd4_minorversion_ops {nfsd4_dec *decoders;int nops;@@ -1607,7 +1679,7 @@ struct nfsd4_minorversion_ops {static struct nfsd4_minorversion_ops nfsd4_minorversion[] = { [0 ] = { nfsd4_dec_ops,ARRAY_SIZE (nfsd4_dec_ops) }, [1 ] = { nfsd41_dec_ops,ARRAY_SIZE (nfsd41_dec_ops) }, - [2 ] = { nfsd41_dec_ops,ARRAY_SIZE (nfsd41_dec_ops) }, + [2 ] = { nfsd42_dec_ops,ARRAY_SIZE (nfsd42_dec_ops) }, }; static __be32@@ -3591,6 +3663,17 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_WANT_DELEGATION] = (nfsd4_enc)nfsd4_encode_noop, [OP_DESTROY_CLIENTID] = (nfsd4_enc)nfsd4_encode_noop, [OP_RECLAIM_COMPLETE] = (nfsd4_enc)nfsd4_encode_noop, + +/* NFSv4.2 operations */ + [OP_COPY] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_ABORT] = (nfsd4_enc)nfsd4_encode_noop, + [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, + [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, + [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop, };/* include/linux/nfs4.h +11
-0@@ -110,6 +110,17 @@ enum nfs_opnum4 {OP_DESTROY_CLIENTID =57 ,OP_RECLAIM_COMPLETE =58 , +/* nfs42 */ +OP_COPY =59 , +OP_OFFLOAD_ABORT =60 , +OP_COPY_NOTIFY =61 , +OP_OFFLOAD_REVOKE =62 , +OP_OFFLOAD_STATUS =63 , +OP_WRITE_PLUS =64 , +OP_READ_PLUS =65 , +OP_SEEK =66 , +OP_IO_ADVISE =67 , + OP_ILLEGAL = 10044, };Signature · 1 line
1.8.4.1
RE: [PATCH 2/4] NFSD: Create nfs v4.2 decode ops
Quoted 32 lines
-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of J. Bruce Fields
Sent: Monday, October 28, 2013 4:54 PM
To: Schumaker, Bryan
Cc: [email protected]
Subject: Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode opsQuoted 2 lines · Anna Schumaker
I'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
Makes sense.
But: now we're duplicating the list of 4.0 op decoders 3 times and the 4.1 ops twice. We'll never need different decoders for different minorversions (worst case we can test for the minorversion in the decoder if necessary).
I wonder if there's a better way to organize this.... Maybe something more like a single array with
[OP_SETCLIENTID] = { .op_decode = (nfsd4_dec)nfsd4_decode_access, .op_unsupported_since_version = 1, } ... [OP_EXCHANGE_ID] = { .op_decode = (nfsd4_dec)nfsd4_decode_exchange_id, .op_first_supported_in_version = 1, }
Quoted 1 line
?
Is that really necessary? Why not just have a single array and have nfsd4_decode_clientid itself check the minor version?
Cheers
Trond
Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode ops
Quoted 32 lines · Trond
Quoted 32 lines
-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of J. Bruce Fields
Sent: Monday, October 28, 2013 4:54 PM
To: Schumaker, Bryan
Cc: [email protected]
Subject: Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode opsQuoted 2 lines · Anna Schumaker
I'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
Makes sense.
But: now we're duplicating the list of 4.0 op decoders 3 times and the 4.1 ops twice. We'll never need different decoders for different minorversions (worst case we can test for the minorversion in the decoder if necessary).
I wonder if there's a better way to organize this.... Maybe something more like a single array with
[OP_SETCLIENTID] = { .op_decode = (nfsd4_dec)nfsd4_decode_access, .op_unsupported_since_version = 1, } ... [OP_EXCHANGE_ID] = { .op_decode = (nfsd4_dec)nfsd4_decode_exchange_id, .op_first_supported_in_version = 1, }
Quoted 3 lines
Quoted 1 line
?
Is that really necessary? Why not just have a single array and have
nfsd4_decode_clientiditself check the minor version?
That'd work too. Every operation that's been introduced more recently than 4.0 or that's since been deprecated would need a version check at the top of its decoder. That'd be a dozen or so.
But the information has to go somewhere and perhaps that's more straightforward than sticking this in data.... OK, I'd be fine with that.
--b.
Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode ops
Quoted 37 lines · J. Bruce Fields
Quoted 32 lines · Trond
Quoted 32 lines
-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of J. Bruce Fields
Sent: Monday, October 28, 2013 4:54 PM
To: Schumaker, Bryan
Cc: [email protected]
Subject: Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode opsQuoted 2 lines · Anna Schumaker
I'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
Makes sense.
But: now we're duplicating the list of 4.0 op decoders 3 times and the 4.1 ops twice. We'll never need different decoders for different minorversions (worst case we can test for the minorversion in the decoder if necessary).
I wonder if there's a better way to organize this.... Maybe something more like a single array with
[OP_SETCLIENTID] = { .op_decode = (nfsd4_dec)nfsd4_decode_access, .op_unsupported_since_version = 1, } ... [OP_EXCHANGE_ID] = { .op_decode = (nfsd4_dec)nfsd4_decode_exchange_id, .op_first_supported_in_version = 1, }Quoted 3 lines
Quoted 1 line
?
Is that really necessary? Why not just have a single array and have
nfsd4_decode_clientiditself check the minor version?
Quoted 7 lines
That'd work too. Every operation that's been introduced more recently than 4.0 or that's since been deprecated would need a version check at the top of its decoder. That'd be a dozen or so.
But the information has to go somewhere and perhaps that's more straightforward than sticking this in data.... OK, I'd be fine with that.
Makes sense. Do you want me to put this in with this patch series or do it as something separate either before or after?
Quoted 2 lines
--b.
Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode ops
Quoted 42 lines · Anna Schumaker
Quoted 41 lines · J. Bruce Fields
Quoted 32 lines · Trond
Quoted 32 lines
-----Original Message-----
From: [email protected] [mailto:linux-nfs- [email protected]] On Behalf Of J. Bruce Fields
Sent: Monday, October 28, 2013 4:54 PM
To: Schumaker, Bryan
Cc: [email protected]
Subject: Re: [PATCH 2/4] NFSD: Create nfs v4.2 decode opsQuoted 2 lines · Anna Schumaker
I'm doing this in a separate patch to keep from putting in a lot of extra code when I go to add operations to the server for real.
Makes sense.
But: now we're duplicating the list of 4.0 op decoders 3 times and the 4.1 ops twice. We'll never need different decoders for different minorversions (worst case we can test for the minorversion in the decoder if necessary).
I wonder if there's a better way to organize this.... Maybe something more like a single array with
[OP_SETCLIENTID] = { .op_decode = (nfsd4_dec)nfsd4_decode_access, .op_unsupported_since_version = 1, } ... [OP_EXCHANGE_ID] = { .op_decode = (nfsd4_dec)nfsd4_decode_exchange_id, .op_first_supported_in_version = 1, }Quoted 3 lines
Quoted 1 line
?
Is that really necessary? Why not just have a single array and have
nfsd4_decode_clientiditself check the minor version?That'd work too. Every operation that's been introduced more recently than 4.0 or that's since been deprecated would need a version check at the top of its decoder. That'd be a dozen or so.
Quoted 6 lines
Quoted 3 lines
But the information has to go somewhere and perhaps that's more straightforward than sticking this in data.... OK, I'd be fine with that.
Makes sense. Do you want me to put this in with this patch series or do it as something separate either before or after?
Best would be to do it as something separate before the rest. So combine the minorversion 0 and minorversion 1 cases into 1 array first. And I can apply that patch now independent of the rest.
--b.
PATCH 3/4 NFSD: Add WRITE_PLUS support for hole punches
In reply to Anna SchumakerThis patch only implements DATA_CONTENT_HOLE support, but I tried to write it so other arms of the discriminated union can be added easily later.
This patch is missing support for decoding multiple requests on the same file. The way it's written now only the last range provided by the client will be written to.
--- fs/nfsd/nfs4proc.c | 45 ++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/vfs.c | 14 ++++++++++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 24 ++++++++++++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 8 ++++++- 7 files changed, 152 insertions(+), 3 deletions(-)
fs/nfsd/nfs4proc.c +45 -0
@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole (struct file *file,struct nfsd4_write_plus *writeplus, +struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ; +gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); + } + +return status; +} + +static __be32 +nfsd4_write_plus (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_write_plus *writeplus) +{ +struct file *file; +__be32 status; +struct net *net =SVC_NET (rqstp); + + status =nfs4_preprocess_stateid_op (net, cstate, &writeplus->wp_stateid, +WR_STATE , &file); +if (status) +return status; + +if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) +return nfsd4_write_plus_hole (file, writeplus, net); +return nfserr_union_notsupp; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME
@@ -1840,6 +1876,15 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid, .op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize, }, + +/* NFSv4.2 operations */ + [OP_WRITE_PLUS] = { + .op_func = (nfsd4op_func)nfsd4_write_plus, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_WRITE_PLUS", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, + }, };#ifdef NFSD_DEBUG
fs/nfsd/nfs4xdr.c +60 -2
86 lines not on this page · raw message
fs/nfsd/vfs.c +14 -0
@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/file.h> #include <linux/splice.h> +#include <linux/falloc.h> #include <linux/fcntl.h> #include <linux/namei.h> #include <linux/delay.h>
@@ -649,6 +650,19 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, }#endif +__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error); +} +#endif /* defined(CONFIG_NFSD_V4) */ #ifdef CONFIG_NFSD_V3
fs/nfsd/vfs.h +1 -0
@@ -57,6 +57,7 @@ __be32 nfsd4_set_nfs4_acl(struct svc_rqst *, struct svc_fh *,int nfsd4_get_nfs4_acl (struct svc_rqst *,struct dentry *,struct nfs4_acl **);__be32 nfsd4_set_nfs4_label (struct svc_rqst *,struct svc_fh *,struct xdr_netobj *); +__be32 nfsd4_vfs_fallocate (struct file *,bool ,loff_t ,loff_t );#endif /* CONFIG_NFSD_V4 */ __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct iattr *attrs,
fs/nfsd/xdr4.h +24 -0
@@ -430,6 +430,27 @@ struct nfsd4_reclaim_complete {u32 rca_one_fs; }; +struct nfsd42_write_res { +struct nfs4_stid *wr_stid; +u64 wr_bytes_written; +u32 wr_stable_how; +nfs4_verifier wr_verifier; +}; + +struct nfsd4_write_plus { +/* request */ +stateid_t wp_stateid; +u32 wp_stable_how; +u32 wp_data_length; +u32 wp_data_content; +u64 wp_offset; +u64 wp_length; +u32 wp_allocated; + +/* response */ +struct nfsd42_write_res wp_res; +}; +struct nfsd4_op {int opnum;__be32 status;
@@ -475,6 +496,9 @@ struct nfsd4_op {struct nfsd4_reclaim_complete reclaim_complete;struct nfsd4_test_stateid test_stateid;struct nfsd4_free_stateid free_stateid; + +/* NFSv4.2 */ +struct nfsd4_write_plus write_plus; } u;struct nfs4_replay * replay; };
fs/open.c +1 -0
@@ -278,6 +278,7 @@ int do_fallocate(struct file *file, int mode, loff_t offset, loff_t len)sb_end_write (inode->i_sb);return ret; } +EXPORT_SYMBOL_GPL (do_fallocate); SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len) {
include/linux/nfs4.h +7 -1
@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_RECLAIM_COMPLETE +#define LAST_NFS4_OP OP_WRITE_PLUSenum nfsstat4 {NFS4_OK =0 ,
@@ -552,4 +552,10 @@ struct nfs4_deviceid { char data[NFS4_DEVICEID4_SIZE]; }; +enum data_content4 { +NFS4_CONTENT_DATA =0 , +NFS4_CONTENT_APP_DATA_HOLE =1 , +NFS4_CONTENT_HOLE =2 , +}; +#endif
Signature · 1 line
1.8.4.1
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 3 lines · Anna Schumaker
This patch only implements
DATA_CONTENT_HOLEsupport, but I tried to write it so other arms of the discriminated union can be added easily later.
And there's an NFS4ERR_UNION_NOTSUPP to handle that case, OK. (Though the spec language is a little ambiguous:
If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with NFS4ERR_UNION_NOTSUPP.
So it's unclear whether we can return this in the case the client is *not* asking for a hole. Are we sure the NFS4_CONTENT_DATA case is optional? The language in 5.3 ("Instead a client should utlize READ_PLUS and WRITE_PLUS") also suggests the CONTENT_DATA case might be meant to be mandatory.)
Quoted 3 lines
This patch is missing support for decoding multiple requests on the same file. The way it's written now only the last range provided by the client will be written to.
But that doesn't seem to be a limitation anticipated by the spec, so I guess we need to fix that.
(Why is there an array, anyway? Wouldn't it work just as well to send multiple RPC's, or a single RPC with multiple WRITE_PLUS ops?)
Quoted 22 lines
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 45 ++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/vfs.c | 14 ++++++++++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 24 ++++++++++++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 8 ++++++- 7 files changed, 152 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;
Do we need to sync?
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1046
Quoted 4 lines
fs/nfsd/nfs4proc.c +4
-0+gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); + } + +return status;
Nit, but I like the usual idiom better in most cases:
if (status) return status; normal case here... return 0;
--b.
Quoted 258 lines
fs/nfsd/nfs4proc.c +29
-0+} + +static __be32 +nfsd4_write_plus (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_write_plus *writeplus) +{ +struct file *file; +__be32 status; +struct net *net =SVC_NET (rqstp); + + status =nfs4_preprocess_stateid_op (net, cstate, &writeplus->wp_stateid, +WR_STATE , &file); +if (status) +return status; + +if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) +return nfsd4_write_plus_hole (file, writeplus, net); +return nfserr_union_notsupp; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1840,6 +1876,15 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid, .op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize, }, + +/* NFSv4.2 operations */ + [OP_WRITE_PLUS] = { + .op_func = (nfsd4op_func)nfsd4_write_plus, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_WRITE_PLUS", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +60
-2@@ -1485,6 +1485,32 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str }static __be32 +nfsd4_decode_write_plus (struct nfsd4_compoundargs *argp, +struct nfsd4_write_plus *writeplus) +{ +DECODE_HEAD ; +unsigned int i; + + status =nfsd4_decode_stateid (argp, &writeplus->wp_stateid); +if (status) +return status; + +READ_BUF (8 ); +READ32 (writeplus->wp_stable_how); +READ32 (writeplus->wp_data_length); + +for (i =0 ; i < writeplus->wp_data_length; i++) { +READ_BUF (24 ); +READ32 (writeplus->wp_data_content); +READ64 (writeplus->wp_offset); +READ64 (writeplus->wp_length); +READ32 (writeplus->wp_allocated); + } + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1665,7 +1691,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_REVOKE] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp,@@ -3591,6 +3617,38 @@ nfsd4_encode_test_stateid(struct nfsd4_compoundres *resp, __be32 nfserr,return nfserr; } +static void +nfsd42_encode_write_res (struct nfsd4_compoundres *resp,struct nfsd42_write_res *write) +{ +__be32 *p; + +RESERVE_SPACE (4 ); + +if (write->wr_stid ==NULL ) { +WRITE32 (0 ); +ADJUST_ARGS (); + }else { +WRITE32 (1 ); +ADJUST_ARGS (); +nfsd4_encode_stateid (resp, &write->wr_stid->sc_stateid); + } + +RESERVE_SPACE (12 +NFS4_VERIFIER_SIZE ); +WRITE64 (write->wr_bytes_written); +WRITE32 (write->wr_stable_how); +WRITEMEM (write->wr_verifier.data,NFS4_VERIFIER_SIZE ); +ADJUST_ARGS (); +} + +static __be32 +nfsd4_encode_write_plus (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_write_plus *writeplus) +{ +if (!nfserr) +nfsd42_encode_write_res (resp, &writeplus->wp_res); +return nfserr; +} +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {@@ -3670,7 +3728,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop,fs/nfsd/vfs.c +14
-0@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/file.h> #include <linux/splice.h> +#include <linux/falloc.h> #include <linux/fcntl.h> #include <linux/namei.h> #include <linux/delay.h> @@ -649,6 +650,19 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, }#endif +__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error); +} +#endif /* defined(CONFIG_NFSD_V4) */ #ifdef CONFIG_NFSD_V3 fs/nfsd/vfs.h +1
-0@@ -57,6 +57,7 @@ __be32 nfsd4_set_nfs4_acl(struct svc_rqst *, struct svc_fh *,int nfsd4_get_nfs4_acl (struct svc_rqst *,struct dentry *,struct nfs4_acl **);__be32 nfsd4_set_nfs4_label (struct svc_rqst *,struct svc_fh *,struct xdr_netobj *); +__be32 nfsd4_vfs_fallocate (struct file *,bool ,loff_t ,loff_t );#endif /* CONFIG_NFSD_V4 */ __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct iattr *attrs,fs/nfsd/xdr4.h +24
-0@@ -430,6 +430,27 @@ struct nfsd4_reclaim_complete {u32 rca_one_fs; }; +struct nfsd42_write_res { +struct nfs4_stid *wr_stid; +u64 wr_bytes_written; +u32 wr_stable_how; +nfs4_verifier wr_verifier; +}; + +struct nfsd4_write_plus { +/* request */ +stateid_t wp_stateid; +u32 wp_stable_how; +u32 wp_data_length; +u32 wp_data_content; +u64 wp_offset; +u64 wp_length; +u32 wp_allocated; + +/* response */ +struct nfsd42_write_res wp_res; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -475,6 +496,9 @@ struct nfsd4_op {struct nfsd4_reclaim_complete reclaim_complete;struct nfsd4_test_stateid test_stateid;struct nfsd4_free_stateid free_stateid; + +/* NFSv4.2 */ +struct nfsd4_write_plus write_plus; } u;struct nfs4_replay * replay; };fs/open.c +1
-0@@ -278,6 +278,7 @@ int do_fallocate(struct file *file, int mode, loff_t offset, loff_t len)sb_end_write (inode->i_sb);return ret; } +EXPORT_SYMBOL_GPL (do_fallocate); SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len) {include/linux/nfs4.h +7
-1@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_RECLAIM_COMPLETE +#define LAST_NFS4_OP OP_WRITE_PLUSenum nfsstat4 {NFS4_OK =0 ,@@ -552,4 +552,10 @@ struct nfs4_deviceid { char data[NFS4_DEVICEID4_SIZE]; }; +enum data_content4 { +NFS4_CONTENT_DATA =0 , +NFS4_CONTENT_APP_DATA_HOLE =1 , +NFS4_CONTENT_HOLE =2 , +}; +#endif Signature · 1 line
1.8.4.1
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 14 lines · J. Bruce Fields
Quoted 3 lines · Anna Schumaker
This patch only implements
DATA_CONTENT_HOLEsupport, but I tried to write it so other arms of the discriminated union can be added easily later.And there's an
NFS4ERR_UNION_NOTSUPPto handle that case, OK. (Though the spec language is a little ambiguous:If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with
NFS4ERR_UNION_NOTSUPP.
Quoted 5 lines
So it's unclear whether we can return this in the case the client is *not* asking for a hole. Are we sure the
NFS4_CONTENT_DATAcase is optional? The language in 5.3 ("Instead a client should utlizeREAD_PLUSandWRITE_PLUS") also suggests theCONTENT_DATAcase might be meant to be mandatory.)
Fair enough. I wasn't planning on implementing the NFS4_CONTENT_DATA arm on the client right now, but I might be able to hack something together to test this on the server!
Quoted 10 lines
Quoted 4 lines
This patch is missing support for decoding multiple requests on the same file. The way it's written now only the last range provided by the client will be written to.
But that doesn't seem to be a limitation anticipated by the spec, so I guess we need to fix that.
(Why is there an array, anyway? Wouldn't it work just as well to send multiple RPC's, or a single RPC with multiple
WRITE_PLUSops?)
Yeah, I'll fix that up. The Linux client will send requests one at a time, so that's what I tested against.
Quoted 308 lines
Quoted 23 lines
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 45 ++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/vfs.c | 14 ++++++++++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 24 ++++++++++++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 8 ++++++- 7 files changed, 152 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1046
Quoted 4 lines
fs/nfsd/nfs4proc.c +4
-0+gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); + } + +return status;Nit, but I like the usual idiom better in most cases:
if (status) return status; normal case here... return 0;--b.
Quoted 258 lines
fs/nfsd/nfs4proc.c +29
-0+} + +static __be32 +nfsd4_write_plus (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_write_plus *writeplus) +{ +struct file *file; +__be32 status; +struct net *net =SVC_NET (rqstp); + + status =nfs4_preprocess_stateid_op (net, cstate, &writeplus->wp_stateid, +WR_STATE , &file); +if (status) +return status; + +if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) +return nfsd4_write_plus_hole (file, writeplus, net); +return nfserr_union_notsupp; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1840,6 +1876,15 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid, .op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize, }, + +/* NFSv4.2 operations */ + [OP_WRITE_PLUS] = { + .op_func = (nfsd4op_func)nfsd4_write_plus, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_WRITE_PLUS", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +60
-2@@ -1485,6 +1485,32 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str }static __be32 +nfsd4_decode_write_plus (struct nfsd4_compoundargs *argp, +struct nfsd4_write_plus *writeplus) +{ +DECODE_HEAD ; +unsigned int i; + + status =nfsd4_decode_stateid (argp, &writeplus->wp_stateid); +if (status) +return status; + +READ_BUF (8 ); +READ32 (writeplus->wp_stable_how); +READ32 (writeplus->wp_data_length); + +for (i =0 ; i < writeplus->wp_data_length; i++) { +READ_BUF (24 ); +READ32 (writeplus->wp_data_content); +READ64 (writeplus->wp_offset); +READ64 (writeplus->wp_length); +READ32 (writeplus->wp_allocated); + } + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1665,7 +1691,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_REVOKE] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp,@@ -3591,6 +3617,38 @@ nfsd4_encode_test_stateid(struct nfsd4_compoundres *resp, __be32 nfserr,return nfserr; } +static void +nfsd42_encode_write_res (struct nfsd4_compoundres *resp,struct nfsd42_write_res *write) +{ +__be32 *p; + +RESERVE_SPACE (4 ); + +if (write->wr_stid ==NULL ) { +WRITE32 (0 ); +ADJUST_ARGS (); + }else { +WRITE32 (1 ); +ADJUST_ARGS (); +nfsd4_encode_stateid (resp, &write->wr_stid->sc_stateid); + } + +RESERVE_SPACE (12 +NFS4_VERIFIER_SIZE ); +WRITE64 (write->wr_bytes_written); +WRITE32 (write->wr_stable_how); +WRITEMEM (write->wr_verifier.data,NFS4_VERIFIER_SIZE ); +ADJUST_ARGS (); +} + +static __be32 +nfsd4_encode_write_plus (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_write_plus *writeplus) +{ +if (!nfserr) +nfsd42_encode_write_res (resp, &writeplus->wp_res); +return nfserr; +} +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {@@ -3670,7 +3728,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop,fs/nfsd/vfs.c +14
-0@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/file.h> #include <linux/splice.h> +#include <linux/falloc.h> #include <linux/fcntl.h> #include <linux/namei.h> #include <linux/delay.h> @@ -649,6 +650,19 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, }#endif +__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error); +} +#endif /* defined(CONFIG_NFSD_V4) */ #ifdef CONFIG_NFSD_V3 fs/nfsd/vfs.h +1
-0@@ -57,6 +57,7 @@ __be32 nfsd4_set_nfs4_acl(struct svc_rqst *, struct svc_fh *,int nfsd4_get_nfs4_acl (struct svc_rqst *,struct dentry *,struct nfs4_acl **);__be32 nfsd4_set_nfs4_label (struct svc_rqst *,struct svc_fh *,struct xdr_netobj *); +__be32 nfsd4_vfs_fallocate (struct file *,bool ,loff_t ,loff_t );#endif /* CONFIG_NFSD_V4 */ __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct iattr *attrs,fs/nfsd/xdr4.h +24
-0@@ -430,6 +430,27 @@ struct nfsd4_reclaim_complete {u32 rca_one_fs; }; +struct nfsd42_write_res { +struct nfs4_stid *wr_stid; +u64 wr_bytes_written; +u32 wr_stable_how; +nfs4_verifier wr_verifier; +}; + +struct nfsd4_write_plus { +/* request */ +stateid_t wp_stateid; +u32 wp_stable_how; +u32 wp_data_length; +u32 wp_data_content; +u64 wp_offset; +u64 wp_length; +u32 wp_allocated; + +/* response */ +struct nfsd42_write_res wp_res; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -475,6 +496,9 @@ struct nfsd4_op {struct nfsd4_reclaim_complete reclaim_complete;struct nfsd4_test_stateid test_stateid;struct nfsd4_free_stateid free_stateid; + +/* NFSv4.2 */ +struct nfsd4_write_plus write_plus; } u;struct nfs4_replay * replay; };fs/open.c +1
-0@@ -278,6 +278,7 @@ int do_fallocate(struct file *file, int mode, loff_t offset, loff_t len)sb_end_write (inode->i_sb);return ret; } +EXPORT_SYMBOL_GPL (do_fallocate); SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len) {include/linux/nfs4.h +7
-2@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_RECLAIM_COMPLETE +#define LAST_NFS4_OP OP_WRITE_PLUSenum nfsstat4 {NFS4_OK =0 ,@@ -552,4 +552,10 @@ struct nfs4_deviceid { char data[NFS4_DEVICEID4_SIZE]; }; +enum data_content4 { +NFS4_CONTENT_DATA =0 , +NFS4_CONTENT_APP_DATA_HOLE =1 , +NFS4_CONTENT_HOLE =2 , +}; +#endif --1.8.4.1
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 21 lines · Anna Schumaker
Quoted 14 lines · J. Bruce Fields
Quoted 3 lines · Anna Schumaker
This patch only implements
DATA_CONTENT_HOLEsupport, but I tried to write it so other arms of the discriminated union can be added easily later.And there's an
NFS4ERR_UNION_NOTSUPPto handle that case, OK. (Though the spec language is a little ambiguous:If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with
NFS4ERR_UNION_NOTSUPP.Quoted 5 lines
So it's unclear whether we can return this in the case the client is *not* asking for a hole. Are we sure the
NFS4_CONTENT_DATAcase is optional? The language in 5.3 ("Instead a client should utlizeREAD_PLUSandWRITE_PLUS") also suggests theCONTENT_DATAcase might be meant to be mandatory.)
Quoted 3 lines
Fair enough. I wasn't planning on implementing the
NFS4_CONTENT_DATAarm on the client right now, but I might be able to hack something together to test this on the server!
OK. Either way, could you also submit a patch to the spec to make it explicit whether CONTENT_DATA is mandatory or not?
--b.
Quoted 325 lines
Quoted 11 lines
Quoted 5 lines
This patch is missing support for decoding multiple requests on the same file. The way it's written now only the last range provided by the client will be written to.
But that doesn't seem to be a limitation anticipated by the spec, so I guess we need to fix that.
(Why is there an array, anyway? Wouldn't it work just as well to send multiple RPC's, or a single RPC with multiple
WRITE_PLUSops?)Yeah, I'll fix that up. The Linux client will send requests one at a time, so that's what I tested against.
Quoted 308 lines
Quoted 23 lines
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 45 ++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/vfs.c | 14 ++++++++++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 24 ++++++++++++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 8 ++++++- 7 files changed, 152 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1046
Quoted 4 lines
fs/nfsd/nfs4proc.c +4
-0+gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); + } + +return status;Nit, but I like the usual idiom better in most cases:
if (status) return status; normal case here... return 0;--b.
Quoted 258 lines
fs/nfsd/nfs4proc.c +29
-0+} + +static __be32 +nfsd4_write_plus (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_write_plus *writeplus) +{ +struct file *file; +__be32 status; +struct net *net =SVC_NET (rqstp); + + status =nfs4_preprocess_stateid_op (net, cstate, &writeplus->wp_stateid, +WR_STATE , &file); +if (status) +return status; + +if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) +return nfsd4_write_plus_hole (file, writeplus, net); +return nfserr_union_notsupp; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1840,6 +1876,15 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid, .op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize, }, + +/* NFSv4.2 operations */ + [OP_WRITE_PLUS] = { + .op_func = (nfsd4op_func)nfsd4_write_plus, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_WRITE_PLUS", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +60
-2@@ -1485,6 +1485,32 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str }static __be32 +nfsd4_decode_write_plus (struct nfsd4_compoundargs *argp, +struct nfsd4_write_plus *writeplus) +{ +DECODE_HEAD ; +unsigned int i; + + status =nfsd4_decode_stateid (argp, &writeplus->wp_stateid); +if (status) +return status; + +READ_BUF (8 ); +READ32 (writeplus->wp_stable_how); +READ32 (writeplus->wp_data_length); + +for (i =0 ; i < writeplus->wp_data_length; i++) { +READ_BUF (24 ); +READ32 (writeplus->wp_data_content); +READ64 (writeplus->wp_offset); +READ64 (writeplus->wp_length); +READ32 (writeplus->wp_allocated); + } + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1665,7 +1691,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_REVOKE] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp,@@ -3591,6 +3617,38 @@ nfsd4_encode_test_stateid(struct nfsd4_compoundres *resp, __be32 nfserr,return nfserr; } +static void +nfsd42_encode_write_res (struct nfsd4_compoundres *resp,struct nfsd42_write_res *write) +{ +__be32 *p; + +RESERVE_SPACE (4 ); + +if (write->wr_stid ==NULL ) { +WRITE32 (0 ); +ADJUST_ARGS (); + }else { +WRITE32 (1 ); +ADJUST_ARGS (); +nfsd4_encode_stateid (resp, &write->wr_stid->sc_stateid); + } + +RESERVE_SPACE (12 +NFS4_VERIFIER_SIZE ); +WRITE64 (write->wr_bytes_written); +WRITE32 (write->wr_stable_how); +WRITEMEM (write->wr_verifier.data,NFS4_VERIFIER_SIZE ); +ADJUST_ARGS (); +} + +static __be32 +nfsd4_encode_write_plus (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_write_plus *writeplus) +{ +if (!nfserr) +nfsd42_encode_write_res (resp, &writeplus->wp_res); +return nfserr; +} +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {@@ -3670,7 +3728,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop,fs/nfsd/vfs.c +14
-0@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/file.h> #include <linux/splice.h> +#include <linux/falloc.h> #include <linux/fcntl.h> #include <linux/namei.h> #include <linux/delay.h> @@ -649,6 +650,19 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, }#endif +__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error); +} +#endif /* defined(CONFIG_NFSD_V4) */ #ifdef CONFIG_NFSD_V3 fs/nfsd/vfs.h +1
-0@@ -57,6 +57,7 @@ __be32 nfsd4_set_nfs4_acl(struct svc_rqst *, struct svc_fh *,int nfsd4_get_nfs4_acl (struct svc_rqst *,struct dentry *,struct nfs4_acl **);__be32 nfsd4_set_nfs4_label (struct svc_rqst *,struct svc_fh *,struct xdr_netobj *); +__be32 nfsd4_vfs_fallocate (struct file *,bool ,loff_t ,loff_t );#endif /* CONFIG_NFSD_V4 */ __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct iattr *attrs,fs/nfsd/xdr4.h +24
-0@@ -430,6 +430,27 @@ struct nfsd4_reclaim_complete {u32 rca_one_fs; }; +struct nfsd42_write_res { +struct nfs4_stid *wr_stid; +u64 wr_bytes_written; +u32 wr_stable_how; +nfs4_verifier wr_verifier; +}; + +struct nfsd4_write_plus { +/* request */ +stateid_t wp_stateid; +u32 wp_stable_how; +u32 wp_data_length; +u32 wp_data_content; +u64 wp_offset; +u64 wp_length; +u32 wp_allocated; + +/* response */ +struct nfsd42_write_res wp_res; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -475,6 +496,9 @@ struct nfsd4_op {struct nfsd4_reclaim_complete reclaim_complete;struct nfsd4_test_stateid test_stateid;struct nfsd4_free_stateid free_stateid; + +/* NFSv4.2 */ +struct nfsd4_write_plus write_plus; } u;struct nfs4_replay * replay; };fs/open.c +1
-0@@ -278,6 +278,7 @@ int do_fallocate(struct file *file, int mode, loff_t offset, loff_t len)sb_end_write (inode->i_sb);return ret; } +EXPORT_SYMBOL_GPL (do_fallocate); SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len) {include/linux/nfs4.h +7
-2@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_RECLAIM_COMPLETE +#define LAST_NFS4_OP OP_WRITE_PLUSenum nfsstat4 {NFS4_OK =0 ,@@ -552,4 +552,10 @@ struct nfs4_deviceid { char data[NFS4_DEVICEID4_SIZE]; }; +enum data_content4 { +NFS4_CONTENT_DATA =0 , +NFS4_CONTENT_APP_DATA_HOLE =1 , +NFS4_CONTENT_HOLE =2 , +}; +#endif --1.8.4.1
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to J. Bruce FieldsQuoted 50 lines · J. Bruce Fields
Quoted 3 lines · Anna Schumaker
This patch only implements
DATA_CONTENT_HOLEsupport, but I tried to write it so other arms of the discriminated union can be added easily later.And there's an
NFS4ERR_UNION_NOTSUPPto handle that case, OK. (Though the spec language is a little ambiguous:If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with
NFS4ERR_UNION_NOTSUPP.So it's unclear whether we can return this in the case the client is *not* asking for a hole. Are we sure the
NFS4_CONTENT_DATAcase is optional? The language in 5.3 ("Instead a client should utlizeREAD_PLUSandWRITE_PLUS") also suggests theCONTENT_DATAcase might be meant to be mandatory.)Quoted 3 lines
This patch is missing support for decoding multiple requests on the same file. The way it's written now only the last range provided by the client will be written to.
But that doesn't seem to be a limitation anticipated by the spec, so I guess we need to fix that.
(Why is there an array, anyway? Wouldn't it work just as well to send multiple RPC's, or a single RPC with multiple
WRITE_PLUSops?)Quoted 20 lines
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 45 ++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++-- fs/nfsd/vfs.c | 14 ++++++++++++ fs/nfsd/vfs.h | 1 + fs/nfsd/xdr4.h | 24 ++++++++++++++++++++ fs/open.c | 1 + include/linux/nfs4.h | 8 ++++++- 7 files changed, 152 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; }
Quoted 14 lines
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 12 lines
fs/nfsd/nfs4proc.c +12
-0+static __be32 +nfsd4_write_plus_hole (struct file *file,struct nfsd4_write_plus *writeplus, +struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
I did the sync in nfsd4_vfs_fallocate (below), but I can move it if that would make more sense.
Quoted 8 lines
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1046
Quoted 5 lines
fs/nfsd/nfs4proc.c +4
-0+gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); + } + +return status;Nit, but I like the usual idiom better in most cases:
Quoted 4 lines
if (status) return status; normal case here... return 0;
Sure, I'll change that.
Quoted 261 lines
--b.
Quoted 258 lines
fs/nfsd/nfs4proc.c +29
-0+} + +static __be32 +nfsd4_write_plus (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_write_plus *writeplus) +{ +struct file *file; +__be32 status; +struct net *net =SVC_NET (rqstp); + + status =nfs4_preprocess_stateid_op (net, cstate, &writeplus->wp_stateid, +WR_STATE , &file); +if (status) +return status; + +if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) +return nfsd4_write_plus_hole (file, writeplus, net); +return nfserr_union_notsupp; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1840,6 +1876,15 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_get_currentstateid = (stateid_getter)nfsd4_get_freestateid, .op_rsize_bop = (nfsd4op_rsize)nfsd4_only_status_rsize, }, + +/* NFSv4.2 operations */ + [OP_WRITE_PLUS] = { + .op_func = (nfsd4op_func)nfsd4_write_plus, + .op_flags = OP_MODIFIES_SOMETHING | OP_CACHEME, + .op_name = "OP_WRITE_PLUS", + .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, + .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +60
-2@@ -1485,6 +1485,32 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str }static __be32 +nfsd4_decode_write_plus (struct nfsd4_compoundargs *argp, +struct nfsd4_write_plus *writeplus) +{ +DECODE_HEAD ; +unsigned int i; + + status =nfsd4_decode_stateid (argp, &writeplus->wp_stateid); +if (status) +return status; + +READ_BUF (8 ); +READ32 (writeplus->wp_stable_how); +READ32 (writeplus->wp_data_length); + +for (i =0 ; i < writeplus->wp_data_length; i++) { +READ_BUF (24 ); +READ32 (writeplus->wp_data_content); +READ64 (writeplus->wp_offset); +READ64 (writeplus->wp_length); +READ32 (writeplus->wp_allocated); + } + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1665,7 +1691,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_REVOKE] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp,@@ -3591,6 +3617,38 @@ nfsd4_encode_test_stateid(struct nfsd4_compoundres *resp, __be32 nfserr,return nfserr; } +static void +nfsd42_encode_write_res (struct nfsd4_compoundres *resp,struct nfsd42_write_res *write) +{ +__be32 *p; + +RESERVE_SPACE (4 ); + +if (write->wr_stid ==NULL ) { +WRITE32 (0 ); +ADJUST_ARGS (); + }else { +WRITE32 (1 ); +ADJUST_ARGS (); +nfsd4_encode_stateid (resp, &write->wr_stid->sc_stateid); + } + +RESERVE_SPACE (12 +NFS4_VERIFIER_SIZE ); +WRITE64 (write->wr_bytes_written); +WRITE32 (write->wr_stable_how); +WRITEMEM (write->wr_verifier.data,NFS4_VERIFIER_SIZE ); +ADJUST_ARGS (); +} + +static __be32 +nfsd4_encode_write_plus (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_write_plus *writeplus) +{ +if (!nfserr) +nfsd42_encode_write_res (resp, &writeplus->wp_res); +return nfserr; +} +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {@@ -3670,7 +3728,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_COPY_NOTIFY] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_REVOKE] = (nfsd4_enc)nfsd4_encode_noop, [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_noop, + [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop,fs/nfsd/vfs.c +14
-0@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/file.h> #include <linux/splice.h> +#include <linux/falloc.h> #include <linux/fcntl.h> #include <linux/namei.h> #include <linux/delay.h> @@ -649,6 +650,19 @@ __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, }#endif +__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error); +} +#endif /* defined(CONFIG_NFSD_V4) */ #ifdef CONFIG_NFSD_V3 fs/nfsd/vfs.h +1
-0@@ -57,6 +57,7 @@ __be32 nfsd4_set_nfs4_acl(struct svc_rqst *, struct svc_fh *,int nfsd4_get_nfs4_acl (struct svc_rqst *,struct dentry *,struct nfs4_acl **);__be32 nfsd4_set_nfs4_label (struct svc_rqst *,struct svc_fh *,struct xdr_netobj *); +__be32 nfsd4_vfs_fallocate (struct file *,bool ,loff_t ,loff_t );#endif /* CONFIG_NFSD_V4 */ __be32 nfsd_create(struct svc_rqst *, struct svc_fh *, char *name, int len, struct iattr *attrs,fs/nfsd/xdr4.h +24
-0@@ -430,6 +430,27 @@ struct nfsd4_reclaim_complete {u32 rca_one_fs; }; +struct nfsd42_write_res { +struct nfs4_stid *wr_stid; +u64 wr_bytes_written; +u32 wr_stable_how; +nfs4_verifier wr_verifier; +}; + +struct nfsd4_write_plus { +/* request */ +stateid_t wp_stateid; +u32 wp_stable_how; +u32 wp_data_length; +u32 wp_data_content; +u64 wp_offset; +u64 wp_length; +u32 wp_allocated; + +/* response */ +struct nfsd42_write_res wp_res; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -475,6 +496,9 @@ struct nfsd4_op {struct nfsd4_reclaim_complete reclaim_complete;struct nfsd4_test_stateid test_stateid;struct nfsd4_free_stateid free_stateid; + +/* NFSv4.2 */ +struct nfsd4_write_plus write_plus; } u;struct nfs4_replay * replay; };fs/open.c +1
-0@@ -278,6 +278,7 @@ int do_fallocate(struct file *file, int mode, loff_t offset, loff_t len)sb_end_write (inode->i_sb);return ret; } +EXPORT_SYMBOL_GPL (do_fallocate); SYSCALL_DEFINE4(fallocate, int, fd, int, mode, loff_t, offset, loff_t, len) {include/linux/nfs4.h +7
-2@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_RECLAIM_COMPLETE +#define LAST_NFS4_OP OP_WRITE_PLUSenum nfsstat4 {NFS4_OK =0 ,@@ -552,4 +552,10 @@ struct nfs4_deviceid { char data[NFS4_DEVICEID4_SIZE]; }; +enum data_content4 { +NFS4_CONTENT_DATA =0 , +NFS4_CONTENT_APP_DATA_HOLE =1 , +NFS4_CONTENT_HOLE =2 , +}; +#endif --1.8.4.1
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 25 lines · Anna Schumaker
Quoted 9 lines · J. Bruce Fields
Quoted 8 lines · Anna Schumaker
fs/nfsd/nfs4proc.c +0
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; }Quoted 14 lines
fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 12 lines
fs/nfsd/nfs4proc.c +12
-0+static __be32 +nfsd4_write_plus_hole (struct file *file,struct nfsd4_write_plus *writeplus, +struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
Quoted 2 lines
I did the sync in
nfsd4_vfs_fallocate(below), but I can move it if that would make more sense.
What I meant was--why are we doing a sync at all, instead of returning NFS_UNSTABLE and making the client commit?
Honest question, I haven't thought about which is best.
--b.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 26 lines · J. Bruce Fields
Quoted 25 lines · Anna Schumaker
Quoted 23 lines · J. Bruce Fields
Quoted 10 lines · Anna Schumaker
fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
Quoted 7 lines
Quoted 2 lines
I did the sync in
nfsd4_vfs_fallocate(below), but I can move it if that would make more sense.What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?Honest question, I haven't thought about which is best.
No reason other than it seemed easier for my initial implementation.
Quoted 2 lines
--b.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to J. Bruce FieldsQuoted 26 lines · J. Bruce Fields
Quoted 25 lines · Anna Schumaker
Quoted 23 lines · J. Bruce Fields
Quoted 10 lines · Anna Schumaker
fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
Quoted 5 lines
Quoted 2 lines
I did the sync in
nfsd4_vfs_fallocate(below), but I can move it if that would make more sense.What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?
What if the client specifies FILE_SYNC? The spec doesn't allow the server to return UNSTABLE in that situation.
Trond
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 29 lines · Trond
Quoted 26 lines · J. Bruce Fields
Quoted 25 lines · Anna Schumaker
Quoted 23 lines · J. Bruce Fields
Quoted 10 lines · Anna Schumaker
fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I did the sync in
nfsd4_vfs_fallocate(below), but I can move it if that would make more sense.What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?What if the client specifies
FILE_SYNC? The spec doesn't allow the server to return UNSTABLE in that situation.
I don't have that check in there either, but I do think it would be useful if the client were prepared to handle an UNSTABLE reply from the server...
Quoted 2 lines
Trond
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Myklebust, TrondQuoted 29 lines · Trond
Quoted 26 lines · J. Bruce Fields
Quoted 25 lines · Anna Schumaker
Quoted 23 lines · J. Bruce Fields
Quoted 10 lines · Anna Schumaker
fs/nfsd/nfs4proc.c +2
-0@@ -1028,6 +1028,42 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } +static __be32 +nfsd4_write_plus_hole(struct file *file, struct nfsd4_write_plus *writeplus,fs/nfsd/nfs4proc.c nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1042
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+ struct net *net) +{ +__be32 status; + + status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + writeplus->wp_offset, writeplus->wp_length); +if (status == nfs_ok) { + writeplus->wp_res.wr_stid =NULL ; + writeplus->wp_res.wr_bytes_written = writeplus->wp_length; + writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ;Do we need to sync?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I did the sync in
nfsd4_vfs_fallocate(below), but I can move it if that would make more sense.What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?What if the client specifies
FILE_SYNC? The spec doesn't allow the server to return UNSTABLE in that situation.
Of course, sorry for being unclear. I was just wondering if there's some particular reason we're not using NFS_UNSTABLE when the client allows it.
But "because it's easier" is a fine answer. We can always add that later if it turns out to be useful.
--b.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to J. Bruce FieldsQuoted 2 lines · J. Bruce Fields
What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?
Did NFSv4.2 introduce a concept of unstable metadata operations?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields
What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?Did NFSv4.2 introduce a concept of unstable metadata operations?
No, but I think WRITE_PLUS does have a stable/unstable bit so I think we could choose not to do the sync if that'd make sense.
--b.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 9 lines · J. Bruce Fields
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields
What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?Did NFSv4.2 introduce a concept of unstable metadata operations?
No, but I think
WRITE_PLUSdoes have a stable/unstable bit so I think we could choose not to do the sync if that'd make sense.
Both operations are idempotent, so supporting it shouldn't be a major obstactle. But suddenly having some metadata operations that can be unstable seems like a major wart in the spec.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · J. Bruce Fields
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · J. Bruce Fields
What I meant was--why are we doing a sync at all, instead of returning
NFS_UNSTABLEand making the client commit?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Did NFSv4.2 introduce a concept of unstable metadata operations?
No, but I think
WRITE_PLUSdoes have a stable/unstable bit so I think we could choose not to do the sync if that'd make sense.Both operations are idempotent, so supporting it shouldn't be a major obstactle. But suddenly having some metadata operations that can be unstable seems like a major wart in the spec.
COMMIT has always had a metadata part to it, though. It guarantees stability of the ctime/mime and change attributes in addition to the file size and data.
IOW: it really is more akin to fsync() than to fdatasync().
Cheers
Trond
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 3 lines · Trond
COMMIT has always had a metadata part to it, though. It guarantees stability of the ctime/mime and change attributes in addition to the file size and data.
IOW: it really is more akin to
fsync()than tofdatasync().
That's fine, but so far it wasn't used for any purely metadata operation. E.g. if we allow it for a preallocation we should also allow it for updating the size put the current file size.
And diverting from the spec to your implementation: we'd need a lot more generic infrastructure to deal with unstable random non-data operations.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to J. Bruce FieldsQuoted 5 lines · J. Bruce Fields
Quoted 3 lines · Anna Schumaker
This patch only implements
DATA_CONTENT_HOLEsupport, but I tried to write it so other arms of the discriminated union can be added easily later.
Quoted 8 lines
And there's an
NFS4ERR_UNION_NOTSUPPto handle that case, OK. (Though the spec language is a little ambiguous:If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with
NFS4ERR_UNION_NOTSUPP.
Where is this language coming from? Seems like the latest document in NFSv4.2/ repo still calls WRITE_PLUS INITIALIZE and doesn't have anything content related, just a weird "ADB" concept.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Anna Schumakerfs/nfsd/vfs.c __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, · line 662
Quoted 10 lines
fs/nfsd/vfs.c +10
-0+__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 );
What are you trying to do with the fsync_range here? If it's just to make sure the metadata operation is stable on disk please use commit_metadata() from fs/nfsd/vfs.c. As there's no data involved here I can't really see what else it would be for.
Also the allocated flag doesn't make any sense for people not taking the same drugs as the spec authors, please add a comment what it means.
Btw, how did anyone come up with the name WRITE PLUS for something that doesn't actually involve any writes?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 7 lines · Christoph Hellwig
Quoted 7 lines
fs/nfsd/vfs.c +7
-0+__be32 nfsd4_vfs_fallocate (struct file *file,bool allocated,loff_t offset,loff_t len) +{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; +
Quoted 8 lines
fs/nfsd/vfs.c __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, · line 662
Quoted 3 lines
fs/nfsd/vfs.c +3
-0+ error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 );What are you trying to do with the
fsync_rangehere? If it's just to make sure the metadata operation is stable on disk please usecommit_metadata()fromfs/nfsd/vfs.c. As there's no data involved here I can't really see what else it would be for.
My understanding is that if I call fallocate with a mode of "0" it will write a range of 0s to the file. I'll move the fsync() call here and only do it if the client doesn't ask for NFS_UNSTABLE.
Quoted 4 lines
Also the allocated flag doesn't make any sense for people not taking the same drugs as the spec authors, please add a comment what it means.
Sure
Quoted 4 lines
Btw, how did anyone come up with the name WRITE PLUS for something that doesn't actually involve any writes?
Write plus does write more than just holes, I just didn't implement that part of the spec. I'm implementing the NFS4_CONTENT_HOLE arm, but NFS4_CONTENT_DATA is intended to eventually replace the WRITE operation.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 3 lines · Anna Schumaker
Quoted 1 line
what else it would be for.
My understanding is that if I call fallocate with a mode of "0" it will write a range of 0s to the file.
While writing zeroes to the file would be a correct implementation, that's now how the operation is defined or usually implemented. The defintion is that the blocks in the range will be allocated, and reads from it will return zero (and the file size will be updated if needed).
The usual way to implement it is to created extents in an "unwritten state" that will return zeroes when read, even if those zeroes didn't make it to disk.
Quoted 1 line
I'll move the
fsync()call here and only do it if the client doesn't ask forNFS_UNSTABLE.
As said the right way to handle it would be to use commit_metadata, as the cache flushes a fsync does would be unessecary and too expensive for many workloads typically using fallocate.
Quoted 6 lines
Quoted 4 lines
Btw, how did anyone come up with the name WRITE PLUS for something that doesn't actually involve any writes?
Write plus does write more than just holes, I just didn't implement that part of the spec. I'm implementing the
NFS4_CONTENT_HOLEarm, butNFS4_CONTENT_DATAis intended to eventually replace the WRITE operation.
Where is WRITE PLUS defined? I the various draft-ietf-nfsv4-minorversion2-*.txt I can only find what appears to be the predecessor, but that doesn't have a data arm.
Also what does the data arm buy us over good old WRITE?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · Anna Schumaker
Quoted 1 line
what else it would be for.
My understanding is that if I call fallocate with a mode of "0" it will write a range of 0s to the file.
Quoted 8 lines
While writing zeroes to the file would be a correct implementation, that's now how the operation is defined or usually implemented. The defintion is that the blocks in the range will be allocated, and reads from it will return zero (and the file size will be updated if needed).
The usual way to implement it is to created extents in an "unwritten state" that will return zeroes when read, even if those zeroes didn't make it to disk.
And that's all done through metadata? Doing the commit_metadata() call makes a bit more sense now.
Quoted 6 lines
Quoted 2 lines
I'll move the
fsync()call here and only do it if the client doesn't ask forNFS_UNSTABLE.As said the right way to handle it would be to use
commit_metadata, as the cache flushes a fsync does would be unessecary and too expensive for many workloads typically using fallocate.
Okay, sorry for me not understanding earlier!
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Btw, how did anyone come up with the name WRITE PLUS for something that doesn't actually involve any writes?
Quoted 8 lines
Quoted 1 line
Write plus does write more than just holes, I just didn't implement that part of the spec. I'm implementing the
NFS4_CONTENT_HOLEarm, butNFS4_CONTENT_DATAis intended to eventually replace the WRITE operation.Where is WRITE PLUS defined? I the various draft-ietf-nfsv4-minorversion2-*.txt I can only find what appears to be the predecessor, but that doesn't have a data arm.
Also what does the data arm buy us over good old WRITE?
I've been working off of draft #21, the most recent commit is 2b3ab1740b1ea843faa59566fb4213a42d8c724a from Aug 19. The eventual goal is to phase out WRITE and replace it with WRITE_PLUS, but that won't happen until at least 4.3. I don't know why it's being done this way instead of just adding an FALLOCATE operation.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 4 lines · Anna Schumaker
Quoted 2 lines
state" that will return zeroes when read, even if those zeroes didn't make it to disk.
And that's all done through metadata? Doing the
commit_metadata()call makes a bit more sense now.
Yes. The only filesystem in tree that actually seems to write zeroes is gfs2, but even then it doesn't do so through the usual mechanisms.
Quoted 4 lines
Quoted 2 lines
Also what does the data arm buy us over good old WRITE?
I've been working off of draft #21, the most recent commit is
2b3ab1740b1ea843faa59566fb4213a42d8c724afrom Aug 19. The eventual goal is to phase out WRITE and replace it withWRITE_PLUS, but that won't happen until at least 4.3. I don't know why it's being done this way instead of just adding an FALLOCATE operation.
Where can I find draft 21? The newest document in the git repo I was pointed to earlier is draft-ietf-nfsv4-minorversion2-13.txt, and the newest I could find on tools.ietf.org is draft 20.
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
I'll try to take it up directly with the working group, but my post to the list yesterday in your SEEK thread didn't seem to have made it
trough. I also tried to research how they came up with this idiotic
design, but the mailing list archives tell very little. Could it be that most of these decisions are actually made in a smokey backroom and not on the list?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Anna Schumaker
Quoted 2 lines
state" that will return zeroes when read, even if those zeroes didn't make it to disk.
And that's all done through metadata? Doing the
commit_metadata()call makes a bit more sense now.Yes. The only filesystem in tree that actually seems to write zeroes is gfs2, but even then it doesn't do so through the usual mechanisms.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Also what does the data arm buy us over good old WRITE?
I've been working off of draft #21, the most recent commit is
2b3ab1740b1ea843faa59566fb4213a42d8c724afrom Aug 19. The eventual goal is to phase out WRITE and replace it withWRITE_PLUS, but that won't happen until at least 4.3. I don't know why it's being done this way instead of just adding an FALLOCATE operation.Where can I find draft 21? The newest document in the git repo I was pointed to earlier is draft-ietf-nfsv4-minorversion2-13.txt, and the newest I could find on tools.ietf.org is draft 20.
You may need to run make in the NFSv4.2 directory.
Quoted 11 lines
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
I'll try to take it up directly with the working group, but my post to the list yesterday in your SEEK thread didn't seem to have made it
trough. I also tried to research how they came up with this idiotic
design, but the mailing list archives tell very little. Could it be that most of these decisions are actually made in a smokey backroom and not on the list?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Christoph HellwigQuoted 12 lines · Christoph Hellwig
Quoted 4 lines · Anna Schumaker
Quoted 2 lines
state" that will return zeroes when read, even if those zeroes didn't make it to disk.
And that's all done through metadata? Doing the
commit_metadata()call makes a bit more sense now.Yes. The only filesystem in tree that actually seems to write zeroes is gfs2, but even then it doesn't do so through the usual mechanisms.
Quoted 3 lines
Quoted 2 lines
Also what does the data arm buy us over good old WRITE?
Quoted 8 lines
Quoted 1 line
I've been working off of draft #21, the most recent commit is
2b3ab1740b1ea843faa59566fb4213a42d8c724afrom Aug 19. The eventual goal is to phase out WRITE and replace it withWRITE_PLUS, but that won't happen until at least 4.3. I don't know why it's being done this way instead of just adding an FALLOCATE operation.Where can I find draft 21? The newest document in the git repo I was pointed to earlier is draft-ietf-nfsv4-minorversion2-13.txt, and the newest I could find on tools.ietf.org is draft 20.
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
Protocol extensibility. WRITE_PLUS adds a discriminated union of data types that can be extended easily to include initialization patterns, integrity metadata, and other things like holes. It's entirely another question as to whether any particular extension is tasteful.
Quoted 6 lines
I'll try to take it up directly with the working group, but my post to the list yesterday in your SEEK thread didn't seem to have made it
trough. I also tried to research how they came up with this idiotic
design, but the mailing list archives tell very little. Could it be that most of these decisions are actually made in a smokey backroom and not on the list?
The documents themselves count as public proposals. That's why they are referred to as "Requests For Comments".
The mailing list is not the only place where they are discussed. We also use conference calls, and there are face-to-face meetings too. Not all of that content is distilled into a public record. Very much like Linux kernel development.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 4 lines · Chuck Lever
Quoted 2 lines
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
Protocol extensibility.
WRITE_PLUSadds a discriminated union of data types that can be extended easily to include initialization patterns, integrity metadata, and other things like holes. It's entirely another question as to whether any particular extension is tasteful.
Maybe I'm missing something, but what does multiplexing entirely different operation actually buy you? It is different operation after all. It's not like adding new operations to NFS is all that hard.
Quoted 1 line
The mailing list is not the only place where they are discussed. We also use conference calls, and there are face-to-face meetings too. Not all of that content is distilled into a public record. Very much like Linux kernel development.
Linux development strives very hard documenting rationales, something I haven't found on the NFS list or in the repository.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Chuck Lever
Quoted 2 lines
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
Protocol extensibility.
WRITE_PLUSadds a discriminated union of data types that can be extended easily to include initialization patterns, integrity metadata, and other things like holes. It's entirely another question as to whether any particular extension is tasteful.Maybe I'm missing something, but what does multiplexing entirely different operation actually buy you? It is different operation after all.
I think a successful argument could be made that initialization or hole punching may not belong in a WRITE_PLUS operation. For example, using a COMMIT for a long-running server-performed file initialization doesn't make sense to me. The infrastructure we introduced for COPY_OFFLOAD seems better suited for file initialization.
Quoted 1 line
It's not like adding new operations to NFS is all that hard.
Actually, it is harder than you think. For various reasons, it takes five or more years to create a new NFS protocol specification, something we hope to address.
Quoted 4 lines
Quoted 1 line
The mailing list is not the only place where they are discussed. We also use conference calls, and there are face-to-face meetings too. Not all of that content is distilled into a public record. Very much like Linux kernel development.
Linux development strives very hard documenting rationales, something I haven't found on the NFS list or in the repository.
There are plenty of times where I find the kernel git log or e-mail archives entirely unhelpful in understanding the why's and wherefor's. It's a human endeavor, and like all such things, is imperfect, and is often a matter of knowing who to ask rather than where to look.
By the way, e-mail from non-members to [email protected] is moderated. The moderator may be traveling or otherwise unavailable, as IETF 88 is happening this week. If you subscribe to the list, you should be able to post without restriction.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 1 line · Chuck Lever
I think a successful argument could be made that initialization or hole punching may not belong in a
WRITE_PLUSoperation. For example, using a COMMIT for a long-running server-performed file initialization doesn't make sense to me. The infrastructure we introduced forCOPY_OFFLOADseems better suited for file initialization.
I've been going through the latest draft and will submit a counter proposal in a few days time.
Quoted 3 lines
Quoted 1 line
It's not like adding new operations to NFS is all that hard.
Actually, it is harder than you think. For various reasons, it takes five or more years to create a new NFS protocol specification, something we hope to address.
I see the pain in getting new minor versions out. But that pain is shared between adding a new operation and a new arm to an union.
Quoted 1 line
By the way, e-mail from non-members to [email protected] is moderated. The moderator may be traveling or otherwise unavailable, as IETF 88 is happening this week. If you subscribe to the list, you should be able to post without restriction.
I have been subscribed longer than I've been posting to it.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Chuck LeverQuoted 13 lines · Chuck Lever
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Chuck Lever
Quoted 2 lines
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
Protocol extensibility.
WRITE_PLUSadds a discriminated union of data types that can be extended easily to include initialization patterns, integrity metadata, and other things like holes. It's entirely another question as to whether any particular extension is tasteful.Maybe I'm missing something, but what does multiplexing entirely different operation actually buy you? It is different operation after all.
Quoted 5 lines
I think a successful argument could be made that initialization or hole punching may not belong in a
WRITE_PLUSoperation. For example, using a COMMIT for a long-running server-performed file initialization doesn't make sense to me. The infrastructure we introduced forCOPY_OFFLOADseems better suited for file initialization.Quoted 1 line
It's not like adding new operations to NFS is all that hard.
Actually, it is harder than you think. For various reasons, it takes five or more years to create a new NFS protocol specification, something we hope to address.
Well, but that's not really a function of the number of operations. It's not as though implementing hole punching will take more or less time depending on whether we implement something as a compound op or a union.
--b.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Chuck LeverQuoted 4 lines · Chuck Lever
Quoted 1 line · Christoph Hellwig
On Mon, Nov 04, 2013 at 11:16:13AM -0800, Chuck Lever wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I still don't understand why anyone would phase out WRITE in favour of something that doesn't actually add any value for the write case.
Protocol extensibility.
WRITE_PLUSadds a discriminated union of data types that can be extended easily to include initialization patterns, integrity metadata, and other things like holes. It's entirely another question as to whether any particular extension is tasteful.Maybe I'm missing something, but what does multiplexing entirely different operation actually buy you? It is different operation after all.
Do you have an issue with READ_PLUS?
I.e., are you okay with that operation because the answer is not known beforehand?
Quoted 2 lines
I think a successful argument could be made that initialization or hole punching may not belong in a
WRITE_PLUSoperation. For example, using a COMMIT for a long-running server-performed file initialization doesn't make sense to me. The infrastructure we introduced forCOPY_OFFLOADseems better suited for file initialization.
WRITE_PLUS does utilize COPY_OFFLOAD for such actions.
Quoted 9 lines
Quoted 2 lines
It's not like adding new operations to NFS is all that hard.
Actually, it is harder than you think. For various reasons, it takes five or more years to create a new NFS protocol specification, something we hope to address.
Quoted 4 lines
Quoted 1 line
The mailing list is not the only place where they are discussed. We also use conference calls, and there are face-to-face meetings too. Not all of that content is distilled into a public record. Very much like Linux kernel development.
Linux development strives very hard documenting rationales, something I haven't found on the NFS list or in the repository.
There was a strong consensus in the biweekly calls to merge all of the operations together into WRITE_PLUS. I think we also voted on this at one of the meetings.
But in any event, the document is still open and I'm glad Anna is funneling implementation experience into it.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 3 lines · Tom
Do you have an issue with
READ_PLUS?I.e., are you okay with that operation because the answer is not known beforehand?
The hole vs data part of it looks reasonable. The whole ADH complex looks a little fishy to me, but I will have to dig deeper into before writing down a detailed critism.
But one very obvious question already comes up: what data is supposed to be returned when a client that doesn't suport 4.2 at all are doesn't support ADHs reads a file that contains them. From a quick glance it seems a mapping from ADHs to flat blocks is possible, but such a mapping should be specified in the standard.
Quoted 1 line
There was a strong consensus in the biweekly calls to merge all of the operations together into
WRITE_PLUS.
Do you remember any arguments for it? Also any rational for adding the plain old data write to it?
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
In reply to Anna Schumakerfs/nfsd/vfs.c __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, · line 663
Quoted 10 lines
fs/nfsd/vfs.c +10
-0+{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error);
Same problem with the falloc vs WRITE_PLUS mismatch here. For XFS you could make the allocated case all XFS_IOC_ZERO_RANGE, but the proposed VFS equivalent never seemed to have made it in.
Also no need for the FALLOC_FL_KEEP_SIZE in the hole punch case, it is ignored.
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 15 lines · Christoph Hellwig
fs/nfsd/vfs.c __be32 nfsd4_set_nfs4_label(struct svc_rqst *rqstp, struct svc_fh *fhp, · line 663
Quoted 10 lines
fs/nfsd/vfs.c +10
-0+{ +int error, mode =0 ; + +if (allocated == false) + mode =FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE ; + + error =do_fallocate (file, mode, offset, len); +if (error ==0 ) + error =vfs_fsync_range (file, offset, offset + len,0 ); +return nfserrno (error);Same problem with the falloc vs
WRITE_PLUSmismatch here. For XFS you could make the allocated case allXFS_IOC_ZERO_RANGE, but the proposed VFS equivalent never seemed to have made it in.
Quoted 3 lines
Also no need for the
FALLOC_FL_KEEP_SIZEin the hole punch case, it is ignored.
It's not ignored in do_fallocate() (fs/open.c) where it returns -EOPNOTSUPP if FALLOC_FL_PUNCH_HOLE is set without FALLOC_FL_KEEP_SIZE :)
Re: [PATCH 3/4] NFSD: Add WRITE_PLUS support for hole punches
Quoted 1 line · Anna Schumaker
It's not ignored in
do_fallocate()(fs/open.c) where it returns-EOPNOTSUPPifFALLOC_FL_PUNCH_HOLEis set withoutFALLOC_FL_KEEP_SIZE:)
Oops, looks like we decided to enforce it, despite hole punches never having an effect on file size. I stand corrected.
PATCH 4/4 NFSD: Implement SEEK
In reply to Anna SchumakerThis patch adds in the SEEK operation used by clients doing an llseek on a file to find either hole or data sections in a file. I'm faking the "allocated" status right now, since I haven't quite figured out how to tell if a range is allocated on disk yet.
This patch is missing correctly determining the "allocated" status of the HOLE / DATA range. I expect I'll need to learn all about how fiemap works before properly setting these values.
--- fs/nfsd/nfs4proc.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 40 ++++++++++++++++++++++++++++++++++++++-- fs/nfsd/xdr4.h | 15 +++++++++++++++ include/linux/nfs4.h | 2 +- 4 files changed, 98 insertions(+), 3 deletions(-)
fs/nfsd/nfs4proc.c +44 -0
@@ -1064,6 +1064,45 @@ nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return nfserr_union_notsupp; } +static __be32 +nfsd4_seek (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_seek *seek) +{ +struct file *file; +loff_t pos, end_pos; +__be32 status; + + status =nfs4_preprocess_stateid_op (SVC_NET (rqstp), cstate, + &seek->seek_stateid, +RD_STATE |WR_STATE , &file); +if (status != nfs_ok) +return status; + +if (seek->seek_whence ==NFS4_CONTENT_DATA ) { + pos =vfs_llseek (file, seek->seek_offset,SEEK_DATA ); + end_pos =vfs_llseek (file, pos,SEEK_HOLE ); + seek->seek_allocated = true; + }else { + pos =vfs_llseek (file, seek->seek_offset,SEEK_HOLE ); + end_pos =vfs_llseek (file, pos,SEEK_DATA ); + seek->seek_allocated = false; + } + +if (pos <0 ) +return nfserrno (pos); +else if (end_pos == -ENXIO ) { + seek->seek_length =0 ; + seek->seek_eof = true; + }else if (end_pos <0 ) +return nfserrno (end_pos); +else + seek->seek_length = end_pos - pos; + + seek->seek_offset = pos; + +return nfs_ok; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME
@@ -1885,6 +1924,11 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, }, + + [OP_SEEK] = { + .op_func = (nfsd4op_func)nfsd4_seek, + .op_name = "OP_SEEK", + }, };#ifdef NFSD_DEBUG
fs/nfsd/nfs4xdr.c +38 -2
@@ -1511,6 +1511,22 @@ nfsd4_decode_write_plus(struct nfsd4_compoundargs *argp, }static __be32 +nfsd4_decode_seek (struct nfsd4_compoundargs *argp,struct nfsd4_seek *seek) +{ +DECODE_HEAD ; + + status =nfsd4_decode_stateid (argp, &seek->seek_stateid); +if (status) +return status; + +READ_BUF (12 ); +READ64 (seek->seek_offset); +READ32 (seek->seek_whence); + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;
@@ -1693,7 +1709,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_SEEK] = (nfsd4_dec)nfsd4_decode_seek, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp, };
@@ -3650,6 +3666,26 @@ nfsd4_encode_write_plus(struct nfsd4_compoundres *resp, __be32 nfserr, }static __be32 +nfsd4_encode_seek (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_seek *seek) +{ +__be32 *p; + +if (nfserr) +return nfserr; + +RESERVE_SPACE (28 ); +WRITE32 (seek->seek_eof); +WRITE32 (seek->seek_whence); +WRITE64 (seek->seek_offset); +WRITE64 (seek->seek_length); +WRITE32 (seek->seek_allocated); +ADJUST_ARGS (); + +return nfserr; +} + +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {return nfserr;
@@ -3730,7 +3766,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, + [OP_SEEK] = (nfsd4_enc)nfsd4_encode_seek, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop, };
fs/nfsd/xdr4.h +15 -0
@@ -451,6 +451,20 @@ struct nfsd4_write_plus {struct nfsd42_write_res wp_res; }; +struct nfsd4_seek { +/* request */ +stateid_t seek_stateid; + +/* Shared between request and response */ +u64 seek_offset; +u32 seek_whence; + +/* response */ +u32 seek_eof; +u64 seek_length; +u32 seek_allocated; +}; +struct nfsd4_op {int opnum;__be32 status;
@@ -499,6 +513,7 @@ struct nfsd4_op {/* NFSv4.2 */ struct nfsd4_write_plus write_plus; +struct nfsd4_seek seek; } u;struct nfs4_replay * replay; };
include/linux/nfs4.h +1 -1
@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_WRITE_PLUS +#define LAST_NFS4_OP OP_SEEKenum nfsstat4 {NFS4_OK =0 ,
Signature · 1 line
1.8.4.1
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 26 lines · Anna Schumaker
This patch adds in the SEEK operation used by clients doing an llseek on a file to find either hole or data sections in a file. I'm faking the "allocated" status right now, since I haven't quite figured out how to tell if a range is allocated on disk yet.
This patch is missing correctly determining the "allocated" status of the HOLE / DATA range. I expect I'll need to learn all about how fiemap works before properly setting these values.
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 40 ++++++++++++++++++++++++++++++++++++++-- fs/nfsd/xdr4.h | 15 +++++++++++++++ include/linux/nfs4.h | 2 +- 4 files changed, 98 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +1
-0@@ -1064,6 +1064,45 @@ nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return nfserr_union_notsupp; } +static __be32
fs/nfsd/nfs4proc.c nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1077
Quoted 10 lines
fs/nfsd/nfs4proc.c +10
-0+nfsd4_seek(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, + struct nfsd4_seek *seek) +{ + struct file *file; +loff_t pos, end_pos; +__be32 status; + + status =nfs4_preprocess_stateid_op (SVC_NET (rqstp), cstate, + &seek->seek_stateid, +RD_STATE |WR_STATE , &file);
Note nfs4_preprocess_stateid_op requires the caller to hold the state lock. Andyou want to either hold it till you're done using "file" or take a reference on file before dropping it.
--b.
Quoted 164 lines
fs/nfsd/nfs4proc.c +33
-0+if (status != nfs_ok) +return status; + +if (seek->seek_whence ==NFS4_CONTENT_DATA ) { + pos =vfs_llseek (file, seek->seek_offset,SEEK_DATA ); + end_pos =vfs_llseek (file, pos,SEEK_HOLE ); + seek->seek_allocated = true; + }else { + pos =vfs_llseek (file, seek->seek_offset,SEEK_HOLE ); + end_pos =vfs_llseek (file, pos,SEEK_DATA ); + seek->seek_allocated = false; + } + +if (pos <0 ) +return nfserrno (pos); +else if (end_pos == -ENXIO ) { + seek->seek_length =0 ; + seek->seek_eof = true; + }else if (end_pos <0 ) +return nfserrno (end_pos); +else + seek->seek_length = end_pos - pos; + + seek->seek_offset = pos; + +return nfs_ok; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1885,6 +1924,11 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, }, + + [OP_SEEK] = { + .op_func = (nfsd4op_func)nfsd4_seek, + .op_name = "OP_SEEK", + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +38
-2@@ -1511,6 +1511,22 @@ nfsd4_decode_write_plus(struct nfsd4_compoundargs *argp, }static __be32 +nfsd4_decode_seek (struct nfsd4_compoundargs *argp,struct nfsd4_seek *seek) +{ +DECODE_HEAD ; + + status =nfsd4_decode_stateid (argp, &seek->seek_stateid); +if (status) +return status; + +READ_BUF (12 ); +READ64 (seek->seek_offset); +READ32 (seek->seek_whence); + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1693,7 +1709,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_SEEK] = (nfsd4_dec)nfsd4_decode_seek, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp, };@@ -3650,6 +3666,26 @@ nfsd4_encode_write_plus(struct nfsd4_compoundres *resp, __be32 nfserr, }static __be32 +nfsd4_encode_seek (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_seek *seek) +{ +__be32 *p; + +if (nfserr) +return nfserr; + +RESERVE_SPACE (28 ); +WRITE32 (seek->seek_eof); +WRITE32 (seek->seek_whence); +WRITE64 (seek->seek_offset); +WRITE64 (seek->seek_length); +WRITE32 (seek->seek_allocated); +ADJUST_ARGS (); + +return nfserr; +} + +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {return nfserr;@@ -3730,7 +3766,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, + [OP_SEEK] = (nfsd4_enc)nfsd4_encode_seek, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop, };fs/nfsd/xdr4.h +15
-0@@ -451,6 +451,20 @@ struct nfsd4_write_plus {struct nfsd42_write_res wp_res; }; +struct nfsd4_seek { +/* request */ +stateid_t seek_stateid; + +/* Shared between request and response */ +u64 seek_offset; +u32 seek_whence; + +/* response */ +u32 seek_eof; +u64 seek_length; +u32 seek_allocated; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -499,6 +513,7 @@ struct nfsd4_op {/* NFSv4.2 */ struct nfsd4_write_plus write_plus; +struct nfsd4_seek seek; } u;struct nfs4_replay * replay; };include/linux/nfs4.h +1
-1@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_WRITE_PLUS +#define LAST_NFS4_OP OP_SEEKenum nfsstat4 {NFS4_OK =0 ,Signature · 1 line
1.8.4.1
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 26 lines · J. Bruce Fields
Quoted 25 lines · Anna Schumaker
This patch adds in the SEEK operation used by clients doing an llseek on a file to find either hole or data sections in a file. I'm faking the "allocated" status right now, since I haven't quite figured out how to tell if a range is allocated on disk yet.
This patch is missing correctly determining the "allocated" status of the HOLE / DATA range. I expect I'll need to learn all about how fiemap works before properly setting these values.
Signed-off-by Anna Schumaker <[email protected]>--- fs/nfsd/nfs4proc.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4xdr.c | 40 ++++++++++++++++++++++++++++++++++++++-- fs/nfsd/xdr4.h | 15 +++++++++++++++ include/linux/nfs4.h | 2 +- 4 files changed, 98 insertions(+), 3 deletions(-)fs/nfsd/nfs4proc.c +0
-0@@ -1064,6 +1064,45 @@ nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return nfserr_union_notsupp; }
Quoted 15 lines
fs/nfsd/nfs4proc.c nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, · line 1077
Quoted 11 lines
fs/nfsd/nfs4proc.c +11
-0+static __be32 +nfsd4_seek (struct svc_rqst *rqstp,struct nfsd4_compound_state *cstate, +struct nfsd4_seek *seek) +{ +struct file *file; +loff_t pos, end_pos; +__be32 status; + + status =nfs4_preprocess_stateid_op (SVC_NET (rqstp), cstate, + &seek->seek_stateid, +RD_STATE |WR_STATE , &file);Note
nfs4_preprocess_stateid_oprequires the caller to hold the state lock. Andyou want to either hold it till you're done using "file" or take a reference on file before dropping it.
That's useful to know. I'll fix it up here, and anywhere else I call nfs4_preprocess_stateid_op()!
Quoted 167 lines
--b.
Quoted 164 lines
fs/nfsd/nfs4proc.c +33
-0+if (status != nfs_ok) +return status; + +if (seek->seek_whence ==NFS4_CONTENT_DATA ) { + pos =vfs_llseek (file, seek->seek_offset,SEEK_DATA ); + end_pos =vfs_llseek (file, pos,SEEK_HOLE ); + seek->seek_allocated = true; + }else { + pos =vfs_llseek (file, seek->seek_offset,SEEK_HOLE ); + end_pos =vfs_llseek (file, pos,SEEK_DATA ); + seek->seek_allocated = false; + } + +if (pos <0 ) +return nfserrno (pos); +else if (end_pos == -ENXIO ) { + seek->seek_length =0 ; + seek->seek_eof = true; + }else if (end_pos <0 ) +return nfserrno (end_pos); +else + seek->seek_length = end_pos - pos; + + seek->seek_offset = pos; + +return nfs_ok; +} +/* This routine never returns NFS_OK! If there are no other errors, it * will return NFSERR_SAME or NFSERR_NOT_SAME depending on whether the * attributes matched. VERIFY is implemented by mapping NFSERR_SAME @@ -1885,6 +1924,11 @@ static struct nfsd4_operation nfsd4_ops[] = { .op_rsize_bop = (nfsd4op_rsize)nfsd4_write_rsize, .op_get_currentstateid = (stateid_getter)nfsd4_get_writestateid, }, + + [OP_SEEK] = { + .op_func = (nfsd4op_func)nfsd4_seek, + .op_name = "OP_SEEK", + }, };#ifdef NFSD_DEBUG fs/nfsd/nfs4xdr.c +38
-2@@ -1511,6 +1511,22 @@ nfsd4_decode_write_plus(struct nfsd4_compoundargs *argp, }static __be32 +nfsd4_decode_seek (struct nfsd4_compoundargs *argp,struct nfsd4_seek *seek) +{ +DECODE_HEAD ; + + status =nfsd4_decode_stateid (argp, &seek->seek_stateid); +if (status) +return status; + +READ_BUF (12 ); +READ64 (seek->seek_offset); +READ32 (seek->seek_whence); + +DECODE_TAIL ; +} + +static __be32 nfsd4_decode_noop (struct nfsd4_compoundargs *argp,void *p) {return nfs_ok;@@ -1693,7 +1709,7 @@ static nfsd4_dec nfsd42_dec_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_dec)nfsd4_decode_notsupp, [OP_WRITE_PLUS] = (nfsd4_dec)nfsd4_decode_write_plus, [OP_READ_PLUS] = (nfsd4_dec)nfsd4_decode_notsupp, - [OP_SEEK] = (nfsd4_dec)nfsd4_decode_notsupp, + [OP_SEEK] = (nfsd4_dec)nfsd4_decode_seek, [OP_IO_ADVISE] = (nfsd4_dec)nfsd4_decode_notsupp, };@@ -3650,6 +3666,26 @@ nfsd4_encode_write_plus(struct nfsd4_compoundres *resp, __be32 nfserr, }static __be32 +nfsd4_encode_seek (struct nfsd4_compoundres *resp,__be32 nfserr, +struct nfsd4_seek *seek) +{ +__be32 *p; + +if (nfserr) +return nfserr; + +RESERVE_SPACE (28 ); +WRITE32 (seek->seek_eof); +WRITE32 (seek->seek_whence); +WRITE64 (seek->seek_offset); +WRITE64 (seek->seek_length); +WRITE32 (seek->seek_allocated); +ADJUST_ARGS (); + +return nfserr; +} + +static __be32 nfsd4_encode_noop (struct nfsd4_compoundres *resp,__be32 nfserr,void *p) {return nfserr;@@ -3730,7 +3766,7 @@ static nfsd4_enc nfsd4_enc_ops[] = { [OP_OFFLOAD_STATUS] = (nfsd4_enc)nfsd4_encode_noop, [OP_WRITE_PLUS] = (nfsd4_enc)nfsd4_encode_write_plus, [OP_READ_PLUS] = (nfsd4_enc)nfsd4_encode_noop, - [OP_SEEK] = (nfsd4_enc)nfsd4_encode_noop, + [OP_SEEK] = (nfsd4_enc)nfsd4_encode_seek, [OP_IO_ADVISE] = (nfsd4_enc)nfsd4_encode_noop, };fs/nfsd/xdr4.h +15
-0@@ -451,6 +451,20 @@ struct nfsd4_write_plus {struct nfsd42_write_res wp_res; }; +struct nfsd4_seek { +/* request */ +stateid_t seek_stateid; + +/* Shared between request and response */ +u64 seek_offset; +u32 seek_whence; + +/* response */ +u32 seek_eof; +u64 seek_length; +u32 seek_allocated; +}; +struct nfsd4_op {int opnum;__be32 status;@@ -499,6 +513,7 @@ struct nfsd4_op {/* NFSv4.2 */ struct nfsd4_write_plus write_plus; +struct nfsd4_seek seek; } u;struct nfs4_replay * replay; };include/linux/nfs4.h +1
-2@@ -128,7 +128,7 @@ enum nfs_opnum4 {Needs to be updated if more operations are defined in future.*/ #define FIRST_NFS4_OP OP_ACCESS -#define LAST_NFS4_OP OP_WRITE_PLUS +#define LAST_NFS4_OP OP_SEEKenum nfsstat4 {NFS4_OK =0 , --1.8.4.1
Re: [PATCH 4/4] NFSD: Implement SEEK
In reply to Anna SchumakerQuoted 8 lines · Anna Schumaker
This patch adds in the SEEK operation used by clients doing an llseek on a file to find either hole or data sections in a file. I'm faking the "allocated" status right now, since I haven't quite figured out how to tell if a range is allocated on disk yet.
This patch is missing correctly determining the "allocated" status of the HOLE / DATA range. I expect I'll need to learn all about how fiemap works before properly setting these values.
What is the definition of allocated in this context? Specificly how does it different from meaning of allocated as used by SEEK_DATA?
Going out to fiemap is something we should absolutely avoid.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Anna Schumaker
This patch adds in the SEEK operation used by clients doing an llseek on a file to find either hole or data sections in a file. I'm faking the "allocated" status right now, since I haven't quite figured out how to tell if a range is allocated on disk yet.
Quoted 6 lines
Quoted 3 lines
This patch is missing correctly determining the "allocated" status of the HOLE / DATA range. I expect I'll need to learn all about how fiemap works before properly setting these values.
What is the definition of allocated in this context? Specificly how does it different from meaning of allocated as used by
SEEK_DATA?
From what I can tell, I think the allocated flag will just tell the
clients if all the blocks exist on disk or not. Is there a way to have a hole with allocated blocks? Or maybe it's supposed to represent partially allocated blocks? I checked the draft, and it doesn't actually say what they expect allocated to represent...
Quoted 3 lines
Going out to fiemap is something we should absolutely avoid.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 5 lines · Anna Schumaker
From what I can tell, I think the allocated flag will just tell the
clients if all the blocks exist on disk or not. Is there a way to have a hole with allocated blocks? Or maybe it's supposed to represent partially allocated blocks? I checked the draft, and it doesn't actually say what they expect allocated to represent...
You can have preallocated space, which behaves like a hole in that reads return zeroes. As far as SEEK_DATA/SEEK_HOLE is concerned we tend to treat them as holes when we can as that's what the users expect.
Note that at least the pnfs block rfc has a special state representing this kind of preallocated space.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Anna Schumaker
From what I can tell, I think the allocated flag will just tell the
clients if all the blocks exist on disk or not. Is there a way to have a hole with allocated blocks? Or maybe it's supposed to represent partially allocated blocks? I checked the draft, and it doesn't actually say what they expect allocated to represent...
Quoted 6 lines
You can have preallocated space, which behaves like a hole in that reads return zeroes. As far as SEEK_DATA/SEEK_HOLE is concerned we tend to treat them as holes when we can as that's what the users expect.
Note that at least the pnfs block rfc has a special state representing this kind of preallocated space.
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-20#section-5.2
E.g. "Hole: A byte range within a Sparse file that contains regions of all zeroes. For block-based file systems, this could also be an unallocated region of the file."
So it sounds like a hole can either be allocated or not, which seems to agree with what you expect?
I haven't read these parts of the spec yet at all....
--b.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 1 line · J. Bruce Fields
I haven't read these parts of the spec yet at all....
Btw, is there a way do to something like a git blame for the spec? I'd really love to confront people personally with some of the odd bits they came up with.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · J. Bruce Fields
I haven't read these parts of the spec yet at all....
Btw, is there a way do to something like a git blame for the spec? I'd really love to confront people personally with some of the odd bits they came up with.
It *is* in git, so you can literally run git blame:
https://github.com/loghyr/NFSv4.2
But it'll probably credit almost everything to Tom and he's typically just the messenger....
Just direct gripes to [email protected]. They also take patches.
--b.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 8 lines · J. Bruce Fields
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · J. Bruce Fields
I haven't read these parts of the spec yet at all....
Btw, is there a way do to something like a git blame for the spec? I'd really love to confront people personally with some of the odd bits they came up with.
Quoted 6 lines
It *is* in git, so you can literally run git blame:
https://github.com/loghyr/NFSv4.2But it'll probably credit almost everything to Tom and he's typically just the messenger....
Indeed, a quick try attributes everything to him. No good way to figure out who is to blame for the whole WRITE PLUS idiocy.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 11 lines · Christoph Hellwig
Quoted 10 lines · J. Bruce Fields
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · J. Bruce Fields
I haven't read these parts of the spec yet at all....
Btw, is there a way do to something like a git blame for the spec? I'd really love to confront people personally with some of the odd bits they came up with.
It *is* in git, so you can literally run git blame:
Quoted 7 lines
Quoted 4 lines
https://github.com/loghyr/NFSv4.2But it'll probably credit almost everything to Tom and he's typically just the messenger....
Indeed, a quick try attributes everything to him. No good way to figure out who is to blame for the whole WRITE PLUS idiocy.
Imagine it was me, if it helps.
The important thing is that we understand what should be fixed.
--b.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 3 lines · J. Bruce Fields
Imagine it was me, if it helps.
The important thing is that we understand what should be fixed.
The important part is how someone got the idea for the pattern writing and the weird ADBs from and why they believe they should overload a single command for all of them. And apparently reqular data writes as well, even if I haven't found that part of the spec yet.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · J. Bruce Fields
Imagine it was me, if it helps.
The important thing is that we understand what should be fixed.
The important part is how someone got the idea for the pattern writing and the weird ADBs from
The original proposal is
http://tools.ietf.org/html/draft-eisler-nfsv4-enterprise-apps-01
and has been somewhat modified in its expression in NFSv4.2.
Quoted 2 lines
and why they believe they should overload a
single command for all of them.
I believe that file initialization is closer in semantics to COPY_OFFLOAD than it is to WRITE, and I find it awkward to use the WRITE_PLUS operation for initialization. But I could never make a strong technical argument why they should not be joined, other than "Clutter!".
Quoted 2 lines
And apparently reqular data writes as
well, even if I haven't found that part of the spec yet.
http://www.ietf.org/id/draft-ietf-nfsv4-minorversion2-20.txt
is the latest version I can find.
The usual process is to explain what needs to be fixed (ie, make a problem statement) as Bruce said. That is often accompanied by an alternate design proposal that addresses your issues.
It is appropriate to address your comments to the group, and not to a single person. As in most online communities, ad hominem and confrontation are not appreciated.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 5 lines · Chuck Lever
The original proposal is
http://tools.ietf.org/html/draft-eisler-nfsv4-enterprise-apps-01
and has been somewhat modified in its expression in NFSv4.2.
The INITIALIZE operation in that proposal look like a sane translation of the SCSI WRITE SAME commands to NFS, but I have no idea
how that eveloved into the monster of ADHs in the NFS4.2 draft.
Re: [PATCH 4/4] NFSD: Implement SEEK
In reply to J. Bruce FieldsQuoted 14 lines · J. Bruce Fields
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · J. Bruce Fields
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · J. Bruce Fields
I haven't read these parts of the spec yet at all....
Btw, is there a way do to something like a git blame for the spec? I'd really love to confront people personally with some of the odd bits they came up with.
It *is* in git, so you can literally run git blame:
https://github.com/loghyr/NFSv4.2
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
But it'll probably credit almost everything to Tom and he's typically just the messenger....
Indeed, a quick try attributes everything to him. No good way to figure out who is to blame for the whole WRITE PLUS idiocy.
Imagine it was me, if it helps.
Nah, most likely me, no imagination needed.
We were also looking at bringing larger WRITEs during 4.3 and I wanted to have WRITE_PLUS already in place for that.
Quoted 7 lines
The important thing is that we understand what should be fixed.
--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 4 lines · Thomas Haynes
Nah, most likely me, no imagination needed.
We were also looking at bringing larger WRITEs during 4.3 and I wanted to have
WRITE_PLUSalready in place for that.
What exactly does a WRITE PLUS with various non-related overloads in 4.2 buy you for introducing large writes in 4.3?
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 8 lines · Christoph Hellwig
Quoted 4 lines · Thomas Haynes
Nah, most likely me, no imagination needed.
We were also looking at bringing larger WRITEs during 4.3 and I wanted to have
WRITE_PLUSalready in place for that.What exactly does a WRITE PLUS with various non-related overloads in 4.2 buy you for introducing large writes in 4.3?
In 4.3, READ_PLUS would be extended for large reads. We would at that time need a WRITE_PLUS to be extended for large writes.
The relationship falls from having WRITE_PLUS handle all of the variants handled by READ_PLUS. It needs to accept all of them because it does not know what will be returned.
We did not want many new operators and also wanted the operators to be extensible. With this approach, you can define a new arm of the discriminated union, not have to implement it, and not burn an operator.
Some of the history is captured here:
http://www.ietf.org/mail-archive/web/nfsv4/current/msg11235.html
http://www.ietf.org/mail-archive/web/nfsv4/current/msg11470.html
http://www.ietf.org/proceedings/84/slides/slides-84-nfsv4-1.pdf (slide 6)
It doesn't capture the intent of NFS4ERR_UNION_NOTSUPP in this decision.
11.1.1.1. NFS4ERR_UNION_NOTSUPP (Error Code 10090)
One of the arguments to the operation is a discriminated union and while the server supports the given operation, it does not support the selected arm of the discriminated union. For an example, see READ_PLUS (Section 14.10).
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 4 lines · Tom
We did not want many new operators and also wanted the operators to be extensible. With this approach, you can define a new arm of the discriminated union, not have to implement it, and not burn an operator.
That seems very much like a non-argument. If saving operators was a good argument the NFS operations should be
MULTIPLEX1 with many sub opcodes, followed by MULTIPLEX2 once it fills up.
Quoted 3 lines
Some of the history is captured here:
http://www.ietf.org/mail-archive/web/nfsv4/current/msg11235.html
That mail seems to draw the wrong conclusion that a hole punching or a preallocation are equivalent to a server side copy from /dev/zero.
Treating a pattern write as a server side copy is fine and I'd fully support that. Hole punching and preallocation on the other hand are primarily metadata operations that reserve or free space. They only happen to zero out the range as zero is the most convenient well known pattern to avoid stale data exposure.
I think Chuck's reply summarizes very well why a pattern initialization should not be mixed with an actual data write.
Quoted 1 line
http://www.ietf.org/proceedings/84/slides/slides-84-nfsv4-1.pdf (slide 6)
That side seems to inadvertently sum up a lot of what's wrong with merging hole punching and preallocations into some form of super write:
- COMMIT doesn't really apply to pure metadata operations like a hole punch and preallocation, so fitting it into a WRITE that expects COMMIT causes all kinds of problems (as we saw in the thread about Annas implementation).
- requiring the server to be able to handle offloads for these operations does not make any sense, because they are again very quick metadata operation, and not long running operation like a pattern initialization on the server.
Note that the arbitrary pattern initialization over multiple blocks is a very different operation from a space allocation even if the latter happens to also zero the range as a side effect.
Quoted 10 lines
It doesn't capture the intent of
NFS4ERR_UNION_NOTSUPPin this decision.11.1.1.1.
NFS4ERR_UNION_NOTSUPP(Error Code 10090)One of the arguments to the operation is a discriminated union and while the server supports the given operation, it does not support the selected arm of the discriminated union. For an example, see
READ_PLUS(Section 14.10).
Btw, there is an odd use of this error in 14.7.3.:
"WRITE_PLUS has to support all of the errors which are returned by
WRITE plus NFS4ERR_UNION_NOTSUPP. If the client asks for a hole and the server does not support that arm of the discriminated union, but does support one or more additional arms, it can signal to the client that it supports the operation, but not the arm with NFS4ERR_UNION_NOTSUPP."
This does not specicly writes but appears to assume hole punching is the only optional arm. On the other hand to me it appears the only interesting arm, with the data arm buying nothing over WRITE in 4.2 and thus being entirely superflous, and ADHs being a complicated map to Unix filesystems on the backend.
Re: [PATCH 4/4] NFSD: Implement SEEK
In reply to Anna SchumakerBtw, I just noticed that current nfs mainline fails xfstests 286 which
test SEEK_DATA/SEEK_HOLE. That sounds like a useful test case for your
patches.
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 4 lines · Christoph Hellwig
Btw, I just noticed that current nfs mainline fails xfstests 286 which
test SEEK_DATA/SEEK_HOLE. That sounds like a useful test case for your
patches.
Yeah, I've been using it to test this. I'm having trouble getting the last few parts of both test 11 and 12 to pass, which is better than what's happening now!
Anna
Re: [PATCH 4/4] NFSD: Implement SEEK
In reply to Christoph HellwigQuoted 4 lines · Christoph Hellwig
Btw, I just noticed that current nfs mainline fails xfstests 286 which
test SEEK_DATA/SEEK_HOLE. That sounds like a useful test case for your
patches.
I read too quickly. I've been using 285 to test against, but I'll see what 286 gets me! Thanks for the tip :)
Re: [PATCH 4/4] NFSD: Implement SEEK
Quoted 7 lines · Anna Schumaker
Quoted 4 lines · Christoph Hellwig
Btw, I just noticed that current nfs mainline fails xfstests 286 which
test SEEK_DATA/SEEK_HOLE. That sounds like a useful test case for your
patches.I read too quickly. I've been using 285 to test against, but I'll see what 286 gets me! Thanks for the tip :)
I actually meant 285. 286 tests SEEK_HOLE as well but passes already with the dumb default behaviour.
[RFC 5/4] NFSD: Add basic CB_OFFLOAD support
In reply to Anna SchumakerThis patch adds basic offload support to the WRITE_PLUS operation. Since I don't implement OFFLOAD_ABORT, OFFLOAD_REVOKE or OFFLOAD_STATUS this patch is NOT spec compliant and should not be applied without further work.
I'm including this patch to show how I handled offloading to test client code.
--- fs/nfsd/nfs4callback.c | 132 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4proc.c | 56 ++++++++++++++++++--- fs/nfsd/nfs4state.c | 11 +++++ fs/nfsd/state.h | 4 ++ fs/nfsd/xdr4.h | 14 ++++++ fs/nfsd/xdr4cb.h | 9 ++++ 6 files changed, 219 insertions(+), 7 deletions(-)
fs/nfsd/nfs4callback.c +132 -0
@@ -35,6 +35,7 @@#include <linux/sunrpc/svc_xprt.h> #include <linux/slab.h> #include "nfsd.h" +#include "xdr4.h" #include "state.h" #include "netns.h" #include "xdr4cb.h"
@@ -52,6 +53,9 @@ enum {NFSPROC4_CLNT_CB_NULL =0 ,NFSPROC4_CLNT_CB_RECALL ,NFSPROC4_CLNT_CB_SEQUENCE , + +/* NFS v4.2 callback */ + NFSPROC4_CLNT_CB_OFFLOAD, }; struct nfs4_cb_compound_hdr {
@@ -110,6 +114,7 @@ enum nfs_cb_opnum4 {OP_CB_WANTS_CANCELLED =12 ,OP_CB_NOTIFY_LOCK =13 ,OP_CB_NOTIFY_DEVICEID =14 , +OP_CB_OFFLOAD =15 , OP_CB_ILLEGAL = 10044 };
@@ -469,6 +474,31 @@ out_default:return nfs_cb_stat_to_errno (nfserr); } +static void encode_cb_offload4args (struct xdr_stream *xdr, +const struct nfsd4_cb_offload *offload, +struct nfs4_cb_compound_hdr *hdr) +{ +__be32 *p; + +if (hdr->minorversion <2 ) +return ; + +encode_nfs_cb_opnum4 (xdr,OP_CB_OFFLOAD ); +encode_nfs_fh4 (xdr, &offload->co_fh); +encode_stateid4 (xdr, &offload->co_res.wr_stid->sc_stateid); + + p =xdr_reserve_space (xdr,4 ); + *p =cpu_to_be32 (1 ); +encode_stateid4 (xdr, &offload->co_res.wr_stid->sc_stateid); + + p =xdr_reserve_space (xdr,12 +NFS4_VERIFIER_SIZE ); + p =xdr_encode_hyper (p, offload->co_res.wr_bytes_written); + *p++ =cpu_to_be32 (offload->co_res.wr_stable_how); +xdr_encode_opaque_fixed (p, offload->co_res.wr_verifier.data,NFS4_VERIFIER_SIZE ); + + hdr->nops++; +} +/* * NFSv4.0 and NFSv4.1 XDR encode functions *
@@ -505,6 +535,23 @@ static void nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, struct xdr_stream *xdr,encode_cb_nops (&hdr); } +/* +* CB_OFFLOAD +*/ +static void nfs4_xdr_enc_cb_offload (struct rpc_rqst *req,struct xdr_stream *xdr, +const struct nfsd4_callback *cb) +{ +const struct nfsd4_cb_offload *args = cb->cb_op; +struct nfs4_cb_compound_hdr hdr = { + .ident = cb->cb_clp->cl_cb_ident, + .minorversion = cb->cb_minorversion, + }; + +encode_cb_compound4args (xdr, &hdr); +encode_cb_sequence4args (xdr, cb, &hdr); +encode_cb_offload4args (xdr, args, &hdr); +encode_cb_nops (&hdr); +}/* * NFSv4.0 and NFSv4.1 XDR decode functions
@@ -552,6 +599,36 @@ out: }/* +* CB_OFFLOAD +*/ +static int nfs4_xdr_dec_cb_offload (struct rpc_rqst *rqstp,struct xdr_stream *xdr, +struct nfsd4_callback *cb) +{ +struct nfs4_cb_compound_hdr hdr; +enum nfsstat4 nfserr; +int status; + + status =decode_cb_compound4res (xdr, &hdr); +if (unlikely (status)) + goto out; + +if (cb !=NULL ) { + status =decode_cb_sequence4res (xdr, cb); +if (unlikely (status)) + goto out; + } + + status =decode_cb_op_status (xdr,OP_CB_OFFLOAD , &nfserr); +if (unlikely (status)) + goto out; +if (unlikely (nfserr !=NFS4_OK )) + status =nfs_cb_stat_to_errno (nfserr); + +out: +return status; +} + +/* * RPC procedure tables */ #define PROC (proc, call, argtype, restype) \
@@ -568,6 +645,7 @@ out:static struct rpc_procinfo nfs4_cb_procedures[] = {PROC (CB_NULL ,NULL , cb_null, cb_null),PROC (CB_RECALL ,COMPOUND , cb_recall, cb_recall), +PROC (CB_OFFLOAD ,COMPOUND , cb_offload, cb_offload), };static struct rpc_version nfs_cb_version4 = {
@@ -1036,3 +1114,57 @@ void nfsd4_cb_recall(struct nfs4_delegation *dp)run_nfsd4_cb (&dp->dl_recall); } + +static void nfsd4_cb_offload_done (struct rpc_task *task,void *calldata) +{ +struct nfsd4_callback *cb = calldata; +struct nfs4_client *clp = cb->cb_clp; +struct rpc_clnt *current_rpc_client = clp->cl_cb_client; + +nfsd4_cb_done (task, calldata); + +if (current_rpc_client != task->tk_client) +return ; + +if (cb->cb_done) +return ; + +if (task->tk_status !=0 ) +nfsd4_mark_cb_down (clp, task->tk_status); + cb->cb_done = true; +} + +static void nfsd4_cb_offload_release (void *calldata) +{ +struct nfsd4_callback *cb = calldata; + struct nfsd4_cb_offload *offload = container_of(cb, struct nfsd4_cb_offload, co_callback); + +if (cb->cb_done) { +nfs4_free_offload_stateid (offload->co_res.wr_stid); +kfree (offload); + } +} + +static const struct rpc_call_ops nfsd4_cb_offload_ops = { + .rpc_call_prepare = nfsd4_cb_prepare, + .rpc_call_done = nfsd4_cb_offload_done, + .rpc_release = nfsd4_cb_offload_release, +}; + +void nfsd4_cb_offload (struct nfsd4_cb_offload *offload) +{ +struct nfsd4_callback *cb = &offload->co_callback; + + cb->cb_op = offload; + cb->cb_clp = offload->co_res.wr_stid->sc_client; + cb->cb_msg.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_OFFLOAD ]; + cb->cb_msg.rpc_argp = cb; + cb->cb_msg.rpc_resp = cb; + + cb->cb_ops = &nfsd4_cb_offload_ops; + +INIT_LIST_HEAD (&cb->cb_per_client); + cb->cb_done = true; + +run_nfsd4_cb (cb); +}
fs/nfsd/nfs4proc.c +49 -7
78 lines not on this page · raw message
fs/nfsd/nfs4state.c +11 -0
@@ -364,6 +364,11 @@ static struct nfs4_ol_stateid * nfs4_alloc_stateid(struct nfs4_client *clp)return openlockstateid (nfs4_alloc_stid (clp, stateid_slab)); } +struct nfs4_stid *nfs4_alloc_offload_stateid (struct nfs4_client *clp) +{ +return nfs4_alloc_stid (clp, stateid_slab); +} +static struct nfs4_delegation *alloc_init_deleg (struct nfs4_client *clp,struct nfs4_ol_stateid *stp,struct svc_fh *current_fh) {
@@ -613,6 +618,12 @@ static void free_generic_stateid(struct nfs4_ol_stateid *stp)kmem_cache_free (stateid_slab, stp); } +void nfs4_free_offload_stateid (struct nfs4_stid *stid) +{ +remove_stid (stid); +kmem_cache_free (stateid_slab, stid); +} +static void release_lock_stateid (struct nfs4_ol_stateid *stp) {struct file *file;
fs/nfsd/state.h +4 -0
@@ -452,6 +452,7 @@ static inline struct nfs4_ol_stateid *openlockstateid(struct nfs4_stid *s)#define WR_STATE 0x00000020struct nfsd4_compound_state ; +struct nfsd4_cb_offload ;struct nfsd_net ; extern __be32 nfs4_preprocess_stateid_op(struct net *net,
@@ -472,6 +473,7 @@ extern void nfsd4_probe_callback(struct nfs4_client *clp);extern void nfsd4_probe_callback_sync (struct nfs4_client *clp);extern void nfsd4_change_callback (struct nfs4_client *clp,struct nfs4_cb_conn *);extern void nfsd4_cb_recall (struct nfs4_delegation *dp); +extern void nfsd4_cb_offload (struct nfsd4_cb_offload *);extern int nfsd4_create_callback_queue (void );extern void nfsd4_destroy_callback_queue (void );extern void nfsd4_shutdown_callback (struct nfs4_client *);
@@ -480,6 +482,8 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name, struct nfsd_net *nn);extern bool nfs4_has_reclaimed_state (const char *name,struct nfsd_net *nn);extern void put_client_renew (struct nfs4_client *clp); +extern struct nfs4_stid *nfs4_alloc_offload_stateid (struct nfs4_client *); +extern void nfs4_free_offload_stateid (struct nfs4_stid *);/* nfs4recover operations */ extern int nfsd4_client_tracking_init (struct net *net);
fs/nfsd/xdr4.h +14 -0
@@ -451,6 +451,20 @@ struct nfsd4_write_plus {struct nfsd42_write_res wp_res; }; +struct nfsd4_cb_offload { +union { +struct nfsd4_write_plus write_plus; + } co_u; + +struct knfsd_fh co_fh; +struct file *co_file; +struct net *co_net; + +struct nfsd42_write_res co_res; +struct work_struct co_work; +struct nfsd4_callback co_callback; +}; +struct nfsd4_seek {/* request */ stateid_t seek_stateid;
fs/nfsd/xdr4cb.h +9 -0
@@ -21,3 +21,12 @@#define NFS4_dec_cb_recall_sz (cb_compound_dec_hdr_sz + \ cb_sequence_dec_sz + \ op_dec_sz) + +#define NFS4_enc_cb_offload_sz (cb_compound_enc_hdr_sz + \ + cb_sequence_enc_sz + \ + 1 + enc_stateid_sz + 2 + 1 + \ + XDR_QUADLEN(NFS4_VERIFIER_SIZE)) + +#define NFS4_dec_cb_offload_sz (cb_compound_dec_hdr_sz + \ + cb_sequence_dec_sz + \ + op_dec_sz)
Signature · 1 line
1.8.4.1
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 4 lines · Anna Schumaker
This patch adds basic offload support to the
WRITE_PLUSoperation. Since I don't implementOFFLOAD_ABORT,OFFLOAD_REVOKEorOFFLOAD_STATUSthis patch is NOT spec compliant and should not be applied without further work.
Ugh. I don't understand why we need asynchronous modes for all these operations.
--b.
Quoted 388 lines
I'm including this patch to show how I handled offloading to test client code.
--- fs/nfsd/nfs4callback.c | 132 +++++++++++++++++++++++++++++++++++++++++++++++++ fs/nfsd/nfs4proc.c | 56 ++++++++++++++++++--- fs/nfsd/nfs4state.c | 11 +++++ fs/nfsd/state.h | 4 ++ fs/nfsd/xdr4.h | 14 ++++++ fs/nfsd/xdr4cb.h | 9 ++++ 6 files changed, 219 insertions(+), 7 deletions(-)fs/nfsd/nfs4callback.c +132
-0@@ -35,6 +35,7 @@#include <linux/sunrpc/svc_xprt.h> #include <linux/slab.h> #include "nfsd.h" +#include "xdr4.h" #include "state.h" #include "netns.h" #include "xdr4cb.h" @@ -52,6 +53,9 @@ enum {NFSPROC4_CLNT_CB_NULL =0 ,NFSPROC4_CLNT_CB_RECALL ,NFSPROC4_CLNT_CB_SEQUENCE , + +/* NFS v4.2 callback */ + NFSPROC4_CLNT_CB_OFFLOAD, }; struct nfs4_cb_compound_hdr {@@ -110,6 +114,7 @@ enum nfs_cb_opnum4 {OP_CB_WANTS_CANCELLED =12 ,OP_CB_NOTIFY_LOCK =13 ,OP_CB_NOTIFY_DEVICEID =14 , +OP_CB_OFFLOAD =15 , OP_CB_ILLEGAL = 10044 };@@ -469,6 +474,31 @@ out_default:return nfs_cb_stat_to_errno (nfserr); } +static void encode_cb_offload4args (struct xdr_stream *xdr, +const struct nfsd4_cb_offload *offload, +struct nfs4_cb_compound_hdr *hdr) +{ +__be32 *p; + +if (hdr->minorversion <2 ) +return ; + +encode_nfs_cb_opnum4 (xdr,OP_CB_OFFLOAD ); +encode_nfs_fh4 (xdr, &offload->co_fh); +encode_stateid4 (xdr, &offload->co_res.wr_stid->sc_stateid); + + p =xdr_reserve_space (xdr,4 ); + *p =cpu_to_be32 (1 ); +encode_stateid4 (xdr, &offload->co_res.wr_stid->sc_stateid); + + p =xdr_reserve_space (xdr,12 +NFS4_VERIFIER_SIZE ); + p =xdr_encode_hyper (p, offload->co_res.wr_bytes_written); + *p++ =cpu_to_be32 (offload->co_res.wr_stable_how); +xdr_encode_opaque_fixed (p, offload->co_res.wr_verifier.data,NFS4_VERIFIER_SIZE ); + + hdr->nops++; +} +/* * NFSv4.0 and NFSv4.1 XDR encode functions * @@ -505,6 +535,23 @@ static void nfs4_xdr_enc_cb_recall(struct rpc_rqst *req, struct xdr_stream *xdr,encode_cb_nops (&hdr); } +/* +* CB_OFFLOAD +*/ +static void nfs4_xdr_enc_cb_offload (struct rpc_rqst *req,struct xdr_stream *xdr, +const struct nfsd4_callback *cb) +{ +const struct nfsd4_cb_offload *args = cb->cb_op; +struct nfs4_cb_compound_hdr hdr = { + .ident = cb->cb_clp->cl_cb_ident, + .minorversion = cb->cb_minorversion, + }; + +encode_cb_compound4args (xdr, &hdr); +encode_cb_sequence4args (xdr, cb, &hdr); +encode_cb_offload4args (xdr, args, &hdr); +encode_cb_nops (&hdr); +}/* * NFSv4.0 and NFSv4.1 XDR decode functions @@ -552,6 +599,36 @@ out: }/* +* CB_OFFLOAD +*/ +static int nfs4_xdr_dec_cb_offload (struct rpc_rqst *rqstp,struct xdr_stream *xdr, +struct nfsd4_callback *cb) +{ +struct nfs4_cb_compound_hdr hdr; +enum nfsstat4 nfserr; +int status; + + status =decode_cb_compound4res (xdr, &hdr); +if (unlikely (status)) + goto out; + +if (cb !=NULL ) { + status =decode_cb_sequence4res (xdr, cb); +if (unlikely (status)) + goto out; + } + + status =decode_cb_op_status (xdr,OP_CB_OFFLOAD , &nfserr); +if (unlikely (status)) + goto out; +if (unlikely (nfserr !=NFS4_OK )) + status =nfs_cb_stat_to_errno (nfserr); + +out: +return status; +} + +/* * RPC procedure tables */ #define PROC (proc, call, argtype, restype) \@@ -568,6 +645,7 @@ out:static struct rpc_procinfo nfs4_cb_procedures[] = {PROC (CB_NULL ,NULL , cb_null, cb_null),PROC (CB_RECALL ,COMPOUND , cb_recall, cb_recall), +PROC (CB_OFFLOAD ,COMPOUND , cb_offload, cb_offload), };static struct rpc_version nfs_cb_version4 = {@@ -1036,3 +1114,57 @@ void nfsd4_cb_recall(struct nfs4_delegation *dp)run_nfsd4_cb (&dp->dl_recall); } + +static void nfsd4_cb_offload_done (struct rpc_task *task,void *calldata) +{ +struct nfsd4_callback *cb = calldata; +struct nfs4_client *clp = cb->cb_clp; +struct rpc_clnt *current_rpc_client = clp->cl_cb_client; + +nfsd4_cb_done (task, calldata); + +if (current_rpc_client != task->tk_client) +return ; + +if (cb->cb_done) +return ; + +if (task->tk_status !=0 ) +nfsd4_mark_cb_down (clp, task->tk_status); + cb->cb_done = true; +} + +static void nfsd4_cb_offload_release (void *calldata) +{ +struct nfsd4_callback *cb = calldata; + struct nfsd4_cb_offload *offload = container_of(cb, struct nfsd4_cb_offload, co_callback); + +if (cb->cb_done) { +nfs4_free_offload_stateid (offload->co_res.wr_stid); +kfree (offload); + } +} + +static const struct rpc_call_ops nfsd4_cb_offload_ops = { + .rpc_call_prepare = nfsd4_cb_prepare, + .rpc_call_done = nfsd4_cb_offload_done, + .rpc_release = nfsd4_cb_offload_release, +}; + +void nfsd4_cb_offload (struct nfsd4_cb_offload *offload) +{ +struct nfsd4_callback *cb = &offload->co_callback; + + cb->cb_op = offload; + cb->cb_clp = offload->co_res.wr_stid->sc_client; + cb->cb_msg.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_OFFLOAD ]; + cb->cb_msg.rpc_argp = cb; + cb->cb_msg.rpc_resp = cb; + + cb->cb_ops = &nfsd4_cb_offload_ops; + +INIT_LIST_HEAD (&cb->cb_per_client); + cb->cb_done = true; + +run_nfsd4_cb (cb); +}fs/nfsd/nfs4proc.c +49
-7@@ -1028,22 +1028,64 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status; } -static __be32 -nfsd4_write_plus_hole (struct file *file,struct nfsd4_write_plus *writeplus, -struct net *net) +static void +nfsd4_offload_work (struct nfsd4_cb_offload *offload, +struct nfsd4_compound_state *cstate, +struct nfsd42_write_res *write_res, +void (*offload_func)(struct work_struct *)) +{ + offload->co_res.wr_stid =nfs4_alloc_offload_stateid (cstate->session->se_client); +memcpy (&offload->co_fh, &cstate->current_fh,sizeof (struct knfsd_fh )); + + write_res->wr_stid = offload->co_res.wr_stid; + write_res->wr_bytes_written =0 ; + write_res->wr_stable_how =NFS_UNSTABLE ; + +nfsd4_init_callback (&offload->co_callback); +INIT_WORK (&offload->co_work, offload_func); +schedule_work (&offload->co_work); +} + +static void +nfsd4_write_plus_hole_async (struct work_struct *work) { +struct nfsd4_cb_offload *offload; +struct nfsd4_write_plus *writeplus;__be32 status; - status =nfsd4_vfs_fallocate (file, writeplus->wp_allocated, + offload = container_of(work, struct nfsd4_cb_offload, co_work); + writeplus = &offload->co_u.write_plus; + + status =nfsd4_vfs_fallocate (offload->co_file, writeplus->wp_allocated, writeplus->wp_offset, writeplus->wp_length);if (status == nfs_ok) { writeplus->wp_res.wr_stid =NULL ; writeplus->wp_res.wr_bytes_written = writeplus->wp_length; writeplus->wp_res.wr_stable_how =NFS_FILE_SYNC ; -gen_boot_verifier (&writeplus->wp_res.wr_verifier, net); +gen_boot_verifier (&writeplus->wp_res.wr_verifier, offload->co_net); } -return status; +fput (offload->co_file); +nfsd4_cb_offload (offload); +} + +static __be32 +nfsd4_write_plus_hole (struct file *file,struct nfsd4_write_plus *writeplus, +struct nfsd4_compound_state *cstate,struct net *net) +{ +struct nfsd4_cb_offload *offload; + + offload =kmalloc (sizeof (struct nfsd4_cb_offload ),GFP_KERNEL ); +if (!offload) +return nfserrno (PTR_ERR (offload)); + +memcpy (&offload->co_u.write_plus, writeplus,sizeof (struct nfsd4_write_plus )); + offload->co_file =get_file (file); + offload->co_net = net; + +nfsd4_offload_work (offload, cstate, &writeplus->wp_res, nfsd4_write_plus_hole_async); + +return 0 ; } static __be32@@ -1060,7 +1102,7 @@ nfsd4_write_plus(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,return status;if (writeplus->wp_data_content ==NFS4_CONTENT_HOLE ) -return nfsd4_write_plus_hole (file, writeplus, net); +return nfsd4_write_plus_hole (file, writeplus, cstate, net);return nfserr_union_notsupp; }fs/nfsd/nfs4state.c +11
-0@@ -364,6 +364,11 @@ static struct nfs4_ol_stateid * nfs4_alloc_stateid(struct nfs4_client *clp)return openlockstateid (nfs4_alloc_stid (clp, stateid_slab)); } +struct nfs4_stid *nfs4_alloc_offload_stateid (struct nfs4_client *clp) +{ +return nfs4_alloc_stid (clp, stateid_slab); +} +static struct nfs4_delegation *alloc_init_deleg (struct nfs4_client *clp,struct nfs4_ol_stateid *stp,struct svc_fh *current_fh) {@@ -613,6 +618,12 @@ static void free_generic_stateid(struct nfs4_ol_stateid *stp)kmem_cache_free (stateid_slab, stp); } +void nfs4_free_offload_stateid (struct nfs4_stid *stid) +{ +remove_stid (stid); +kmem_cache_free (stateid_slab, stid); +} +static void release_lock_stateid (struct nfs4_ol_stateid *stp) {struct file *file;fs/nfsd/state.h +4
-0@@ -452,6 +452,7 @@ static inline struct nfs4_ol_stateid *openlockstateid(struct nfs4_stid *s)#define WR_STATE 0x00000020struct nfsd4_compound_state ; +struct nfsd4_cb_offload ;struct nfsd_net ; extern __be32 nfs4_preprocess_stateid_op(struct net *net,@@ -472,6 +473,7 @@ extern void nfsd4_probe_callback(struct nfs4_client *clp);extern void nfsd4_probe_callback_sync (struct nfs4_client *clp);extern void nfsd4_change_callback (struct nfs4_client *clp,struct nfs4_cb_conn *);extern void nfsd4_cb_recall (struct nfs4_delegation *dp); +extern void nfsd4_cb_offload (struct nfsd4_cb_offload *);extern int nfsd4_create_callback_queue (void );extern void nfsd4_destroy_callback_queue (void );extern void nfsd4_shutdown_callback (struct nfs4_client *);@@ -480,6 +482,8 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name, struct nfsd_net *nn);extern bool nfs4_has_reclaimed_state (const char *name,struct nfsd_net *nn);extern void put_client_renew (struct nfs4_client *clp); +extern struct nfs4_stid *nfs4_alloc_offload_stateid (struct nfs4_client *); +extern void nfs4_free_offload_stateid (struct nfs4_stid *);/* nfs4recover operations */ extern int nfsd4_client_tracking_init (struct net *net);fs/nfsd/xdr4.h +14
-0@@ -451,6 +451,20 @@ struct nfsd4_write_plus {struct nfsd42_write_res wp_res; }; +struct nfsd4_cb_offload { +union { +struct nfsd4_write_plus write_plus; + } co_u; + +struct knfsd_fh co_fh; +struct file *co_file; +struct net *co_net; + +struct nfsd42_write_res co_res; +struct work_struct co_work; +struct nfsd4_callback co_callback; +}; +struct nfsd4_seek {/* request */ stateid_t seek_stateid;fs/nfsd/xdr4cb.h +9
-0@@ -21,3 +21,12 @@#define NFS4_dec_cb_recall_sz (cb_compound_dec_hdr_sz + \ cb_sequence_dec_sz + \ op_dec_sz) + +#define NFS4_enc_cb_offload_sz (cb_compound_enc_hdr_sz + \ + cb_sequence_enc_sz + \ + 1 + enc_stateid_sz + 2 + 1 + \ + XDR_QUADLEN(NFS4_VERIFIER_SIZE)) + +#define NFS4_dec_cb_offload_sz (cb_compound_dec_hdr_sz + \ + cb_sequence_dec_sz + \ + op_dec_sz)Signature · 1 line
1.8.4.1
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 8 lines · J. Bruce Fields
Quoted 4 lines · Anna Schumaker
This patch adds basic offload support to the
WRITE_PLUSoperation. Since I don't implementOFFLOAD_ABORT,OFFLOAD_REVOKEorOFFLOAD_STATUSthis patch is NOT spec compliant and should not be applied without further work.Ugh. I don't understand why we need asynchronous modes for all these operations.
Hole punches as implemented by any filesystem at the moement are pure metadata manipulations and should not require "async" versions that offload to a workqueue.
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · J. Bruce Fields
Quoted 4 lines · Anna Schumaker
This patch adds basic offload support to the
WRITE_PLUSoperation. Since I don't implementOFFLOAD_ABORT,OFFLOAD_REVOKEorOFFLOAD_STATUSthis patch is NOT spec compliant and should not be applied without further work.
Quoted 6 lines
Quoted 2 lines
Ugh. I don't understand why we need asynchronous modes for all these operations.
Hole punches as implemented by any filesystem at the moement are pure metadata manipulations and should not require "async" versions that offload to a workqueue.
Yeah, understood, I'm glad we're not implementing that, I just wonder why every one of these operations (COPY, WRITE_PLUS, etc.) has to have this asynchronous option.
The client's still stuck implementing it even if the server does, it's extra protocol verbage even if nobody uses it, and I'm not completely clear what it's for.
--b.
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 7 lines · J. Bruce Fields
Yeah, understood, I'm glad we're not implementing that, I just wonder why every one of these operations (COPY,
WRITE_PLUS, etc.) has to have this asynchronous option.The client's still stuck implementing it even if the server does, it's extra protocol verbage even if nobody uses it, and I'm not completely clear what it's for.
Seems like Trond answered that question: feature creep that people without the slightest sense of abstraction tried to overload over a few operations.
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 5 lines · Christoph Hellwig
Quoted 4 lines · J. Bruce Fields
Yeah, understood, I'm glad we're not implementing that, I just wonder why every one of these operations (COPY,
WRITE_PLUS, etc.) has to have this asynchronous option.
Quoted 7 lines
Quoted 3 lines
The client's still stuck implementing it even if the server does, it's extra protocol verbage even if nobody uses it, and I'm not completely clear what it's for.
Seems like Trond answered that question: feature creep that people without the slightest sense of abstraction tried to overload over a few operations.
Your complaint as I understand it is that quick and long-running operations were combined into one one operation when they would have better been separated. I agree.
But I also don't understand why the long-running operations need an async option. Maybe they do, I just don't understand why.
Alternatives might include just letting the operation hog a request slot for the whole time, or making it work in chunks. (E.g. allowing COPY to return short writes.)
--b.
Re: [RFC 5/4] NFSD: Add basic CB_OFFLOAD support
Quoted 3 lines · J. Bruce Fields
Your complaint as I understand it is that quick and long-running operations were combined into one one operation when they would have better been separated. I agree.
Not just quick and long running but semantically different - preallocating space and writing zeroes over every byte are fundamentally different operations.
Quoted 2 lines
But I also don't understand why the long-running operations need an async option. Maybe they do, I just don't understand why.
I didn't even bother questioning that, but it's not a bad question either.
Quoted 3 lines · Anna Schumaker
Patch 3 (Add
WRITE_PLUSsupport for hole punches) exports thedo_fallocate()function fromfs/open.c. Should this be its own patch submitted somewhere else? Is there a better way to get into the VFS fallocate routines?
Should defintively be a separate patch, and Cc linux-fsdevel on the whole series. Also I'd suggest to rename it to vfs_fallocate given that it takes a file structure.