ceph+fscrypt: context, filename and symlink support
Subsystems (4)
v8 cover ceph+fscrypt: context, filename and symlink support
v8: bugfixes, comment cleanups, etc.
It's been a little while since my last posting of this series. Not a lot has changed in this series since then. This is mostly addressing review comments on the v7 posting. Many thanks to Eric, Xiubo and Luis for helping review the last set.
There are a few smaller bugfixes, some comment fixes, and a new helper to determine the length of a ceph_fscrypt_auth field was added.
This support requires changes to the MDS that are currently being tracked here:
https://github.com/ceph/ceph/pull/41284
This patchset also requires a patch that's not yet in tree from Eric Biggers:
[PATCH] fscrypt: align Base64 encoding with RFC 4648 base64url
Hopefully, that will go into v5.15.
Work continues on the content piece, but I don't think we want to merge any of this until we have a fully-working prototype. I'm planning to send a ceph+fscrypt state of the union email in a bit that covers the state of the larger project.
Stay tuned...
Jeff Layton (24):
vfs: export new_inode_pseudo
fscrypt: export fscrypt_base64url_encode and fscrypt_base64url_decode
fscrypt: export fscrypt_fname_encrypt and fscrypt_fname_encrypted_size
fscrypt: add fscrypt_context_for_new_inode
ceph: preallocate inode for ops that may create one
ceph: parse new fscrypt_auth and fscrypt_file fields in inode traces
ceph: add fscrypt_* handling to caps.c
ceph: crypto context handling for ceph
ceph: add ability to set fscrypt_auth via setattr
ceph: implement -o test_dummy_encryption mount option
ceph: add fscrypt ioctls
ceph: decode alternate_name in lease info
ceph: make ceph_msdc_build_path use ref-walk
ceph: add encrypted fname handling to ceph_mdsc_build_path
ceph: send altname in MClientRequest
ceph: encode encrypted name in dentry release
ceph: properly set DCACHE_NOKEY_NAME flag in lookup
ceph: make d_revalidate call fscrypt revalidator for encrypted
dentries
ceph: add helpers for converting names for userland presentation
ceph: add fscrypt support to ceph_fill_trace
ceph: add support to readdir for encrypted filenames
ceph: create symlinks with encrypted and base64-encoded targets
ceph: make ceph_get_name decrypt filenames
ceph: add a new ceph.fscrypt.auth vxattr
fs/ceph/Makefile | 1 + fs/ceph/acl.c | 4 +- fs/ceph/caps.c | 93 ++++++++-- fs/ceph/crypto.c | 254 ++++++++++++++++++++++++++ fs/ceph/crypto.h | 129 +++++++++++++ fs/ceph/dir.c | 198 +++++++++++++++----- fs/ceph/export.c | 44 +++-- fs/ceph/file.c | 64 ++++--- fs/ceph/inode.c | 304 ++++++++++++++++++++++++++++--- fs/ceph/ioctl.c | 83 +++++++++ fs/ceph/mds_client.c | 342 +++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 22 ++- fs/ceph/super.c | 82 ++++++++- fs/ceph/super.h | 31 +++- fs/ceph/xattr.c | 25 +++ fs/crypto/fname.c | 40 +++- fs/crypto/fscrypt_private.h | 9 +- fs/crypto/hooks.c | 6 +- fs/crypto/policy.c | 34 +++- fs/inode.c | 1 + include/linux/ceph/ceph_fs.h | 21 ++- include/linux/fscrypt.h | 10 + 22 files changed, 1572 insertions(+), 225 deletions(-) create mode 100644 fs/ceph/crypto.c create mode 100644 fs/ceph/crypto.h
Signature · 1 line
2.31.1
RFC PATCH v8 1/24 vfs: export new_inode_pseudo
Ceph needs to be able to allocate inodes ahead of a create that might involve a fscrypt-encrypted inode. new_inode() almost fits the bill, but it puts the inode on the sb->s_inodes list and when we go to hash it, that might be done again.
We could work around that by setting I_CREATING on the new inode, but that causes ilookup5 to return -ESTALE if something tries to find it before I_NEW is cleared. This is desirable behavior for most filesystems, but doesn't work for ceph.
To work around all of this, just use new_inode_pseudo which doesn't add it to the sb->s_inodes list.
--- fs/inode.c | 1 + 1 file changed, 1 insertion(+)
fs/inode.c +1 -0
@@ -941,6 +941,7 @@ struct inode *new_inode_pseudo(struct super_block *sb) }return inode; } +EXPORT_SYMBOL (new_inode_pseudo);/** * new_inode - obtain an inode
Signature · 1 line
2.31.1
RFC PATCH v8 2/24 fscrypt: export fscrypt_base64url_encode and fscrypt_base64url_decode
In reply to Jeff LaytonCeph is going to add fscrypt support, but we still want encrypted filenames to be composed of printable characters, so we can maintain compatibility with clients that don't support fscrypt.
We could just adopt fscrypt's current nokey name format, but that is subject to change in the future, and it also contains dirhash fields that we don't need for cephfs. Because of this, we're going to concoct our own scheme for encoding encrypted filenames. It's very similar to fscrypt's current scheme, but doesn't bother with the dirhash fields.
The ceph encoding scheme will use base64 encoding as well, and we also want it to avoid characters that are illegal in filenames. Export the fscrypt base64 encoding/decoding routines so we can use them in ceph's fscrypt implementation.
--- fs/crypto/fname.c | 8 ++++---- include/linux/fscrypt.h | 5 +++++ 2 files changed, 9 insertions(+), 4 deletions(-)
fs/crypto/fname.c +4 -4
@@ -182,8 +182,6 @@ static int fname_decrypt(const struct inode *inode,static const char base64url_table[65 ] ="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_" ; -#define FSCRYPT_BASE64URL_CHARS (nbytes) DIV_ROUND_UP((nbytes) * 4, 3) -/** * fscrypt_base64url_encode() - base64url-encode some binary data * @src: the binary data to encode
@@ -198,7 +196,7 @@ static const char base64url_table[65] =* Return: length of the resulting base64url-encoded string in bytes. * This will be equal to FSCRYPT_BASE64URL_CHARS(srclen). */ -static int fscrypt_base64url_encode (const u8 *src,int srclen,char *dst) +int fscrypt_base64url_encode (const u8 *src,int srclen,char *dst) {u32 ac =0 ;int bits =0 ;
@@ -217,6 +215,7 @@ static int fscrypt_base64url_encode(const u8 *src, int srclen, char *dst) *cp++ = base64url_table[(ac << (6 - bits)) &0x3f ];return cp - dst; } +EXPORT_SYMBOL (fscrypt_base64url_encode);/** * fscrypt_base64url_decode() - base64url-decode a string
@@ -233,7 +232,7 @@ static int fscrypt_base64url_encode(const u8 *src, int srclen, char *dst)* Return: the length of the resulting decoded binary data in bytes, * or -1 if the string isn't a valid base64url string. */ -static int fscrypt_base64url_decode (const char *src,int srclen,u8 *dst) +int fscrypt_base64url_decode (const char *src,int srclen,u8 *dst) {u32 ac =0 ;int bits =0 ;
@@ -256,6 +255,7 @@ static int fscrypt_base64url_decode(const char *src, int srclen, u8 *dst)return -1 ;return bp - dst; } +EXPORT_SYMBOL (fscrypt_base64url_decode); bool fscrypt_fname_encrypted_size(const union fscrypt_policy *policy, u32 orig_len, u32 max_len,
include/linux/fscrypt.h +5 -0
@@ -46,6 +46,9 @@ struct fscrypt_name {/* Maximum value for the third parameter of fscrypt_operations.set_context(). */ #define FSCRYPT_SET_CONTEXT_MAX_SIZE 40 +/* len of resulting string (sans NUL terminator) after base64 encoding nbytes */ +#define FSCRYPT_BASE64URL_CHARS (nbytes) DIV_ROUND_UP((nbytes) * 4, 3) +#ifdef CONFIG_FS_ENCRYPTION /* * fscrypt superblock flags
@@ -207,6 +210,8 @@ void fscrypt_free_inode(struct inode *inode);int fscrypt_drop_inode (struct inode *inode);/* fname.c */ +int fscrypt_base64url_encode (const u8 *src,int len,char *dst); +int fscrypt_base64url_decode (const char *src,int len,u8 *dst);int fscrypt_setup_filename (struct inode *inode,const struct qstr *iname,int lookup,struct fscrypt_name *fname);
Signature · 1 line
2.31.1
RFC PATCH v8 3/24 fscrypt: export fscrypt_fname_encrypt and fscrypt_fname_encrypted_size
In reply to Jeff LaytonFor ceph, we want to use our own scheme for handling filenames that are are longer than NAME_MAX after encryption and base64 encoding. This allows us to have a consistent view of the encrypted filenames for clients that don't support fscrypt and clients that do but that don't have the key.
Currently, fs/crypto only supports encrypting filenames using fscrypt_setup_filename, but that also handles encoding nokey names. Ceph can't use that because it handles nokey names in a different way.
Export fscrypt_fname_encrypt. Rename fscrypt_fname_encrypted_size to __fscrypt_fname_encrypted_size and add a new wrapper called fscrypt_fname_encrypted_size that takes an inode argument rather than a pointer to a fscrypt_policy union.
--- fs/crypto/fname.c | 32 +++++++++++++++++++++++++++----- fs/crypto/fscrypt_private.h | 9 +++------ fs/crypto/hooks.c | 6 +++--- include/linux/fscrypt.h | 4 ++++ 4 files changed, 37 insertions(+), 14 deletions(-)
fs/crypto/fname.c +27 -5
@@ -130,6 +130,7 @@ int fscrypt_fname_encrypt(const struct inode *inode, const struct qstr *iname,return 0 ; } +EXPORT_SYMBOL (fscrypt_fname_encrypt);/** * fname_decrypt() - decrypt a filename
@@ -257,9 +258,9 @@ int fscrypt_base64url_decode(const char *src, int srclen, u8 *dst) }EXPORT_SYMBOL (fscrypt_base64url_decode); -bool fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, -u32 orig_len,u32 max_len, -u32 *encrypted_len_ret) +bool __fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, +u32 orig_len,u32 max_len, +u32 *encrypted_len_ret) {int padding =4 << (fscrypt_policy_flags (policy) &FSCRYPT_POLICY_FLAGS_PAD_MASK );
@@ -273,6 +274,28 @@ bool fscrypt_fname_encrypted_size(const union fscrypt_policy *policy,return true; } +/** +* fscrypt_fname_encrypted_size() - calculate length of encrypted filename +* @inode: parent inode of dentry name being encrypted +* @orig_len: length of the original filename +* @max_len: maximum length to return +* @encrypted_len_ret: where calculated length should be returned (on success) +* +* Filenames must be padded out to at least the end of an fscrypt block before +* encrypting them. This calculates the length of an encrypted filename. +* +* Return: false if the orig_len is shorter than max_len. Otherwise, true and +* fill out encrypted_len_ret with the length (up to max_len). +*/ +bool fscrypt_fname_encrypted_size (const struct inode *inode,u32 orig_len, +u32 max_len,u32 *encrypted_len_ret) +{ +return __fscrypt_fname_encrypted_size (&inode->i_crypt_info->ci_policy, + orig_len, max_len, + encrypted_len_ret); +} +EXPORT_SYMBOL (fscrypt_fname_encrypted_size); +/** * fscrypt_fname_alloc_buffer() - allocate a buffer for presented filenames * @max_encrypted_len: maximum length of encrypted filenames the buffer will be
@@ -428,8 +451,7 @@ int fscrypt_setup_filename(struct inode *dir, const struct qstr *iname,return ret;if (fscrypt_has_encryption_key (dir)) { -if (!fscrypt_fname_encrypted_size (&dir->i_crypt_info->ci_policy, - iname->len, +if (!fscrypt_fname_encrypted_size (dir, iname->len, dir->i_sb->s_cop->max_namelen, &fname->crypto_buf.len))return -ENAMETOOLONG ;
fs/crypto/fscrypt_private.h +3 -6
@@ -292,14 +292,11 @@ void fscrypt_generate_iv(union fscrypt_iv *iv, u64 lblk_num, const struct fscrypt_info *ci);/* fname.c */ -int fscrypt_fname_encrypt (const struct inode *inode,const struct qstr *iname, -u8 *out,unsigned int olen); -bool fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, -u32 orig_len,u32 max_len, -u32 *encrypted_len_ret); +bool __fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, +u32 orig_len,u32 max_len, +u32 *encrypted_len_ret);/* hkdf.c */ -struct fscrypt_hkdf {struct crypto_shash *hmac_tfm; };
fs/crypto/hooks.c +3 -3
@@ -228,9 +228,9 @@ int fscrypt_prepare_symlink(struct inode *dir, const char *target,* counting it (even though it is meaningless for ciphertext) is simpler * for now since filesystems will assume it is there and subtract it. */ -if (!fscrypt_fname_encrypted_size (policy, len, - max_len -sizeof (struct fscrypt_symlink_data ), - &disk_link->len)) +if (!__fscrypt_fname_encrypted_size (policy, len, + max_len -sizeof (struct fscrypt_symlink_data ), + &disk_link->len))return -ENAMETOOLONG ; disk_link->len +=sizeof (struct fscrypt_symlink_data );
include/linux/fscrypt.h +4 -0
@@ -210,8 +210,12 @@ void fscrypt_free_inode(struct inode *inode);int fscrypt_drop_inode (struct inode *inode);/* fname.c */ +int fscrypt_fname_encrypt (const struct inode *inode,const struct qstr *iname, +u8 *out,unsigned int olen);int fscrypt_base64url_encode (const u8 *src,int len,char *dst);int fscrypt_base64url_decode (const char *src,int len,u8 *dst); +bool fscrypt_fname_encrypted_size (const struct inode *inode,u32 orig_len, +u32 max_len,u32 *encrypted_len_ret);int fscrypt_setup_filename (struct inode *inode,const struct qstr *iname,int lookup,struct fscrypt_name *fname);
Signature · 1 line
2.31.1
RFC PATCH v8 4/24 fscrypt: add fscrypt_context_for_new_inode
In reply to Jeff LaytonMost filesystems just call fscrypt_set_context on new inodes, which usually causes a setxattr. That's a bit late for ceph, which can send along a full set of attributes with the create request.
Doing so allows it to avoid race windows that where the new inode could be seen by other clients without the crypto context attached. It also avoids the separate round trip to the server.
Refactor the fscrypt code a bit to allow us to create a new crypto context, attach it to the inode, and write it to the buffer, but without calling set_context on it. ceph can later use this to marshal the context into the attributes we send along with the create request.
--- fs/crypto/policy.c | 34 ++++++++++++++++++++++++++++------ include/linux/fscrypt.h | 1 + 2 files changed, 29 insertions(+), 6 deletions(-)
fs/crypto/policy.c +28 -6
@@ -664,6 +664,31 @@ const union fscrypt_policy *fscrypt_policy_to_inherit(struct inode *dir)return fscrypt_get_dummy_policy (dir->i_sb); } +/** +* fscrypt_context_for_new_inode() - create an encryption context for a new inode +* @ctx: where context should be written +* @inode: inode from which to fetch policy and nonce +* +* Given an in-core "prepared" (via fscrypt_prepare_new_inode) inode, +* generate a new context and write it to ctx. ctx _must_ be at least +* FSCRYPT_SET_CONTEXT_MAX_SIZE bytes. +* +* Return: size of the resulting context or a negative error code. +*/ +int fscrypt_context_for_new_inode (void *ctx,struct inode *inode) +{ +struct fscrypt_info *ci = inode->i_crypt_info; + +BUILD_BUG_ON (sizeof (union fscrypt_context ) !=FSCRYPT_SET_CONTEXT_MAX_SIZE ); + +/* fscrypt_prepare_new_inode() should have set up the key already. */ +if (WARN_ON_ONCE (!ci)) +return -ENOKEY ; + +return fscrypt_new_context (ctx, &ci->ci_policy, ci->ci_nonce); +} +EXPORT_SYMBOL_GPL (fscrypt_context_for_new_inode); +/** * fscrypt_set_context() - Set the fscrypt context of a new inode * @inode: a new inode
@@ -680,12 +705,9 @@ int fscrypt_set_context(struct inode *inode, void *fs_data)union fscrypt_context ctx;int ctxsize; -/* fscrypt_prepare_new_inode() should have set up the key already. */ -if (WARN_ON_ONCE (!ci)) -return -ENOKEY ; - -BUILD_BUG_ON (sizeof (ctx) !=FSCRYPT_SET_CONTEXT_MAX_SIZE ); - ctxsize =fscrypt_new_context (&ctx, &ci->ci_policy, ci->ci_nonce); + ctxsize =fscrypt_context_for_new_inode (&ctx, inode); +if (ctxsize <0 ) +return ctxsize;/* * This may be the first time the inode number is available, so do any
include/linux/fscrypt.h +1 -0
@@ -178,6 +178,7 @@ int fscrypt_ioctl_get_policy(struct file *filp, void __user *arg);int fscrypt_ioctl_get_policy_ex (struct file *filp,void __user *arg);int fscrypt_ioctl_get_nonce (struct file *filp,void __user *arg);int fscrypt_has_permitted_context (struct inode *parent,struct inode *child); +int fscrypt_context_for_new_inode (void *ctx,struct inode *inode);int fscrypt_set_context (struct inode *inode,void *fs_data);struct fscrypt_dummy_policy {
Signature · 1 line
2.31.1
RFC PATCH v8 5/24 ceph: preallocate inode for ops that may create one
In reply to Jeff LaytonWhen creating a new inode, we need to determine the crypto context before we can transmit the RPC. The fscrypt API has a routine for getting a crypto context before a create occurs, but it requires an inode.
Change the ceph code to preallocate an inode in advance of a create of any sort (open(), mknod(), symlink(), etc). Move the existing code that generates the ACL and SELinux blobs into this routine since that's mostly common across all the different codepaths.
In most cases, we just want to allow ceph_fill_trace to use that inode after the reply comes in, so add a new field to the MDS request for it (r_new_inode).
The async create codepath is a bit different though. In that case, we want to hash the inode in advance of the RPC so that it can be used before the reply comes in. If the call subsequently fails with -EJUKEBOX, then just put the references and clean up the as_ctx. Note that with this change, we now need to regenerate the as_ctx when this occurs, but it's quite rare for it to happen.
--- fs/ceph/dir.c | 70 ++++++++++++++++++++----------------- fs/ceph/file.c | 62 ++++++++++++++++++++------------- fs/ceph/inode.c | 82 ++++++++++++++++++++++++++++++++++++++++---- fs/ceph/mds_client.c | 3 +- fs/ceph/mds_client.h | 1 + fs/ceph/super.h | 7 +++- 6 files changed, 160 insertions(+), 65 deletions(-)
fs/ceph/dir.c +38 -32
@@ -852,13 +852,6 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - err =ceph_pre_init_acls (dir, &mode, &as_ctx); -if (err <0 ) - goto out; - err =ceph_security_init_secctx (dentry, mode, &as_ctx); -if (err <0 ) - goto out; -dout ("mknod in dir %p dentry %p mode 0%ho rdev %d\n" , dir, dentry, mode, rdev); req =ceph_mdsc_create_request (mdsc,CEPH_MDS_OP_MKNOD ,USE_AUTH_MDS );
@@ -866,6 +859,14 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, err =PTR_ERR (req); goto out; } + + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_dentry =dget (dentry); req->r_num_caps =2 ; req->r_parent = dir;
@@ -875,13 +876,13 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, req->r_args.mknod.rdev =cpu_to_le32 (rdev); req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (!err)
@@ -904,6 +905,7 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir,struct ceph_mds_client *mdsc =ceph_sb_to_mdsc (dir->i_sb);struct ceph_mds_request *req;struct ceph_acl_sec_ctx as_ctx = {}; +umode_t mode =S_IFLNK |0777 ;int err;if (ceph_snap (dir) !=CEPH_NOSNAP )
@@ -914,21 +916,24 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - err =ceph_security_init_secctx (dentry,S_IFLNK |0777 , &as_ctx); -if (err <0 ) - goto out; -dout ("symlink in dir %p dentry %p to '%s'\n" , dir, dentry, dest); req =ceph_mdsc_create_request (mdsc,CEPH_MDS_OP_SYMLINK ,USE_AUTH_MDS );if (IS_ERR (req)) { err =PTR_ERR (req); goto out; } + + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_path2 =kstrdup (dest,GFP_KERNEL );if (!req->r_path2) { err = -ENOMEM ; -ceph_mdsc_put_request (req); - goto out; + goto out_req; } req->r_parent = dir;ihold (dir);
@@ -938,13 +943,13 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir, req->r_num_caps =2 ; req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (err)
@@ -980,13 +985,6 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - mode |=S_IFDIR ; - err =ceph_pre_init_acls (dir, &mode, &as_ctx); -if (err <0 ) - goto out; - err =ceph_security_init_secctx (dentry, mode, &as_ctx); -if (err <0 ) - goto out; req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS );if (IS_ERR (req)) {
@@ -994,6 +992,14 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, goto out; } + mode |=S_IFDIR ; + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_dentry =dget (dentry); req->r_num_caps =2 ; req->r_parent = dir;
@@ -1002,15 +1008,15 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, req->r_args.mkdir.mode =cpu_to_le32 (mode); req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_target && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (!err)
fs/ceph/file.c +38 -24
134 lines not on this page · raw message
fs/ceph/inode.c +75 -7
113 lines not on this page · raw message
fs/ceph/mds_client.c +2 -1
@@ -851,6 +851,7 @@ void ceph_mdsc_release_request(struct kref *kref)iput (req->r_parent); }iput (req->r_target_inode); +iput (req->r_new_inode);if (req->r_dentry)dput (req->r_dentry);if (req->r_old_dentry)
@@ -3315,7 +3316,7 @@ static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg) .snap = le64_to_cpu(rinfo->targeti.in->snapid) }; - in =ceph_get_inode (mdsc->fsc->sb, tvino); + in =ceph_get_inode (mdsc->fsc->sb, tvino,xchg (&req->r_new_inode,NULL ));if (IS_ERR (in)) { err =PTR_ERR (in);mutex_lock (&session->s_mutex);
fs/ceph/mds_client.h +1 -0
@@ -259,6 +259,7 @@ struct ceph_mds_request {struct inode *r_parent;/* parent dir inode */ struct inode *r_target_inode;/* resulting inode */ +struct inode *r_new_inode;/* new inode (for creates) */ #define CEPH_MDS_R_DIRECT_IS_HASH (1)/* r_direct_hash is valid */ #define CEPH_MDS_R_ABORTED (2)/* call was aborted */
fs/ceph/super.h +6 -1
@@ -964,6 +964,7 @@ static inline bool __ceph_have_pending_cap_snap(struct ceph_inode_info *ci)/* inode.c */ struct ceph_mds_reply_info_in ;struct ceph_mds_reply_dirfrag ; +struct ceph_acl_sec_ctx ;extern const struct inode_operations ceph_file_iops;
@@ -971,8 +972,12 @@ extern struct inode *ceph_alloc_inode(struct super_block *sb);extern void ceph_evict_inode (struct inode *inode);extern void ceph_free_inode (struct inode *inode); +struct inode *ceph_new_inode (struct inode *dir,struct dentry *dentry, +umode_t *mode,struct ceph_acl_sec_ctx *as_ctx); +void ceph_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as_ctx); +extern struct inode *ceph_get_inode (struct super_block *sb, -struct ceph_vino vino); +struct ceph_vino vino,struct inode *newino);extern struct inode *ceph_get_snapdir (struct inode *parent);extern int ceph_fill_file_size (struct inode *inode,int issued,u32 truncate_seq,u64 truncate_size,u64 size);
Signature · 1 line
2.31.1
RFC PATCH v8 6/24 ceph: parse new fscrypt_auth and fscrypt_file fields in inode traces
In reply to Jeff Layton...and store them in the ceph_inode_info.
--- fs/ceph/file.c | 2 ++ fs/ceph/inode.c | 18 +++++++++++++++ fs/ceph/mds_client.c | 55 ++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/mds_client.h | 4 ++++ fs/ceph/super.h | 5 ++++ 5 files changed, 84 insertions(+)
fs/ceph/file.c +2 -0
@@ -592,6 +592,8 @@ static int ceph_finish_async_create(struct inode *dir, struct inode *inode, iinfo.xattr_data = xattr_buf;memset (iinfo.xattr_data,0 , iinfo.xattr_len); +/* FIXME: set fscrypt_auth and fscrypt_file */ + in.ino =cpu_to_le64 (vino.ino); in.snapid =cpu_to_le64 (CEPH_NOSNAP ); in.version =cpu_to_le64 (1 );// ???
fs/ceph/inode.c +18 -0
@@ -612,6 +612,11 @@ struct inode *ceph_alloc_inode(struct super_block *sb) ci->i_meta_err =0 ; +#ifdef CONFIG_FS_ENCRYPTION + ci->fscrypt_auth =NULL ; + ci->fscrypt_auth_len =0 ; +#endif +return &ci->vfs_inode; }
@@ -620,6 +625,9 @@ void ceph_free_inode(struct inode *inode)struct ceph_inode_info *ci =ceph_inode (inode);kfree (ci->i_symlink); +#ifdef CONFIG_FS_ENCRYPTION +kfree (ci->fscrypt_auth); +#endif kmem_cache_free (ceph_inode_cachep, ci); }
@@ -1015,6 +1023,16 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page, xattr_blob =NULL ; } +#ifdef CONFIG_FS_ENCRYPTION +if (iinfo->fscrypt_auth_len && !ci->fscrypt_auth) { + ci->fscrypt_auth_len = iinfo->fscrypt_auth_len; + ci->fscrypt_auth = iinfo->fscrypt_auth; + iinfo->fscrypt_auth =NULL ; + iinfo->fscrypt_auth_len =0 ; +inode_set_flags (inode,S_ENCRYPTED ,S_ENCRYPTED ); + } +#endif +/* finally update i_version */ if (le64_to_cpu (info->version) > ci->i_version) ci->i_version =le64_to_cpu (info->version);
fs/ceph/mds_client.c +55 -0
71 lines not on this page · raw message
fs/ceph/mds_client.h +4 -0
@@ -88,6 +88,10 @@ struct ceph_mds_reply_info_in {s32 dir_pin;struct ceph_timespec btime;struct ceph_timespec snap_btime; +u8 *fscrypt_auth; +u8 *fscrypt_file; +u32 fscrypt_auth_len; +u32 fscrypt_file_len;u64 rsnaps;u64 change_attr; };
fs/ceph/super.h +5 -0
@@ -435,6 +435,11 @@ struct ceph_inode_info {#ifdef CONFIG_CEPH_FSCACHE struct fscache_cookie *fscache;#endif +u32 fscrypt_auth_len; +u32 fscrypt_file_len; +u8 *fscrypt_auth; +u8 *fscrypt_file; +errseq_t i_meta_err;struct inode vfs_inode;/* at end */
Signature · 1 line
2.31.1
RFC PATCH v8 7/24 ceph: add fscrypt_* handling to caps.c
In reply to Jeff Layton--- fs/ceph/caps.c | 62 +++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 49 insertions(+), 13 deletions(-)
fs/ceph/caps.c +49 -13
@@ -13,6 +13,7 @@#include "super.h" #include "mds_client.h" #include "cache.h" +#include "crypto.h" #include <linux/ceph/decode.h> #include <linux/ceph/messenger.h>
@@ -1214,15 +1215,12 @@ struct cap_msg_args {umode_t mode;bool inline_data;bool wake; +u32 fscrypt_auth_len; +u32 fscrypt_file_len; +u8 fscrypt_auth[sizeof (struct ceph_fscrypt_auth )];// for context +u8 fscrypt_file[sizeof (u64)];// for size }; -/* -* cap struct size + flock buffer size + inline version + inline data size + -* osd_epoch_barrier + oldest_flush_tid -*/ -#define CAP_MSG_SIZE (sizeof(struct ceph_mds_caps) + \ - 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4) -/* Marshal up the cap msg to the MDS */ static void encode_cap_msg (struct ceph_msg *msg,struct cap_msg_args *arg) {
@@ -1238,7 +1236,7 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg) arg->size, arg->max_size, arg->xattr_version, arg->xattr_buf ? (int)arg->xattr_buf->vec.iov_len : 0); - msg->hdr.version =cpu_to_le16 (10 ); + msg->hdr.version =cpu_to_le16 (12 ); msg->hdr.tid =cpu_to_le64 (arg->flush_tid); fc = msg->front.iov_base;
@@ -1309,6 +1307,16 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg)/* Advisory flags (version 10) */ ceph_encode_32 (&p, arg->flags); + +/* dirstats (version 11) - these are r/o on the client */ +ceph_encode_64 (&p,0 ); +ceph_encode_64 (&p,0 ); + +/* fscrypt_auth and fscrypt_file (version 12) */ +ceph_encode_32 (&p, arg->fscrypt_auth_len); +ceph_encode_copy (&p, arg->fscrypt_auth, arg->fscrypt_auth_len); +ceph_encode_32 (&p, arg->fscrypt_file_len); +ceph_encode_copy (&p, arg->fscrypt_file, arg->fscrypt_file_len); }/*
@@ -1430,6 +1438,26 @@ static void __prep_cap(struct cap_msg_args *arg, struct ceph_cap *cap, } } arg->flags = flags; +if (ci->fscrypt_auth_len && +WARN_ON_ONCE (ci->fscrypt_auth_len !=sizeof (struct ceph_fscrypt_auth ))) { +/* Don't set this if it isn't right size */ + arg->fscrypt_auth_len =0 ; + }else { + arg->fscrypt_auth_len = ci->fscrypt_auth_len; +memcpy (arg->fscrypt_auth, ci->fscrypt_auth, +min_t (size_t, ci->fscrypt_auth_len,sizeof (arg->fscrypt_auth))); + } +/* FIXME: use this to track "real" size */ + arg->fscrypt_file_len =0 ; +} + +#define CAP_MSG_FIXED_FIELDS (sizeof(struct ceph_mds_caps) + \ + 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4 + 8 + 8 + 4 + 4) + +static inline int cap_msg_size (struct cap_msg_args *arg) +{ +return CAP_MSG_FIXED_FIELDS + arg->fscrypt_auth_len + + arg->fscrypt_file_len; }/*
@@ -1442,7 +1470,7 @@ static void __send_cap(struct cap_msg_args *arg, struct ceph_inode_info *ci)struct ceph_msg *msg;struct inode *inode = &ci->vfs_inode; - msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,CAP_MSG_SIZE ,GFP_NOFS , false); + msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,cap_msg_size (arg),GFP_NOFS , false);if (!msg) { pr_err("error allocating cap msg: ino (%llx.%llx) flushing %s tid %llu, requeuing cap.\n", ceph_vinop(inode), ceph_cap_string(arg->dirty),
@@ -1468,10 +1496,6 @@ static inline int __send_flush_snap(struct inode *inode,struct cap_msg_args arg;struct ceph_msg *msg; - msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,CAP_MSG_SIZE ,GFP_NOFS , false); -if (!msg) -return -ENOMEM ; - arg.session = session; arg.ino =ceph_vino (inode).ino; arg.cid =0 ;
@@ -1509,6 +1533,18 @@ static inline int __send_flush_snap(struct inode *inode, arg.flags =0 ; arg.wake = false; +/* +* No fscrypt_auth changes from a capsnap. It will need +* to update fscrypt_file on size changes (TODO). +*/ + arg.fscrypt_auth_len =0 ; + arg.fscrypt_file_len =0 ; + + msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,cap_msg_size (&arg), +GFP_NOFS , false); +if (!msg) +return -ENOMEM ; +encode_cap_msg (msg, &arg);ceph_con_send (&arg.session->s_con, msg);return 0 ;
Signature · 1 line
2.31.1
RFC PATCH v8 8/24 ceph: crypto context handling for ceph
In reply to Jeff LaytonHave set_context do a setattr that sets the fscrypt_auth value, and get_context just return the contents of that field.
--- fs/ceph/Makefile | 1 + fs/ceph/crypto.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 29 ++++++++++++++++++ fs/ceph/inode.c | 3 ++ fs/ceph/super.c | 3 ++ 5 files changed, 113 insertions(+) create mode 100644 fs/ceph/crypto.c create mode 100644 fs/ceph/crypto.h
fs/ceph/Makefile +1 -0
@@ -12,3 +12,4 @@ ceph-y := super.o inode.o dir.o file.o locks.o addr.o ioctl.o \ ceph-$(CONFIG_CEPH_FSCACHE) += cache.o ceph-$(CONFIG_CEPH_FS_POSIX_ACL) += acl.o +ceph-$(CONFIG_FS_ENCRYPTION) += crypto.o
fs/ceph/crypto.c new file +77 -0
77 lines not on this page · raw message
fs/ceph/crypto.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Ceph fscrypt functionality +*/ + +#ifndef _CEPH_CRYPTO_H +#define _CEPH_CRYPTO_H + +#include <linux/fscrypt.h> + +struct ceph_fscrypt_auth { +__le32 cfa_version; +__le32 cfa_blob_len; +u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; +} __packed; + +#ifdef CONFIG_FS_ENCRYPTION +#define CEPH_FSCRYPT_AUTH_VERSION 1 +void ceph_fscrypt_set_ops (struct super_block *sb); + +#else /* CONFIG_FS_ENCRYPTION */ + +static inline void ceph_fscrypt_set_ops (struct super_block *sb) +{ +} + +#endif /* CONFIG_FS_ENCRYPTION */ + +#endif
fs/ceph/inode.c +3 -0
@@ -14,10 +14,12 @@#include <linux/random.h> #include <linux/sort.h> #include <linux/iversion.h> +#include <linux/fscrypt.h> #include "super.h" #include "mds_client.h" #include "cache.h" +#include "crypto.h" #include <linux/ceph/decode.h> /*
@@ -646,6 +648,7 @@ void ceph_evict_inode(struct inode *inode)clear_inode (inode);ceph_fscache_unregister_inode_cookie (ci); +fscrypt_put_encryption_info (inode);__ceph_remove_caps (ci);
fs/ceph/super.c +3 -0
@@ -20,6 +20,7 @@#include "super.h" #include "mds_client.h" #include "cache.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/decode.h>
@@ -1109,6 +1110,8 @@ static int ceph_set_super(struct super_block *s, struct fs_context *fc) s->s_time_min =0 ; s->s_time_max =U32_MAX ; +ceph_fscrypt_set_ops (s); + ret =set_anon_super_fc (s, fc);if (ret !=0 ) fsc->sb =NULL ;
Signature · 1 line
2.31.1
RFC PATCH v8 9/24 ceph: add ability to set fscrypt_auth via setattr
In reply to Jeff Layton--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)
fs/ceph/acl.c +2 -2
@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }
@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }
fs/ceph/crypto.h +8 -1
@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */
fs/ceph/inode.c +31 -2
@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;
@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued)); +if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ; +spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); + +if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) { + req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),
@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);
@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);
fs/ceph/mds_client.c +45 -9
110 lines not on this page · raw message
fs/ceph/mds_client.h +3 -0
@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;
fs/ceph/super.h +6 -1
@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,
include/linux/ceph/ceph_fs.h +13 -8
@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Signature · 1 line
2.31.1
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 76 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +6
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued)); +if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +
Maybe this should return -EEXIST if already set ?
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 6 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {
For this, shouldn't we always set the req->r_fscrypt_auth even the "CEPH_CAP_AUTH_SHARED" cap is issued ?
Maybe this should be:
} else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {
??
Quoted 214 lines
fs/ceph/inode.c +9
-1+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts;int i;@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p,for (i =0 ; i < req->r_cred->group_info->ngroups; i++)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + +/* v5: altname (TODO: skip for now) */ +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false);if (!msg) { msg =ERR_PTR (-ENOMEM );@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 81 lines · Xiubo Li
Quoted 80 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +10
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued)); +if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ; +spin_unlock (&ci->i_ceph_lock); + goto out; + } +
Quoted 9 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 6 lines
fs/ceph/inode.c +6
-0+dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).
Quoted 11 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 7 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp ci->fscrypt_auth and cia->fscrypt_auth?
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 215 lines
Quoted 214 lines
fs/ceph/inode.c +9
-1+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 90 lines · Jeff Layton
Quoted 72 lines · Xiubo Li
Quoted 71 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +1
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued));Quoted 17 lines
Quoted 5 lines
fs/ceph/inode.c +5
-0+if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?
Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).
I may miss some code in the fs/crypto/ layer.
I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return -EEXIST, or will try to call ceph layer to set it.
So once this is set, my understanding is that it shouldn't be here ?
Quoted 9 lines
Quoted 9 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 6 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?
Quoted 7 lines
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp
ci->fscrypt_authandcia->fscrypt_auth?
My understanding is that, if the 'As' cap is not issued, that means maybe another client has been issued the 'Ax' cap. For the current client, if !ci->fscrypt_auth == true and
no matter whether the 'As' cap is issued or not it should try to set the fscrypt_auth to MDS. But this could fail if another client also trying to set the fscrypt_auth by holding the 'Ax' cap ?
Or won't the new one override the old context in MDS side ?
Quoted 218 lines
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 214 lines
Quoted 214 lines
fs/ceph/inode.c +9
-1+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 99 lines · Xiubo Li
Quoted 90 lines · Jeff Layton
Quoted 72 lines · Xiubo Li
Quoted 71 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +1
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued));Quoted 17 lines
Quoted 5 lines
fs/ceph/inode.c +5
-0+if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).
Quoted 8 lines
I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
Quoted 22 lines
Quoted 10 lines
Quoted 10 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 7 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?Quoted 7 lines
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp
ci->fscrypt_authandcia->fscrypt_auth?My understanding is that, if the 'As' cap is not issued, that means maybe another client has been issued the 'Ax' cap. For the current client, if !ci->fscrypt_auth == true and
Quoted 6 lines
no matter whether the 'As' cap is issued or not it should try to set the
fscrypt_authto MDS. But this could fail if another client also trying to set thefscrypt_authby holding the 'Ax' cap ?Or won't the new one override the old context in MDS side ?
The update to the code that I'm testing now looks like this.
if (issued & CEPH_CAP_AUTH_EXCL) {
dirtied |= CEPH_CAP_AUTH_EXCL;
kfree(ci->fscrypt_auth);
ci->fscrypt_auth = (u8 *)cia->fscrypt_auth;
ci->fscrypt_auth_len = len;
} else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
ci->fscrypt_auth_len != len ||
memcmp(ci->fscrypt_auth, cia->fscrypt_auth, len)) {
req->r_fscrypt_auth = cia->fscrypt_auth;
mask |= CEPH_SETATTR_FSCRYPT_AUTH;
release |= CEPH_CAP_AUTH_SHARED;
}
cia->fscrypt_auth = NULL;Basically, the logic is:
If we have Ax caps, we can do the change locally and eventually flush it to the server in a cap update. If we have As caps then we can skip issuing a SETATTR if nothing actually changed. If we don't have either, then we just have to issue the SETATTR since we can't tell.
Quoted 221 lines
Quoted 220 lines
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 214 lines
Quoted 214 lines
fs/ceph/inode.c +9
-1+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Signature · 1 line
Jeff Layton <[email protected]>
Eric Biggers
maintainerRe: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 19 lines · Jeff Layton
Quoted 19 lines
Quoted 5 lines
Quoted 5 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 3 lines
fs/ceph/inode.c +3
-0+/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.
Quoted 6 lines
Quoted 2 lines
So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
That is how FS_IOC_SET_ENCRYPTION_POLICY behaves, but the check for an existing policy already happens in fscrypt_ioctl_set_policy(), so ->set_context doesn't need to worry about it.
- Eric
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 19 lines · Eric Biggers
Quoted 18 lines · Jeff Layton
Quoted 18 lines
Quoted 5 lines
Quoted 5 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 3 lines
fs/ceph/inode.c +3
-0+/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
That is how
FS_IOC_SET_ENCRYPTION_POLICYbehaves, but the check for an existing policy already happens infscrypt_ioctl_set_policy(), so ->set_context doesn't need to worry about it.
Yeah, the FS_IOC_SET_ENCRYPTION_POLICY section in "Documentation/filesystems/fscrypt.rst".
Quoted 2 lines
- Eric
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
In reply to Jeff LaytonQuoted 138 lines · Jeff Layton
Quoted 98 lines · Xiubo Li
Quoted 90 lines · Jeff Layton
Quoted 72 lines · Xiubo Li
Quoted 71 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +1
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued));Quoted 17 lines
Quoted 5 lines
fs/ceph/inode.c +5
-0+if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).Quoted 8 lines
I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 6 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?Quoted 7 lines
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp
ci->fscrypt_authandcia->fscrypt_auth?My understanding is that, if the 'As' cap is not issued, that means maybe another client has been issued the 'Ax' cap. For the current client, if !ci->fscrypt_auth == true and
Quoted 6 lines
no matter whether the 'As' cap is issued or not it should try to set the
fscrypt_authto MDS. But this could fail if another client also trying to set thefscrypt_authby holding the 'Ax' cap ?Or won't the new one override the old context in MDS side ?
The update to the code that I'm testing now looks like this.
Quoted 8 lines
if (issued & CEPH_CAP_AUTH_EXCL) {
dirtied |= CEPH_CAP_AUTH_EXCL;
kfree(ci->fscrypt_auth);
ci->fscrypt_auth = (u8 *)cia->fscrypt_auth;
ci->fscrypt_auth_len = len;
} else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
ci->fscrypt_auth_len != len ||
memcmp(ci->fscrypt_auth, cia->fscrypt_auth, len)) {Checked the fs/scrypto code again, the memcmp won't work here IMO.
Such as in fscrypt_ioctl_set_policy() it already helpes comparing that without the extra random nonce bytes in the context. In ceph layer the context will contain some extra random nonce bytes following the policy struct, more detail please see set_encryption_policy(), I am sure that the fscrypt_auth will be a different one every time in ceph layer even the contents in encrypt policy struct are the same. So I am afraid the memcmp() in ceph will always return none zero.
Quoted 10 lines
req->r_fscrypt_auth = cia->fscrypt_auth; mask |= CEPH_SETATTR_FSCRYPT_AUTH; release |= CEPH_CAP_AUTH_SHARED; } cia->fscrypt_auth = NULL;Basically, the logic is:
If we have Ax caps, we can do the change locally and eventually flush it to the server in a cap update.
Yeah, then my understand of this is correct :-)
Quoted 4 lines
If we have As caps then we can skip
issuing a SETATTR if nothing actually changed. If we don't have either, then we just have to issue the SETATTR since we can't tell.
So if the As cap is not issued to us and at the same time locally the ci->fscrypt_auth is NULL, since we can't tell then will TRY to issue the SETATTR, while maybe another client has already holding the Ax cap and also is trying to set the fscrypt_auth.
Then when the MDS receives our setattr client request, it will try to revoke the Ax cap, then after revoking is done the fscrypt_auth will be flushed from that client and set it in MDS. So IMO our try should fail ? Or ours will override the existence one.
Quoted 218 lines
Quoted 218 lines
Quoted 218 lines
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 214 lines
Quoted 214 lines
fs/ceph/inode.c +9
-1+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +2312,7 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr) req->r_stamp = attr->ia_ctime; err =ceph_mdsc_do_request (mdsc,NULL , req); } +out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 150 lines · Xiubo Li
Quoted 138 lines · Jeff Layton
Quoted 98 lines · Xiubo Li
Quoted 90 lines · Jeff Layton
Quoted 72 lines · Xiubo Li
Quoted 71 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +1
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued));Quoted 17 lines
Quoted 5 lines
fs/ceph/inode.c +5
-0+if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).Quoted 8 lines
I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 6 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?Quoted 7 lines
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp
ci->fscrypt_authandcia->fscrypt_auth?My understanding is that, if the 'As' cap is not issued, that means maybe another client has been issued the 'Ax' cap. For the current client, if !ci->fscrypt_auth == true and
Quoted 6 lines
no matter whether the 'As' cap is issued or not it should try to set the
fscrypt_authto MDS. But this could fail if another client also trying to set thefscrypt_authby holding the 'Ax' cap ?Or won't the new one override the old context in MDS side ?
The update to the code that I'm testing now looks like this.
Quoted 8 lines
if (issued & CEPH_CAP_AUTH_EXCL) { dirtied |= CEPH_CAP_AUTH_EXCL; kfree(ci->fscrypt_auth); ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; ci->fscrypt_auth_len = len; } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 || ci->fscrypt_auth_len != len || memcmp(ci->fscrypt_auth, cia->fscrypt_auth, len)) {Checked the fs/scrypto code again, the memcmp won't work here IMO.
Quoted 9 lines
Such as in
fscrypt_ioctl_set_policy()it already helpes comparing that without the extra random nonce bytes in the context. In ceph layer the context will contain some extra random nonce bytes following the policy struct, more detail please seeset_encryption_policy(), I am sure that thefscrypt_authwill be a different one every time in ceph layer even the contents in encrypt policy struct are the same. So I am afraid thememcmp()in ceph will always return none zero.
Initially, the only way to set this is via the fscrypt ioctls, but eventually we may want to add the ability to set this via another means such as the the vxattr. For instance, if you need to restore from backups, it's not clear to me how you would set the right context back onto the inode so it could be decrypted/
In any case, the memcmp does serve a purpose here. If someone tries to set the exact same context that we had before, it'll skip the SETATTR. fscrypt_auth is just like any other inode metadata field.
Quoted 25 lines
Quoted 8 lines
req->r_fscrypt_auth = cia->fscrypt_auth; mask |= CEPH_SETATTR_FSCRYPT_AUTH; release |= CEPH_CAP_AUTH_SHARED; } cia->fscrypt_auth = NULL;Quoted 4 lines
Basically, the logic is:
If we have Ax caps, we can do the change locally and eventually flush it to the server in a cap update.
Yeah, then my understand of this is correct :-)
Quoted 4 lines
If we have As caps then we can skip
issuing a SETATTR if nothing actually changed. If we don't have either, then we just have to issue the SETATTR since we can't tell.So if the As cap is not issued to us and at the same time locally the
ci->fscrypt_authis NULL, since we can't tell then will TRY to issue the SETATTR, while maybe another client has already holding the Ax cap and also is trying to set thefscrypt_auth.
Quoted 6 lines
Then when the MDS receives our setattr client request, it will try to revoke the Ax cap, then after revoking is done the
fscrypt_authwill be flushed from that client and set it in MDS. So IMO our try should fail ? Or ours will override the existence one.
Yep. It's up to the MDS to mediate that interaction.
This is one of the reasons that I think we probably can't treat these values as entirely opaque. We need some rules about conflicting attempts to set a new context on an inode and I think those may need to be enforced by the MDS.
Quoted 220 lines
Quoted 219 lines
Quoted 219 lines
Quoted 219 lines
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 214 lines
Quoted 214 lines
fs/ceph/inode.c +7
-0+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +fscrypt_ioctl_set_policy2312,7 @@ int
__ceph_setattr(struct inode *inode, struct iattr *attr)req->r_stamp = attr->ia_ctime; err = ceph_mdsc_do_request(mdsc, NULL, req); }fs/ceph/inode.c +2
-1+out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [RFC PATCH v8 09/24] ceph: add ability to set fscrypt_auth via setattr
Quoted 165 lines · Jeff Layton
Quoted 149 lines · Xiubo Li
Quoted 138 lines · Jeff Layton
Quoted 98 lines · Xiubo Li
Quoted 90 lines · Jeff Layton
Quoted 72 lines · Xiubo Li
Quoted 71 lines · Jeff Layton
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/acl.c | 4 +-- fs/ceph/crypto.h | 9 +++++- fs/ceph/inode.c | 33 ++++++++++++++++++++-- fs/ceph/mds_client.c | 54 ++++++++++++++++++++++++++++++------ fs/ceph/mds_client.h | 3 ++ fs/ceph/super.h | 7 ++++- include/linux/ceph/ceph_fs.h | 21 ++++++++------ 7 files changed, 108 insertions(+), 23 deletions(-)fs/ceph/acl.c +2
-2@@ -136,7 +136,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }@@ -147,7 +147,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }fs/ceph/crypto.h +8
-1@@ -14,8 +14,15 @@ struct ceph_fscrypt_auth {u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; } __packed; -#ifdef CONFIG_FS_ENCRYPTION #define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb);#else /* CONFIG_FS_ENCRYPTION */ fs/ceph/inode.c +1
-1@@ -2083,7 +2083,7 @@ static const struct inode_operations ceph_symlink_iops = { .listxattr = ceph_listxattr, }; -int __ceph_setattr (struct inode *inode,struct iattr *attr) +int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia) {struct ceph_inode_info *ci =ceph_inode (inode);unsigned int ia_valid = attr->ia_valid;@@ -2124,6 +2124,34 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr)dout ("setattr %p issued %s\n" , inode,ceph_cap_string (issued));Quoted 17 lines
Quoted 5 lines
fs/ceph/inode.c +5
-0+if (cia && cia->fscrypt_auth) { +u32 len =ceph_fscrypt_auth_len (cia->fscrypt_auth); + +if (len >sizeof (*cia->fscrypt_auth)) { + err = -EINVAL ;fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2141
Quoted 10 lines
fs/ceph/inode.c +10
-0+spin_unlock (&ci->i_ceph_lock); + goto out; + } + +dout ("setattr %llx:%llx fscrypt_auth len %u to %u)\n" , +ceph_vinop (inode), ci->fscrypt_auth_len, len); + +/* It should never be re-set once set */ +WARN_ON_ONCE (ci->fscrypt_auth); +Maybe this should return
-EEXISTif already set ?Quoted 7 lines
I don't know. In general, once the context is set on an inode, we shouldn't ever reset it. That said, I think we might need to allow admins to override an existing context if it's corrupted.
So, that's the rationale for the
WARN_ON_ONCE. The admins should never do this under normal circumstances but they do have the ability to change it if needed (and we'll see a warning in the logs in that case).Quoted 8 lines
I may miss some code in the
fs/crypto/layer.I readed that once the directory/file has set the policy context, it will just return 0 if the new one matches the existence, if not match it will return
-EEXIST, or will try to call ceph layer to set it.So once this is set, my understanding is that it shouldn't be here ?
Where did you read that? If we have documented semantics we need to follow here, then we should change it to comply with them.
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
fs/ceph/inode.c int __ceph_setattr(struct inode *inode, struct iattr *attr) · line 2147
Quoted 6 lines
fs/ceph/inode.c +6
-0+if (issued &CEPH_CAP_AUTH_EXCL ) { + dirtied |=CEPH_CAP_AUTH_EXCL ; +kfree (ci->fscrypt_auth); + ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; + ci->fscrypt_auth_len = len; + }else if ((issued &CEPH_CAP_AUTH_SHARED ) ==0 ) {For this, shouldn't we always set the
req->r_fscrypt_autheven the "CEPH_CAP_AUTH_SHARED" cap is issued ?Quoted 7 lines
Quoted 6 lines
Maybe this should be:
} else if ((issued &
CEPH_CAP_AUTH_SHARED) == 0 || !ci->fscrypt_auth) {??
...or maybe we need to memcmp
ci->fscrypt_authandcia->fscrypt_auth?My understanding is that, if the 'As' cap is not issued, that means maybe another client has been issued the 'Ax' cap. For the current client, if !ci->fscrypt_auth == true and
Quoted 6 lines
no matter whether the 'As' cap is issued or not it should try to set the
fscrypt_authto MDS. But this could fail if another client also trying to set thefscrypt_authby holding the 'Ax' cap ?Or won't the new one override the old context in MDS side ?
The update to the code that I'm testing now looks like this.
Quoted 8 lines
if (issued & CEPH_CAP_AUTH_EXCL) { dirtied |= CEPH_CAP_AUTH_EXCL; kfree(ci->fscrypt_auth); ci->fscrypt_auth = (u8 *)cia->fscrypt_auth; ci->fscrypt_auth_len = len; } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 || ci->fscrypt_auth_len != len || memcmp(ci->fscrypt_auth, cia->fscrypt_auth, len)) {Checked the fs/scrypto code again, the memcmp won't work here IMO.
Quoted 9 lines
Such as in
fscrypt_ioctl_set_policy()it already helpes comparing that without the extra random nonce bytes in the context. In ceph layer the context will contain some extra random nonce bytes following the policy struct, more detail please seeset_encryption_policy(), I am sure that thefscrypt_authwill be a different one every time in ceph layer even the contents in encrypt policy struct are the same. So I am afraid thememcmp()in ceph will always return none zero.Initially, the only way to set this is via the fscrypt ioctls, but eventually we may want to add the ability to set this via another means such as the the vxattr. For instance, if you need to restore from backups, it's not clear to me how you would set the right context back onto the inode so it could be decrypted/
Quoted 3 lines
In any case, the memcmp does serve a purpose here. If someone tries to set the exact same context that we had before, it'll skip the SETATTR.
fscrypt_authis just like any other inode metadata field.
Okay.
Once it's here and the ci->fscyrpt_auth exists that means it's restoring the backups, right ?
Quoted 29 lines
Quoted 23 lines
Quoted 11 lines
req->r_fscrypt_auth = cia->fscrypt_auth; mask |= CEPH_SETATTR_FSCRYPT_AUTH; release |= CEPH_CAP_AUTH_SHARED; } cia->fscrypt_auth = NULL;Basically, the logic is:
If we have Ax caps, we can do the change locally and eventually flush it to the server in a cap update.
Yeah, then my understand of this is correct :-)
Quoted 4 lines
If we have As caps then we can skip
issuing a SETATTR if nothing actually changed. If we don't have either, then we just have to issue the SETATTR since we can't tell.So if the As cap is not issued to us and at the same time locally the
ci->fscrypt_authis NULL, since we can't tell then will TRY to issue the SETATTR, while maybe another client has already holding the Ax cap and also is trying to set thefscrypt_auth.Quoted 6 lines
Then when the MDS receives our setattr client request, it will try to revoke the Ax cap, then after revoking is done the
fscrypt_authwill be flushed from that client and set it in MDS. So IMO our try should fail ? Or ours will override the existence one.
Quoted 6 lines
Yep. It's up to the MDS to mediate that interaction.
This is one of the reasons that I think we probably can't treat these values as entirely opaque. We need some rules about conflicting attempts to set a new context on an inode and I think those may need to be enforced by the MDS.
If there has two client conflicts, the MDS could return -EEXIST and at the same time returns that context back to kclient and let the kclient decide what to do next. The kclient know how to compare them by stripping the nonce extra bytes. Then we can keep this field opaque ?
Quoted 217 lines
Quoted 217 lines
Quoted 217 lines
Quoted 217 lines
Quoted 217 lines
In any case, you're right that testing for As caps alone is not sufficient. I'll fix that up soon.
Quoted 214 lines
Quoted 214 lines
fs/ceph/inode.c +7
-0+ req->r_fscrypt_auth = cia->fscrypt_auth; + mask |=CEPH_SETATTR_FSCRYPT_AUTH ; + release |=CEPH_CAP_AUTH_SHARED ; + } + cia->fscrypt_auth =NULL ; + } +if (ia_valid &ATTR_UID ) { dout("setattr %p uid %d -> %d\n", inode, from_kuid(&init_user_ns, inode->i_uid),@@ -2284,6 +fscrypt_ioctl_set_policy2312,7 @@ int
__ceph_setattr(struct inode *inode, struct iattr *attr)req->r_stamp = attr->ia_ctime; err = ceph_mdsc_do_request(mdsc, NULL, req); }fs/ceph/inode.c +2
-1+out:dout ("setattr %p result=%d (%s locally, %d remote)\n" , inode, err,ceph_cap_string (dirtied), mask);@@ -2321,7 +2350,7 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry, ceph_quota_is_max_bytes_exceeded(inode, attr->ia_size))return -EDQUOT ; - err =__ceph_setattr (inode, attr); + err =__ceph_setattr (inode, attr,NULL );if (err >=0 && (attr->ia_valid &ATTR_MODE )) err =posix_acl_chmod (&init_user_ns, inode, attr->ia_mode);fs/ceph/mds_client.c +45
-9@@ -15,6 +15,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/messenger.h> @@ -927,6 +928,7 @@ void ceph_mdsc_release_request(struct kref *kref)put_cred (req->r_cred);if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist); +kfree (req->r_fscrypt_auth);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));@@ -2618,8 +2620,7 @@ static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,return r; } -static void encode_timestamp_and_gids (void **p, -const struct ceph_mds_request *req) +static void encode_mclientrequest_tail (void **p,const struct ceph_mds_request *req) {struct ceph_timespec ts; int i;16:51 < batrick> done@@ -2632,6 +2633,20 @@ static void encode_timestamp_and_gids(void **p, for (i = 0; i < req->r_cred->group_info->ngroups; i++)load more comments (105 replies)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); + + /* v5: altname (TODO: skip for now) */load more comments (105 replies) +ceph_encode_32 (p,0 ); + +/* v6: fscrypt_auth and fscrypt_file */ +if (req->r_fscrypt_auth) { +u32 authlen =ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +ceph_encode_32 (p, authlen); +ceph_encode_copy (p, req->r_fscrypt_auth, authlen); + }else { +ceph_encode_32 (p,0 ); + } +ceph_encode_32 (p,0 );// fscrypt_file for now }/* @@ -2676,12 +2691,14 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } +/* head */ len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head ); - len += pathlen1 + pathlen2 +2 *(1 +sizeof (u32) +sizeof (u64)) + -sizeof (struct ceph_timespec ); - len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); -/* calculate (max) length for cap releases */ +/* filepaths */ + len +=2 * (1 +sizeof (u32) +sizeof (u64)); + len += pathlen1 + pathlen2; + +/* cap releases */ len +=sizeof (struct ceph_mds_request_release ) * (!!req->r_inode_drop + !!req->r_dentry_drop + !!req->r_old_inode_drop + !!req->r_old_dentry_drop);@@ -2691,6 +2708,25 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,if (req->r_old_dentry_drop) len += pathlen2; +/* MClientRequest tail */ + +/* req->r_stamp */ + len +=sizeof (struct ceph_timespec ); + +/* gid list */ + len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups); + +/* alternate name */ + len +=sizeof (u32);// TODO + +/* fscrypt_auth */ + len +=sizeof (u32);// fscrypt_auth +if (req->r_fscrypt_auth) + len +=ceph_fscrypt_auth_len (req->r_fscrypt_auth); + +/* fscrypt_file */ + len +=sizeof (u32); + msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false); if (!msg) {load more comments (105 replies) msg = ERR_PTR(-ENOMEM);@@ -2710,7 +2746,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, }else {struct ceph_mds_request_head *new_head = msg->front.iov_base; - msg->hdr.version =cpu_to_le16 (4 ); + msg->hdr.version =cpu_to_le16 (6 ); new_head->version =cpu_to_le16 (CEPH_MDS_REQUEST_HEAD_VERSION ); head = (struct ceph_mds_request_head_old *)&new_head->oldest_client_tid; p = msg->front.iov_base +sizeof (*new_head);@@ -2761,7 +2797,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, head->num_releases =cpu_to_le16 (releases); -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req);if (WARN_ONCE (p > end,"p=%p end=%p len=%d\n" , p, end, len)) {ceph_msg_put (msg);@@ -2870,7 +2906,7 @@ static int __prepare_send_request(struct ceph_mds_session *session, rhead->num_releases =0 ; p = msg->front.iov_base + req->r_request_release_offset; -encode_timestamp_and_gids (&p, req); +encode_mclientrequest_tail (&p, req); msg->front.iov_len = p - msg->front.iov_base; msg->hdr.front_len =cpu_to_le32 (msg->front.iov_len);fs/ceph/mds_client.h +3
-0@@ -278,6 +278,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;fs/ceph/super.h +6
-1@@ -1040,7 +1040,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,include/linux/ceph/ceph_fs.h +13
-8@@ -356,14 +356,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
RFC PATCH v8 10/24 ceph: implement -o test_dummy_encryption mount option
In reply to Jeff Layton--- fs/ceph/crypto.c | 53 ++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 26 ++++++++++++++++ fs/ceph/inode.c | 10 ++++-- fs/ceph/super.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++-- fs/ceph/super.h | 12 +++++++- fs/ceph/xattr.c | 3 ++ 6 files changed, 177 insertions(+), 6 deletions(-)
fs/ceph/crypto.c +53 -0
@@ -4,6 +4,7 @@#include <linux/fscrypt.h> #include "super.h" +#include "mds_client.h" #include "crypto.h" static int ceph_crypt_get_context (struct inode *inode,void *ctx,size_t len)
@@ -64,9 +65,20 @@ static bool ceph_crypt_empty_dir(struct inode *inode)return ci->i_rsubdirs + ci->i_rfiles ==1 ; } +void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc) +{ +fscrypt_free_dummy_policy (&fsc->dummy_enc_policy); +} + +static const union fscrypt_policy *ceph_get_dummy_policy (struct super_block *sb) +{ +return ceph_sb_to_client (sb)->dummy_enc_policy.policy; +} +static struct fscrypt_operations ceph_fscrypt_ops = { .get_context = ceph_crypt_get_context, .set_context = ceph_crypt_set_context, + .get_dummy_policy = ceph_get_dummy_policy, .empty_dir = ceph_crypt_empty_dir, .max_namelen =NAME_MAX , };
@@ -75,3 +87,44 @@ void ceph_fscrypt_set_ops(struct super_block *sb) {fscrypt_set_ops (sb, &ceph_fscrypt_ops); } + +int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as) +{ +int ret, ctxsize; +bool encrypted = false; +struct ceph_inode_info *ci =ceph_inode (inode); + + ret =fscrypt_prepare_new_inode (dir, inode, &encrypted); +if (ret) +return ret; +if (!encrypted) +return 0 ; + + as->fscrypt_auth =kzalloc (sizeof (*as->fscrypt_auth),GFP_KERNEL ); +if (!as->fscrypt_auth) +return -ENOMEM ; + + ctxsize =fscrypt_context_for_new_inode (as->fscrypt_auth->cfa_blob, inode); +if (ctxsize <0 ) +return ctxsize; + + as->fscrypt_auth->cfa_version =cpu_to_le32 (CEPH_FSCRYPT_AUTH_VERSION ); + as->fscrypt_auth->cfa_blob_len =cpu_to_le32 (ctxsize); + +WARN_ON_ONCE (ci->fscrypt_auth); +kfree (ci->fscrypt_auth); + ci->fscrypt_auth_len =ceph_fscrypt_auth_len (as->fscrypt_auth); + ci->fscrypt_auth =kmemdup (as->fscrypt_auth, ci->fscrypt_auth_len,GFP_KERNEL ); +if (!ci->fscrypt_auth) +return -ENOMEM ; + + inode->i_flags |=S_ENCRYPTED ; + +return 0 ; +} + +void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as) +{ +swap (req->r_fscrypt_auth, as->fscrypt_auth); +}
fs/ceph/crypto.h +26 -0
@@ -8,6 +8,10 @@#include <linux/fscrypt.h> +struct ceph_fs_client ; +struct ceph_acl_sec_ctx ; +struct ceph_mds_request ; +struct ceph_fscrypt_auth {__le32 cfa_version;__le32 cfa_blob_len;
@@ -25,12 +29,34 @@ static inline u32 ceph_fscrypt_auth_len(struct ceph_fscrypt_auth *fa)#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb); +void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc); + +int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as); +void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); +#else /* CONFIG_FS_ENCRYPTION */ static inline void ceph_fscrypt_set_ops (struct super_block *sb) { } +static inline void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc) +{ +} + +static inline int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as) +{ +if (IS_ENCRYPTED (dir)) +return -EOPNOTSUPP ; +return 0 ; +} + +static inline void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req, +struct ceph_acl_sec_ctx *as_ctx) +{ +}#endif /* CONFIG_FS_ENCRYPTION */ #endif
fs/ceph/inode.c +8 -2
@@ -83,12 +83,17 @@ struct inode *ceph_new_inode(struct inode *dir, struct dentry *dentry, goto out_err; } + inode->i_state =0 ; + inode->i_mode = *mode; + err =ceph_security_init_secctx (dentry, *mode, as_ctx);if (err <0 ) goto out_err; - inode->i_state =0 ; - inode->i_mode = *mode; + err =ceph_fscrypt_prepare_context (dir, inode, as_ctx); +if (err) + goto out_err; +return inode; out_err:iput (inode);
@@ -101,6 +106,7 @@ void ceph_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_sec_ctx *a req->r_pagelist = as_ctx->pagelist; as_ctx->pagelist =NULL ; } +ceph_fscrypt_as_ctx_to_req (req, as_ctx); }/**
fs/ceph/super.c +76 -3
147 lines not on this page · raw message
fs/ceph/super.h +11 -1
@@ -17,6 +17,7 @@#include <linux/posix_acl.h> #include <linux/refcount.h> #include <linux/security.h> +#include <linux/fscrypt.h> #include <linux/ceph/libceph.h>
@@ -25,6 +26,8 @@#include <linux/fscache.h> #endif +#include "crypto.h" +/* f_type in struct statfs */ #define CEPH_SUPER_MAGIC 0x00c36400
@@ -45,6 +48,7 @@#define CEPH_MOUNT_OPT_NOQUOTADF (1<<13)/* no root dir quota in statfs */ #define CEPH_MOUNT_OPT_NOCOPYFROM (1<<14)/* don't use RADOS 'copy-from' op */ #define CEPH_MOUNT_OPT_ASYNC_DIROPS (1<<15)/* allow async directory ops */ +#define CEPH_MOUNT_OPT_TEST_DUMMY_ENC (1<<16)/* enable dummy encryption (for testing) */ #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \
@@ -101,6 +105,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +char *test_dummy_encryption;/* default NULL */ };struct ceph_fs_client {
@@ -140,9 +145,11 @@ struct ceph_fs_client {#ifdef CONFIG_CEPH_FSCACHE struct fscache_cookie *fscache;#endif +#ifdef CONFIG_FS_ENCRYPTION +struct fscrypt_dummy_policy dummy_enc_policy; +#endif }; -/* * File i/o capability. This tracks shared state with the metadata * server that allows us to cache or writeback attributes or to read
@@ -1068,6 +1075,9 @@ struct ceph_acl_sec_ctx {#ifdef CONFIG_CEPH_FS_SECURITY_LABEL void *sec_ctx;u32 sec_ctxlen; +#endif +#ifdef CONFIG_FS_ENCRYPTION +struct ceph_fscrypt_auth *fscrypt_auth;#endif struct ceph_pagelist *pagelist; };
fs/ceph/xattr.c +3 -0
@@ -1381,6 +1381,9 @@ void ceph_release_acl_sec_ctx(struct ceph_acl_sec_ctx *as_ctx)#endif #ifdef CONFIG_CEPH_FS_SECURITY_LABEL security_release_secctx (as_ctx->sec_ctx, as_ctx->sec_ctxlen); +#endif +#ifdef CONFIG_FS_ENCRYPTION +kfree (as_ctx->fscrypt_auth);#endif if (as_ctx->pagelist)ceph_pagelist_release (as_ctx->pagelist);
Signature · 1 line
2.31.1
RFC PATCH v8 11/24 ceph: add fscrypt ioctls
In reply to Jeff LaytonWe gate most of the ioctls on MDS feature support. The exception is the key removal and status functions that we still want to work if the MDS's were to (inexplicably) lose the feature.
For the set_policy ioctl, we take Fcx caps to ensure that nothing can create files in the directory while the ioctl is running. That should be enough to ensure that the "empty_dir" check is reliable.
--- fs/ceph/ioctl.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
fs/ceph/ioctl.c +83 -0
@@ -6,6 +6,7 @@#include "mds_client.h" #include "ioctl.h" #include <linux/ceph/striper.h> +#include <linux/fscrypt.h> /* * ioctls
@@ -268,8 +269,54 @@ static long ceph_ioctl_syncio(struct file *file)return 0 ; } +static int vet_mds_for_fscrypt (struct file *file) +{ +int i, ret = -EOPNOTSUPP ; +struct ceph_mds_client *mdsc =ceph_sb_to_mdsc (file_inode (file)->i_sb); + +mutex_lock (&mdsc->mutex); +for (i =0 ; i < mdsc->max_sessions; i++) { +struct ceph_mds_session *s = mdsc->sessions[i]; + +if (!s) +continue ; +if (test_bit (CEPHFS_FEATURE_ALTERNATE_NAME , &s->s_features)) + ret =0 ; +break ; + } +mutex_unlock (&mdsc->mutex); +return ret; +} + +static long ceph_set_encryption_policy (struct file *file,unsigned long arg) +{ +int ret, got =0 ; +struct inode *inode =file_inode (file); +struct ceph_inode_info *ci =ceph_inode (inode); + + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; + +/* +* Ensure we hold these caps so that we _know_ that the rstats check +* in the empty_dir check is reliable. +*/ + ret =ceph_get_caps (file,CEPH_CAP_FILE_SHARED ,0 ,-1 , &got); +if (ret) +return ret; + + ret =fscrypt_ioctl_set_policy (file, (const void __user *)arg); +if (got) +ceph_put_cap_refs (ci, got); + +return ret; +} +long ceph_ioctl (struct file *file,unsigned int cmd,unsigned long arg) { +int ret; +dout ("ioctl file %p cmd %u arg %lu\n" , file, cmd, arg);switch (cmd) {case CEPH_IOC_GET_LAYOUT :
@@ -289,6 +336,42 @@ long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg)case CEPH_IOC_SYNCIO :return ceph_ioctl_syncio (file); + +case FS_IOC_SET_ENCRYPTION_POLICY : +return ceph_set_encryption_policy (file, arg); + +case FS_IOC_GET_ENCRYPTION_POLICY : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_policy (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_POLICY_EX : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_policy_ex (file, (void __user *)arg); + +case FS_IOC_ADD_ENCRYPTION_KEY : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_add_key (file, (void __user *)arg); + +case FS_IOC_REMOVE_ENCRYPTION_KEY : +return fscrypt_ioctl_remove_key (file, (void __user *)arg); + +case FS_IOC_REMOVE_ENCRYPTION_KEY_ALL_USERS : +return fscrypt_ioctl_remove_key_all_users (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_KEY_STATUS : +return fscrypt_ioctl_get_key_status (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_NONCE : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_nonce (file, (void __user *)arg); }return -ENOTTY ;
Signature · 1 line
2.31.1
RFC PATCH v8 12/24 ceph: decode alternate_name in lease info
In reply to Jeff LaytonCeph is a bit different from local filesystems, in that we don't want to store filenames as raw binary data, since we may also be dealing with clients that don't support fscrypt.
We could just base64-encode the encrypted filenames, but that could leave us with filenames longer than NAME_MAX. It turns out that the MDS doesn't care much about filename length, but the clients do.
To manage this, we've added a new "alternate name" field that can be optionally added to any dentry that we'll use to store the binary crypttext of the filename if its base64-encoded value will be longer than NAME_MAX. When a dentry has one of these names attached, the MDS will send it along in the lease info, which we can then store for later usage.
--- fs/ceph/mds_client.c | 40 ++++++++++++++++++++++++++++++---------- fs/ceph/mds_client.h | 11 +++++++---- 2 files changed, 37 insertions(+), 14 deletions(-)
fs/ceph/mds_client.c +30 -10
@@ -306,27 +306,44 @@ static int parse_reply_info_dir(void **p, void *end,static int parse_reply_info_lease (void **p,void *end,struct ceph_mds_reply_lease **lease, -u64 features) +u64 features,u32 *altname_len,u8 **altname) { +u8 struct_v; +u32 struct_len; +if (features == (u64)-1 ) { -u8 struct_v, struct_compat; -u32 struct_len; +u8 struct_compat; +ceph_decode_8_safe (p, end, struct_v, bad);ceph_decode_8_safe (p, end, struct_compat, bad); +/* struct_v is expected to be >= 1. we only understand * encoding whose struct_compat == 1. */ if (!struct_v || struct_compat !=1 ) goto bad; +ceph_decode_32_safe (p, end, struct_len, bad); -ceph_decode_need (p, end, struct_len, bad); - end = *p + struct_len; + }else { + struct_len =sizeof (**lease); + *altname_len =0 ; + *altname =NULL ; } -ceph_decode_need (p, end,sizeof (**lease), bad); +ceph_decode_need (p, end, struct_len, bad); *lease = *p; *p +=sizeof (**lease); -if (features == (u64)-1 ) - *p = end; + +if (features == (u64)-1 ) { +if (struct_v >=2 ) { +ceph_decode_32_safe (p, end, *altname_len, bad); +ceph_decode_need (p, end, *altname_len, bad); + *altname = *p; + *p += *altname_len; + }else { + *altname =NULL ; + *altname_len =0 ; + } + }return 0 ; bad:return -EIO ;
@@ -356,7 +373,8 @@ static int parse_reply_info_trace(void **p, void *end, info->dname = *p; *p += info->dname_len; - err =parse_reply_info_lease (p, end, &info->dlease, features); + err =parse_reply_info_lease (p, end, &info->dlease, features, + &info->altname_len, &info->altname);if (err <0 ) goto out_bad; }
@@ -423,9 +441,11 @@ static int parse_reply_info_readdir(void **p, void *end,dout ("parsed dir dname '%.*s'\n" , rde->name_len, rde->name);/* dentry lease */ - err =parse_reply_info_lease (p, end, &rde->lease, features); + err =parse_reply_info_lease (p, end, &rde->lease, features, + &rde->altname_len, &rde->altname);if (err) goto out_bad; +/* inode */ err =parse_reply_info_in (p, end, &rde->inode, features);if (err <0 )
fs/ceph/mds_client.h +7 -4
@@ -29,8 +29,8 @@ enum ceph_feature_type {CEPHFS_FEATURE_MULTI_RECONNECT ,CEPHFS_FEATURE_DELEG_INO ,CEPHFS_FEATURE_METRIC_COLLECT , - - CEPHFS_FEATURE_MAX = CEPHFS_FEATURE_METRIC_COLLECT, +CEPHFS_FEATURE_ALTERNATE_NAME , + CEPHFS_FEATURE_MAX = CEPHFS_FEATURE_ALTERNATE_NAME, };/*
@@ -45,8 +45,7 @@ enum ceph_feature_type {CEPHFS_FEATURE_MULTI_RECONNECT , \CEPHFS_FEATURE_DELEG_INO , \CEPHFS_FEATURE_METRIC_COLLECT , \ - \ -CEPHFS_FEATURE_MAX , \ +CEPHFS_FEATURE_ALTERNATE_NAME , \ } #define CEPHFS_FEATURES_CLIENT_REQUIRED {}
@@ -98,7 +97,9 @@ struct ceph_mds_reply_info_in {struct ceph_mds_reply_dir_entry {char *name; +u8 *altname;u32 name_len; +u32 altname_len;struct ceph_mds_reply_lease *lease;struct ceph_mds_reply_info_in inode;loff_t offset;
@@ -117,7 +118,9 @@ struct ceph_mds_reply_info_parsed {struct ceph_mds_reply_info_in diri, targeti;struct ceph_mds_reply_dirfrag *dirfrag;char *dname; +u8 *altname;u32 dname_len; +u32 altname_len;struct ceph_mds_reply_lease *dlease;/* extra */
Signature · 1 line
2.31.1
RFC PATCH v8 13/24 ceph: make ceph_msdc_build_path use ref-walk
In reply to Jeff LaytonEncryption potentially requires allocation, at which point we'll need to be in a non-atomic context. Convert ceph_msdc_build_path to take dentry spinlocks and references instead of using rcu_read_lock to walk the path.
This is slightly less efficient, and we may want to eventually allow using RCU when the leaf dentry isn't encrypted.
--- fs/ceph/mds_client.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)
fs/ceph/mds_client.c +19 -16
@@ -2487,7 +2487,8 @@ static inline u64 __get_oldest_tid(struct ceph_mds_client *mdsc)char *ceph_mdsc_build_path (struct dentry *dentry,int *plen,u64 *pbase,int stop_on_nosnap) { -struct dentry *temp; +struct dentry *cur; +struct inode *inode;char *path;int pos;unsigned seq;
@@ -2504,34 +2505,35 @@ char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *pbase, path[pos] ='\0' ; seq =read_seqbegin (&rename_lock); -rcu_read_lock (); - temp = dentry; + cur =dget (dentry);for (;;) { -struct inode *inode; +struct dentry *temp; -spin_lock (&temp->d_lock); - inode =d_inode (temp); +spin_lock (&cur->d_lock); + inode =d_inode (cur);if (inode &&ceph_snap (inode) ==CEPH_SNAPDIR ) {dout ("build_path path+%d: %p SNAPDIR\n" , - pos, temp); - }else if (stop_on_nosnap && inode && dentry != temp && + pos, cur); + }else if (stop_on_nosnap && inode && dentry != cur &&ceph_snap (inode) ==CEPH_NOSNAP ) { -spin_unlock (&temp->d_lock); +spin_unlock (&cur->d_lock); pos++;/* get rid of any prepended '/' */ break ; }else { - pos -= temp->d_name.len; + pos -= cur->d_name.len;if (pos <0 ) { -spin_unlock (&temp->d_lock); +spin_unlock (&cur->d_lock);break ; } -memcpy (path + pos, temp->d_name.name, temp->d_name.len); +memcpy (path + pos, cur->d_name.name, cur->d_name.len); } + temp = cur;spin_unlock (&temp->d_lock); - temp =READ_ONCE (temp->d_parent); + cur =dget_parent (temp); +dput (temp);/* Are we at the root? */ -if (IS_ROOT (temp)) +if (IS_ROOT (cur))break ;/* Are we out of buffer? */
@@ -2540,8 +2542,9 @@ char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *pbase, path[pos] ='/' ; } - base =ceph_ino (d_inode (temp)); -rcu_read_unlock (); + inode =d_inode (cur); + base = inode ?ceph_ino (inode) :0 ; +dput (cur);if (read_seqretry (&rename_lock, seq)) goto retry;
Signature · 1 line
2.31.1
RFC PATCH v8 14/24 ceph: add encrypted fname handling to ceph_mdsc_build_path
In reply to Jeff LaytonAllow ceph_mdsc_build_path to encrypt and base64 encode the filename when the parent is encrypted and we're sending the path to the MDS.
In most cases, we just encrypt the filenames and base64 encode them, but when the name is longer than CEPH_NOHASH_NAME_MAX, we use a similar scheme to fscrypt proper, and hash the remaning bits with sha256.
When doing this, we then send along the full crypttext of the name in the new alternate_name field of the MClientRequest. The MDS can then send that along in readdir responses and traces.
--- fs/ceph/crypto.c | 48 ++++++++++++++++++++++++++ fs/ceph/crypto.h | 26 ++++++++++++++ fs/ceph/mds_client.c | 80 ++++++++++++++++++++++++++++++++++---------- 3 files changed, 136 insertions(+), 18 deletions(-)
fs/ceph/crypto.c +48 -0
@@ -128,3 +128,51 @@ void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_se {swap (req->r_fscrypt_auth, as->fscrypt_auth); } + +int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) +{ +u32 len; +int elen; +int ret; +u8 *cryptbuf; + +WARN_ON_ONCE (!fscrypt_has_encryption_key (parent)); + +/* +* convert cleartext dentry name to ciphertext +* if result is longer than CEPH_NOKEY_NAME_MAX, +* sha256 the remaining bytes +* +* See: fscrypt_setup_filename +*/ +if (!fscrypt_fname_encrypted_size (parent, dentry->d_name.len,NAME_MAX , &len)) +return -ENAMETOOLONG ; + +/* Allocate a buffer appropriate to hold the result */ + cryptbuf =kmalloc (len >CEPH_NOHASH_NAME_MAX ?NAME_MAX : len,GFP_KERNEL ); +if (!cryptbuf) +return -ENOMEM ; + + ret =fscrypt_fname_encrypt (parent, &dentry->d_name, cryptbuf, len); +if (ret) { +kfree (cryptbuf); +return ret; + } + +/* hash the end if the name is long enough */ +if (len >CEPH_NOHASH_NAME_MAX ) { +u8 hash[SHA256_DIGEST_SIZE ]; +u8 *extra = cryptbuf +CEPH_NOHASH_NAME_MAX ; + +/* hash the extra bytes and overwrite crypttext beyond that point with it */ +sha256 (extra, len -CEPH_NOHASH_NAME_MAX , hash); +memcpy (extra, hash,SHA256_DIGEST_SIZE ); + len =CEPH_NOHASH_NAME_MAX +SHA256_DIGEST_SIZE ; + } + +/* base64 encode the encrypted name */ + elen =fscrypt_base64url_encode (cryptbuf, len, buf); +kfree (cryptbuf); +dout ("base64-encoded ciphertext name = %.*s\n" , elen, buf); +return elen; +}
fs/ceph/crypto.h +26 -0
@@ -6,6 +6,7 @@#ifndef _CEPH_CRYPTO_H#define _CEPH_CRYPTO_H +#include <crypto/sha2.h> #include <linux/fscrypt.h> struct ceph_fs_client ;
@@ -27,6 +28,24 @@ static inline u32 ceph_fscrypt_auth_len(struct ceph_fscrypt_auth *fa) }#ifdef CONFIG_FS_ENCRYPTION +/* +* We want to encrypt filenames when creating them, but the encrypted +* versions of those names may have illegal characters in them. To mitigate +* that, we base64 encode them, but that gives us a result that can exceed +* NAME_MAX. +* +* Follow a similar scheme to fscrypt itself, and cap the filename to a +* smaller size. If the ciphertext name is longer than the value below, then +* sha256 hash the remaining bytes. +* +* 189 bytes => 252 bytes base64-encoded, which is <= NAME_MAX (255) +* +* Note that for long names that end up having their tail portion hashed, we +* must also store the full encrypted name (in the dentry's alternate_name +* field). +*/ +#define CEPH_NOHASH_NAME_MAX (189 - SHA256_DIGEST_SIZE) +void ceph_fscrypt_set_ops (struct super_block *sb);void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc);
@@ -34,6 +53,7 @@ void ceph_fscrypt_free_dummy_policy(struct ceph_fs_client *fsc);int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode,struct ceph_acl_sec_ctx *as);void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); +int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf);#else /* CONFIG_FS_ENCRYPTION */
@@ -57,6 +77,12 @@ static inline void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_sec_ctx *as_ctx) { } + +static inline int ceph_encode_encrypted_fname (const struct inode *parent, +struct dentry *dentry,char *buf) +{ +return -EOPNOTSUPP ; +}#endif /* CONFIG_FS_ENCRYPTION */ #endif
fs/ceph/mds_client.c +62 -18
131 lines not on this page · raw message
Signature · 1 line
2.31.1
RFC PATCH v8 15/24 ceph: send altname in MClientRequest
In reply to Jeff LaytonIn the event that we have a filename longer than CEPH_NOHASH_NAME_MAX, we'll need to hash the tail of the filename. The client however will still need to know the full name of the file if it has a key.
To support this, the MClientRequest field has grown a new alternate_name field that we populate with the full (binary) crypttext of the filename. This is then transmitted to the clients in readdir or traces as part of the dentry lease.
Add support for populating this field when the filenames are very long.
--- fs/ceph/mds_client.c | 75 +++++++++++++++++++++++++++++++++++++++++--- fs/ceph/mds_client.h | 3 ++ 2 files changed, 73 insertions(+), 5 deletions(-)
fs/ceph/mds_client.c +70 -5
@@ -950,6 +950,7 @@ void ceph_mdsc_release_request(struct kref *kref)if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist);kfree (req->r_fscrypt_auth); +kfree (req->r_altname);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));
@@ -2475,6 +2476,63 @@ static inline u64 __get_oldest_tid(struct ceph_mds_client *mdsc)return mdsc->oldest_tid; } +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +static u8 *get_fscrypt_altname (const struct ceph_mds_request *req,u32 *plen) +{ +struct inode *dir = req->r_parent; +struct dentry *dentry = req->r_dentry; +u8 *cryptbuf =NULL ; +u32 len =0 ; +int ret =0 ; + +/* only encode if we have parent and dentry */ +if (!dir || !dentry) + goto success; + +/* No-op unless this is encrypted */ +if (!IS_ENCRYPTED (dir)) + goto success; + + ret =__fscrypt_prepare_readdir (dir); +if (ret) +return ERR_PTR (ret); + +/* No key? Just ignore it. */ +if (!fscrypt_has_encryption_key (dir)) + goto success; + +if (!fscrypt_fname_encrypted_size (dir, dentry->d_name.len,NAME_MAX , &len)) { +WARN_ON_ONCE (1 ); +return ERR_PTR (-ENAMETOOLONG ); + } + +/* No need to append altname if name is short enough */ +if (len <=CEPH_NOHASH_NAME_MAX ) { + len =0 ; + goto success; + } + + cryptbuf =kmalloc (len,GFP_KERNEL ); +if (!cryptbuf) +return ERR_PTR (-ENOMEM ); + + ret =fscrypt_fname_encrypt (dir, &dentry->d_name, cryptbuf, len); +if (ret) { +kfree (cryptbuf); +return ERR_PTR (ret); + } +success: + *plen = len; +return cryptbuf; +} +#else +static u8 *get_fscrypt_altname (const struct ceph_mds_request *req,u32 *plen) +{ + *plen =0 ; +return NULL ; +} +#endif +/** * ceph_mdsc_build_path - build a path string to a given dentry * @dentry: dentry to which path should be built
@@ -2695,14 +2753,15 @@ static void encode_mclientrequest_tail(void **p, const struct ceph_mds_request *ceph_encode_timespec64 (&ts, &req->r_stamp);ceph_encode_copy (p, &ts,sizeof (ts)); -/* gid_list */ +/* v4: gid_list */ ceph_encode_32 (p, req->r_cred->group_info->ngroups);for (i =0 ; i < req->r_cred->group_info->ngroups; i++)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); -/* v5: altname (TODO: skip for now) */ -ceph_encode_32 (p,0 ); +/* v5: altname */ +ceph_encode_32 (p, req->r_altname_len); +ceph_encode_copy (p, req->r_altname, req->r_altname_len);/* v6: fscrypt_auth and fscrypt_file */ if (req->r_fscrypt_auth) {
@@ -2758,7 +2817,13 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } -/* head */ + req->r_altname =get_fscrypt_altname (req, &req->r_altname_len); +if (IS_ERR (req->r_altname)) { + msg =ERR_CAST (req->r_altname); + req->r_altname =NULL ; + goto out_free2; + } + len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head );/* filepaths */
@@ -2784,7 +2849,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups);/* alternate name */ - len +=sizeof (u32);// TODO + len +=sizeof (u32) + req->r_altname_len;/* fscrypt_auth */ len +=sizeof (u32);// fscrypt_auth
fs/ceph/mds_client.h +3 -0
@@ -284,6 +284,9 @@ struct ceph_mds_request {struct ceph_fscrypt_auth *r_fscrypt_auth; +u8 *r_altname;/* fscrypt binary crypttext for long filenames */ +u32 r_altname_len;/* length of r_altname */ +int r_fmode;/* file mode, if expecting cap */ const struct cred *r_cred;int r_request_release_offset;
Signature · 1 line
2.31.1
RFC PATCH v8 16/24 ceph: encode encrypted name in dentry release
In reply to Jeff Layton--- fs/ceph/caps.c | 31 +++++++++++++++++++++++++++---- fs/ceph/mds_client.c | 20 ++++++++++++++++---- 2 files changed, 43 insertions(+), 8 deletions(-)
fs/ceph/caps.c +27 -4
@@ -4570,6 +4570,18 @@ int ceph_encode_inode_release(void **p, struct inode *inode,return ret; } +/** +* ceph_encode_dentry_release - encode a dentry release into an outgoing request +* @p: outgoing request buffer +* @dentry: dentry to release +* @dir: dir to release it from +* @mds: mds that we're speaking to +* @drop: caps being dropped +* @unless: unless we have these caps +* +* Encode a dentry release into an outgoing request buffer. Returns 1 if the +* thing was released, or a negative error code otherwise. +*/ int ceph_encode_dentry_release (void **p,struct dentry *dentry,struct inode *dir,int mds,int drop,int unless)
@@ -4602,12 +4614,23 @@ int ceph_encode_dentry_release(void **p, struct dentry *dentry,if (ret && di->lease_session && di->lease_session->s_mds == mds) {dout ("encode_dentry_release %p mds%d seq %d\n" , dentry, mds, (int )di->lease_seq); - rel->dname_len =cpu_to_le32 (dentry->d_name.len); -memcpy (*p, dentry->d_name.name, dentry->d_name.len); - *p += dentry->d_name.len; rel->dname_seq =cpu_to_le32 (di->lease_seq);__ceph_mdsc_drop_dentry_lease (dentry); +spin_unlock (&dentry->d_lock); +if (IS_ENCRYPTED (dir) &&fscrypt_has_encryption_key (dir)) { +int ret2 =ceph_encode_encrypted_fname (dir, dentry, *p); +if (ret2 <0 ) +return ret2; + + rel->dname_len =cpu_to_le32 (ret2); + *p += ret2; + }else { + rel->dname_len =cpu_to_le32 (dentry->d_name.len); +memcpy (*p, dentry->d_name.name, dentry->d_name.len); + *p += dentry->d_name.len; + } + }else { +spin_unlock (&dentry->d_lock); } -spin_unlock (&dentry->d_lock);return ret; }
fs/ceph/mds_client.c +16 -4
@@ -2908,15 +2908,23 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, req->r_inode ? req->r_inode :d_inode (req->r_dentry), mds, req->r_inode_drop, req->r_inode_unless, req->r_op == CEPH_MDS_OP_READDIR); -if (req->r_dentry_drop) - releases +=ceph_encode_dentry_release (&p, req->r_dentry, +if (req->r_dentry_drop) { + ret =ceph_encode_dentry_release (&p, req->r_dentry, req->r_parent, mds, req->r_dentry_drop, req->r_dentry_unless); -if (req->r_old_dentry_drop) - releases +=ceph_encode_dentry_release (&p, req->r_old_dentry, +if (ret <0 ) + goto out_err; + releases += ret; + } +if (req->r_old_dentry_drop) { + ret =ceph_encode_dentry_release (&p, req->r_old_dentry, req->r_old_dentry_dir, mds, req->r_old_dentry_drop, req->r_old_dentry_unless); +if (ret <0 ) + goto out_err; + releases += ret; + }if (req->r_old_inode_drop) releases += ceph_encode_inode_release(&p, d_inode(req->r_old_dentry),
@@ -2958,6 +2966,10 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,ceph_mdsc_free_path ((char *)path1, pathlen1); out:return msg; +out_err: +ceph_msg_put (msg); + msg =ERR_PTR (ret); + goto out_free2; }/*
Signature · 1 line
2.31.1
RFC PATCH v8 17/24 ceph: properly set DCACHE_NOKEY_NAME flag in lookup
In reply to Jeff LaytonThis is required so that we know to invalidate these dentries when the directory is unlocked.
--- fs/ceph/dir.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
fs/ceph/dir.c +11 -0
@@ -751,6 +751,17 @@ static struct dentry *ceph_lookup(struct inode *dir, struct dentry *dentry,if (dentry->d_name.len >NAME_MAX )return ERR_PTR (-ENAMETOOLONG ); +if (IS_ENCRYPTED (dir)) { + err =__fscrypt_prepare_readdir (dir); +if (err) +return ERR_PTR (err); +if (!fscrypt_has_encryption_key (dir)) { +spin_lock (&dentry->d_lock); + dentry->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dentry->d_lock); + } + } +/* can we conclude ENOENT locally? */ if (d_really_is_negative (dentry)) {struct ceph_inode_info *ci =ceph_inode (dir);
Signature · 1 line
2.31.1
RFC PATCH v8 18/24 ceph: make d_revalidate call fscrypt revalidator for encrypted dentries
In reply to Jeff LaytonIf we have a dentry which represents a no-key name, then we need to test whether the parent directory's encryption key has since been added. Do that before we test anything else about the dentry.
--- fs/ceph/dir.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
fs/ceph/dir.c +6 -2
@@ -1700,6 +1700,10 @@ static int ceph_d_revalidate(struct dentry *dentry, unsigned int flags)struct inode *dir, *inode;struct ceph_mds_client *mdsc; + valid =fscrypt_d_revalidate (dentry, flags); +if (valid <=0 ) +return valid; +if (flags &LOOKUP_RCU ) { parent =READ_ONCE (dentry->d_parent); dir =d_inode_rcu (parent);
@@ -1712,8 +1716,8 @@ static int ceph_d_revalidate(struct dentry *dentry, unsigned int flags) inode =d_inode (dentry); } -dout ("d_revalidate %p '%pd' inode %p offset 0x%llx\n" , dentry, - dentry, inode,ceph_dentry (dentry)->offset); +dout ("d_revalidate %p '%pd' inode %p offset 0x%llx nokey %d\n" , dentry, + dentry, inode,ceph_dentry (dentry)->offset, !!(dentry->d_flags &DCACHE_NOKEY_NAME )); mdsc =ceph_sb_to_client (dir->i_sb)->mdsc;
Signature · 1 line
2.31.1
RFC PATCH v8 19/24 ceph: add helpers for converting names for userland presentation
In reply to Jeff Layton--- fs/ceph/crypto.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 41 ++++++++++++++++++++++++++ 2 files changed, 117 insertions(+)
fs/ceph/crypto.c +76 -0
@@ -176,3 +176,79 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentrdout ("base64-encoded ciphertext name = %.*s\n" , elen, buf);return elen; } + +/** +* ceph_fname_to_usr - convert a filename for userland presentation +* @fname: ceph_fname to be converted +* @tname: temporary name buffer to use for conversion (may be NULL) +* @oname: where converted name should be placed +* @is_nokey: set to true if key wasn't available during conversion (may be NULL) +* +* Given a filename (usually from the MDS), format it for presentation to +* userland. If @parent is not encrypted, just pass it back as-is. +* +* Otherwise, base64 decode the string, and then ask fscrypt to format it +* for userland presentation. +* +* Returns 0 on success or negative error code on error. +*/ +int ceph_fname_to_usr (const struct ceph_fname *fname,struct fscrypt_str *tname, +struct fscrypt_str *oname,bool *is_nokey) +{ +int ret; +struct fscrypt_str _tname =FSTR_INIT (NULL ,0 ); +struct fscrypt_str iname; + +if (!IS_ENCRYPTED (fname->dir)) { + oname->name = fname->name; + oname->len = fname->name_len; +return 0 ; + } + +/* Sanity check that the resulting name will fit in the buffer */ +if (fname->name_len >FSCRYPT_BASE64URL_CHARS (NAME_MAX )) +return -EIO ; + + ret =__fscrypt_prepare_readdir (fname->dir); +if (ret) +return ret; + +/* +* Use the raw dentry name as sent by the MDS instead of +* generating a nokey name via fscrypt. +*/ +if (!fscrypt_has_encryption_key (fname->dir)) { +memcpy (oname->name, fname->name, fname->name_len); + oname->len = fname->name_len; +if (is_nokey) + *is_nokey = true; +return 0 ; + } + +if (fname->ctext_len ==0 ) { +int declen; + +if (!tname) { + ret =fscrypt_fname_alloc_buffer (NAME_MAX , &_tname); +if (ret) +return ret; + tname = &_tname; + } + + declen =fscrypt_base64url_decode (fname->name, fname->name_len, tname->name); +if (declen <=0 ) { + ret = -EIO ; + goto out; + } + iname.name = tname->name; + iname.len = declen; + }else { + iname.name = fname->ctext; + iname.len = fname->ctext_len; + } + + ret =fscrypt_fname_disk_to_usr (fname->dir,0 ,0 , &iname, oname); +out: +fscrypt_fname_free_buffer (&_tname); +return ret; +}
fs/ceph/crypto.h +41 -0
59 lines not on this page · raw message
Signature · 1 line
2.31.1
RFC PATCH v8 20/24 ceph: add fscrypt support to ceph_fill_trace
In reply to Jeff LaytonWhen we get a dentry in a trace, decrypt the name so we can properly instantiate the dentry.
--- fs/ceph/inode.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-)
fs/ceph/inode.c +28 -2
@@ -1395,8 +1395,15 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req)if (dir && req->r_op ==CEPH_MDS_OP_LOOKUPNAME &&test_bit (CEPH_MDS_R_PARENT_LOCKED , &req->r_req_flags) && !test_bit (CEPH_MDS_R_ABORTED , &req->r_req_flags)) { +bool is_nokey = false;struct qstr dname;struct dentry *dn, *parent; +struct fscrypt_str oname =FSTR_INIT (NULL ,0 ); +struct ceph_fname fname = { .dir = dir, + .name = rinfo->dname, + .ctext = rinfo->altname, + .name_len = rinfo->dname_len, + .ctext_len = rinfo->altname_len };BUG_ON (!rinfo->head->is_target);BUG_ON (req->r_dentry);
@@ -1404,8 +1411,20 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req) parent =d_find_any_alias (dir);BUG_ON (!parent); - dname.name = rinfo->dname; - dname.len = rinfo->dname_len; + err =ceph_fname_alloc_buffer (dir, &oname); +if (err <0 ) { +dput (parent); + goto done; + } + + err =ceph_fname_to_usr (&fname,NULL , &oname, &is_nokey); +if (err <0 ) { +dput (parent); +ceph_fname_free_buffer (dir, &oname); + goto done; + } + dname.name = oname.name; + dname.len = oname.len; dname.hash =full_name_hash (parent, dname.name, dname.len); tvino.ino =le64_to_cpu (rinfo->targeti.in->ino); tvino.snap =le64_to_cpu (rinfo->targeti.in->snapid);
@@ -1420,9 +1439,15 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req) dname.len, dname.name, dn);if (!dn) {dput (parent); +ceph_fname_free_buffer (dir, &oname); err = -ENOMEM ; goto done; } +if (is_nokey) { +spin_lock (&dn->d_lock); + dn->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dn->d_lock); + } err =0 ; } else if (d_really_is_positive(dn) && (ceph_ino(d_inode(dn)) != tvino.ino ||
@@ -1434,6 +1459,7 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req)dput (dn); goto retry_lookup; } +ceph_fname_free_buffer (dir, &oname); req->r_dentry = dn;dput (parent);
Signature · 1 line
2.31.1
RFC PATCH v8 21/24 ceph: add support to readdir for encrypted filenames
In reply to Jeff LaytonAdd helper functions for buffer management and for decrypting filenames returned by the MDS. Wire those into the readdir codepaths.
--- fs/ceph/dir.c | 62 +++++++++++++++++++++++++++++++++++++++---------- fs/ceph/inode.c | 38 +++++++++++++++++++++++++++--- 2 files changed, 85 insertions(+), 15 deletions(-)
fs/ceph/dir.c +50 -12
@@ -9,6 +9,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" /* * Directory operations: readdir, lookup, create, link, unlink,
@@ -241,7 +242,9 @@ static int __dcache_readdir(struct file *file, struct dir_context *ctx, di =ceph_dentry (dentry);if (d_unhashed (dentry) ||d_really_is_negative (dentry) || - di->lease_shared_gen != shared_gen) { + di->lease_shared_gen != shared_gen || + ((dentry->d_flags &DCACHE_NOKEY_NAME ) && +fscrypt_has_encryption_key (dir))) {spin_unlock (&dentry->d_lock);dput (dentry); err = -EAGAIN ;
@@ -313,6 +316,8 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx)int err;unsigned frag =-1 ;struct ceph_mds_reply_info_parsed *rinfo; +struct fscrypt_str tname =FSTR_INIT (NULL ,0 ); +struct fscrypt_str oname =FSTR_INIT (NULL ,0 );dout ("readdir %p file %p pos %llx\n" , inode, file, ctx->pos);if (dfi->file_info.flags &CEPH_F_ATEND )
@@ -340,6 +345,10 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) ctx->pos =2 ; } + err =fscrypt_prepare_readdir (inode); +if (err) + goto out; +spin_lock (&ci->i_ceph_lock);/* request Fx cap. if have Fx, we don't need to release Fs cap * for later create/unlink. */
@@ -360,6 +369,14 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx)spin_unlock (&ci->i_ceph_lock); } + err =ceph_fname_alloc_buffer (inode, &tname); +if (err <0 ) + goto out; + + err =ceph_fname_alloc_buffer (inode, &oname); +if (err <0 ) + goto out; +/* proceed with a normal readdir */ more:/* do we have the correct frag content buffered? */
@@ -387,12 +404,14 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx)dout ("readdir fetching %llx.%llx frag %x offset '%s'\n" ,ceph_vinop (inode), frag, dfi->last_name); req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS ); -if (IS_ERR (req)) -return PTR_ERR (req); +if (IS_ERR (req)) { + err =PTR_ERR (req); + goto out; + } err =ceph_alloc_readdir_reply_buffer (req, inode);if (err) {ceph_mdsc_put_request (req); -return err; + goto out; }/* hints to request -> mds selection code */ req->r_direct_mode =USE_AUTH_MDS ;
@@ -405,7 +424,8 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) req->r_path2 =kstrdup (dfi->last_name,GFP_KERNEL );if (!req->r_path2) {ceph_mdsc_put_request (req); -return -ENOMEM ; + err = -ENOMEM ; + goto out; } }else if (is_hash_order (ctx->pos)) { req->r_args.readdir.offset_hash =
@@ -426,7 +446,7 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) err =ceph_mdsc_do_request (mdsc,NULL , req);if (err <0 ) {ceph_mdsc_put_request (req); -return err; + goto out; } dout("readdir got and parsed readdir result=%d on " "frag %x, end=%d, complete=%d, hash_order=%d\n",
@@ -479,7 +499,7 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) err =note_last_dentry (dfi, rde->name, rde->name_len, next_offset);if (err) -return err; + goto out; }else if (req->r_reply_info.dir_end) { dfi->next_offset =2 ;/* keep last name */
@@ -507,22 +527,37 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) }for (; i < rinfo->dir_nr; i++) {struct ceph_mds_reply_dir_entry *rde = rinfo->dir_entries + i; +struct ceph_fname fname = { .dir = inode, + .name = rde->name, + .name_len = rde->name_len, + .ctext = rde->altname, + .ctext_len = rde->altname_len }; +u32 olen = oname.len;BUG_ON (rde->offset < ctx->pos); +BUG_ON (!rde->inode.in); ctx->pos = rde->offset;dout ("readdir (%d/%d) -> %llx '%.*s' %p\n" , i, rinfo->dir_nr, ctx->pos, rde->name_len, rde->name, &rde->inode.in); -BUG_ON (!rde->inode.in); + err =ceph_fname_to_usr (&fname, &tname, &oname,NULL ); +if (err) { +dout ("Unable to decode %.*s. Skipping it.\n" , rde->name_len, rde->name); +continue ; + } -if (!dir_emit (ctx, rde->name, rde->name_len, +if (!dir_emit (ctx, oname.name, oname.len,ceph_present_ino (inode->i_sb,le64_to_cpu (rde->inode.in->ino)),le32_to_cpu (rde->inode.in->mode) >>12 )) {dout ("filldir stopping us...\n" ); -return 0 ; + err =0 ; + goto out; } + +/* Reset the lengths to their original allocated vals */ + oname.len = olen; ctx->pos++; }
@@ -577,9 +612,12 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) dfi->dir_ordered_count);spin_unlock (&ci->i_ceph_lock); } - + err =0 ;dout ("readdir %p file %p done.\n" , inode, file); -return 0 ; +out: +ceph_fname_free_buffer (inode, &tname); +ceph_fname_free_buffer (inode, &oname); +return err; }static void reset_readdir (struct ceph_dir_file_info *dfi)
fs/ceph/inode.c +35 -3
@@ -1739,7 +1739,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, struct ceph_mds_session *session) {struct dentry *parent = req->r_dentry; -struct ceph_inode_info *ci =ceph_inode (d_inode (parent)); +struct inode *inode =d_inode (parent); +struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;struct qstr dname;struct dentry *dn;
@@ -1749,6 +1750,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req,u32 last_hash =0 ;u32 fpos_offset;struct ceph_readdir_cache_control cache_ctl = {}; +struct fscrypt_str tname =FSTR_INIT (NULL ,0 ); +struct fscrypt_str oname =FSTR_INIT (NULL ,0 );if (test_bit (CEPH_MDS_R_ABORTED , &req->r_req_flags))return readdir_prepopulate_inodes_only (req, session);
@@ -1800,14 +1803,36 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, cache_ctl.index = req->r_readdir_cache_idx; fpos_offset = req->r_readdir_offset; + err =ceph_fname_alloc_buffer (inode, &tname); +if (err <0 ) + goto out; + + err =ceph_fname_alloc_buffer (inode, &oname); +if (err <0 ) + goto out; +/* FIXME: release caps/leases if error occurs */ for (i =0 ; i < rinfo->dir_nr; i++) { +bool is_nokey = false;struct ceph_mds_reply_dir_entry *rde = rinfo->dir_entries + i;struct ceph_vino tvino; +u32 olen = oname.len; +struct ceph_fname fname = { .dir = inode, + .name = rde->name, + .name_len = rde->name_len, + .ctext = rde->altname, + .ctext_len = rde->altname_len }; + + err =ceph_fname_to_usr (&fname, &tname, &oname, &is_nokey); +if (err) { +dout ("Unable to decode %.*s. Skipping it." , rde->name_len, rde->name); +continue ; + } - dname.name = rde->name; - dname.len = rde->name_len; + dname.name = oname.name; + dname.len = oname.len; dname.hash =full_name_hash (parent, dname.name, dname.len); + oname.len = olen; tvino.ino =le64_to_cpu (rde->inode.in->ino); tvino.snap =le64_to_cpu (rde->inode.in->snapid);
@@ -1838,6 +1863,11 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, err = -ENOMEM ; goto out; } +if (is_nokey) { +spin_lock (&dn->d_lock); + dn->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dn->d_lock); + } }else if (d_really_is_positive (dn) && (ceph_ino (d_inode (dn)) != tvino.ino ||ceph_snap (d_inode (dn)) != tvino.snap)) {
@@ -1926,6 +1956,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, req->r_readdir_cache_idx = cache_ctl.index; }ceph_readdir_cache_release (&cache_ctl); +ceph_fname_free_buffer (inode, &tname); +ceph_fname_free_buffer (inode, &oname);dout ("readdir_prepopulate done\n" );return err; }
Signature · 1 line
2.31.1
RFC PATCH v8 22/24 ceph: create symlinks with encrypted and base64-encoded targets
In reply to Jeff LaytonWhen creating symlinks in encrypted directories, encrypt and base64-encode the target with the new inode's key before sending to the MDS.
When filling a symlinked inode, base64-decode it into a buffer that we'll keep in ci->i_symlink. When get_link is called, decrypt the buffer into a new one that will hang off i_link.
--- fs/ceph/dir.c | 51 ++++++++++++++++++++++++--- fs/ceph/inode.c | 94 ++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 128 insertions(+), 17 deletions(-)
fs/ceph/dir.c +46 -5
@@ -948,6 +948,40 @@ static int ceph_create(struct user_namespace *mnt_userns, struct inode *dir,return ceph_mknod (mnt_userns, dir, dentry, mode,0 ); } +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +static int prep_encrypted_symlink_target (struct ceph_mds_request *req,const char *dest) +{ +int err; +int len =strlen (dest); +struct fscrypt_str osd_link =FSTR_INIT (NULL ,0 ); + + err =fscrypt_prepare_symlink (req->r_parent, dest, len,PATH_MAX , &osd_link); +if (err) + goto out; + + err =fscrypt_encrypt_symlink (req->r_new_inode, dest, len, &osd_link); +if (err) + goto out; + + req->r_path2 =kmalloc (FSCRYPT_BASE64URL_CHARS (osd_link.len) +1 ,GFP_KERNEL ); +if (!req->r_path2) { + err = -ENOMEM ; + goto out; + } + + len =fscrypt_base64url_encode (osd_link.name, osd_link.len, req->r_path2); + req->r_path2[len] ='\0' ; +out: +fscrypt_fname_free_buffer (&osd_link); +return err; +} +#else +static int prep_encrypted_symlink_target (struct ceph_mds_request *req,const char *dest) +{ +return -EOPNOTSUPP ; +} +#endif +static int ceph_symlink (struct user_namespace *mnt_userns,struct inode *dir,struct dentry *dentry,const char *dest) {
@@ -979,14 +1013,21 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir, goto out_req; } - req->r_path2 =kstrdup (dest,GFP_KERNEL ); -if (!req->r_path2) { - err = -ENOMEM ; - goto out_req; - } req->r_parent = dir;ihold (dir); +if (IS_ENCRYPTED (req->r_new_inode)) { + err =prep_encrypted_symlink_target (req, dest); +if (err) + goto out_req; + }else { + req->r_path2 =kstrdup (dest,GFP_KERNEL ); +if (!req->r_path2) { + err = -ENOMEM ; + goto out_req; + } + } +set_bit (CEPH_MDS_R_PARENT_LOCKED , &req->r_req_flags); req->r_dentry =dget (dentry); req->r_num_caps =2 ;
fs/ceph/inode.c +82 -12
145 lines not on this page · raw message
Signature · 1 line
2.31.1
RFC PATCH v8 23/24 ceph: make ceph_get_name decrypt filenames
In reply to Jeff LaytonWhen we do a lookupino to the MDS, we get a filename in the trace. ceph_get_name uses that name directly, so we must properly decrypt it before copying it to the name buffer.
--- fs/ceph/export.c | 44 ++++++++++++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 12 deletions(-)
fs/ceph/export.c +32 -12
@@ -7,6 +7,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" /* * Basic fh
@@ -524,7 +525,9 @@ static int ceph_get_name(struct dentry *parent, char *name, {struct ceph_mds_client *mdsc;struct ceph_mds_request *req; +struct inode *dir =d_inode (parent);struct inode *inode =d_inode (child); +struct ceph_mds_reply_info_parsed *rinfo;int err;if (ceph_snap (inode) !=CEPH_NOSNAP )
@@ -536,30 +539,47 @@ static int ceph_get_name(struct dentry *parent, char *name,if (IS_ERR (req))return PTR_ERR (req); -inode_lock (d_inode (parent)); - +inode_lock (dir); req->r_inode = inode;ihold (inode); req->r_ino2 =ceph_vino (d_inode (parent)); - req->r_parent =d_inode (parent); -ihold (req->r_parent); + req->r_parent = dir; +ihold (dir);set_bit (CEPH_MDS_R_PARENT_LOCKED , &req->r_req_flags); req->r_num_caps =2 ; err =ceph_mdsc_do_request (mdsc,NULL , req); +inode_unlock (dir); -inode_unlock (d_inode (parent)); +if (err) + goto out; -if (!err) { -struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info; + rinfo = &req->r_reply_info; +if (!IS_ENCRYPTED (dir)) {memcpy (name, rinfo->dname, rinfo->dname_len); name[rinfo->dname_len] =0 ; -dout ("get_name %p ino %llx.%llx name %s\n" , - child,ceph_vinop (inode), name); }else { -dout ("get_name %p ino %llx.%llx err %d\n" , - child,ceph_vinop (inode), err); - } +struct fscrypt_str oname =FSTR_INIT (NULL ,0 ); +struct ceph_fname fname = { .dir = dir, + .name = rinfo->dname, + .ctext = rinfo->altname, + .name_len = rinfo->dname_len, + .ctext_len = rinfo->altname_len }; + + err =ceph_fname_alloc_buffer (dir, &oname); +if (err <0 ) + goto out; + err =ceph_fname_to_usr (&fname,NULL , &oname,NULL ); +if (!err) { +memcpy (name, oname.name, oname.len); + name[oname.len] =0 ; + } +ceph_fname_free_buffer (dir, &oname); + } +out: +dout ("get_name %p ino %llx.%llx err %d %s%s\n" , + child,ceph_vinop (inode), err, + err ?"" :"name " , err ?"" : name);ceph_mdsc_put_request (req);return err; }
Signature · 1 line
2.31.1
RFC PATCH v8 24/24 ceph: add a new ceph.fscrypt.auth vxattr
In reply to Jeff LaytonGive the clietn away to get at the xattr from userland, mostly for future debugging purposes.
--- fs/ceph/xattr.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
fs/ceph/xattr.c +22 -0
@@ -352,6 +352,21 @@ static ssize_t ceph_vxattrcb_auth_mds(struct ceph_inode_info *ci,return ret; } +static bool ceph_vxattrcb_fscrypt_auth_exists (struct ceph_inode_info *ci) +{ +return ci->fscrypt_auth_len; +} + +static ssize_t ceph_vxattrcb_fscrypt_auth (struct ceph_inode_info *ci,char *val,size_t size) +{ +if (size) { +if (size < ci->fscrypt_auth_len) +return -ERANGE ; +memcpy (val, ci->fscrypt_auth, ci->fscrypt_auth_len); + } +return ci->fscrypt_auth_len; +} +#define CEPH_XATTR_NAME (_type, _name) XATTR_CEPH_PREFIX #_type "." #_name#define CEPH_XATTR_NAME2 (_type, _name, _name2) \ XATTR_CEPH_PREFIX #_type "." #_name "." #_name2
@@ -492,6 +507,13 @@ static struct ceph_vxattr ceph_common_vxattrs[] = { .exists_cb = NULL, .flags = VXATTR_FLAG_READONLY, }, + { + .name = "ceph.fscrypt.auth", + .name_size = sizeof("ceph.fscrypt.auth"), + .getxattr_cb = ceph_vxattrcb_fscrypt_auth, + .exists_cb = ceph_vxattrcb_fscrypt_auth_exists, + .flags = VXATTR_FLAG_READONLY, + }, { .name = NULL, 0 } /* Required table terminator */ };
Signature · 1 line
2.31.1