ceph+fscrypt: full support
Subsystems (5)
v14 cover ceph+fscrypt: full support
Yet another ceph+fscrypt posting. The main changes since v13:
- rebased onto v5.18-rc4 + ceph testing branch patches, fixed up minor merge conflicts, and squashed a few patches together
- squashed in a number of patches from Xiubo to fix issues with truncation handling
- incorporated Luís' patches to encrypt snapshot names
- dropped a patch to export fscrypt's base64 implementation, since Luís' snapshot implementation added a new variant that we need to use.
At this point, I'm mainly waiting on Al to merge this patch into -next and eventually into v5.19:
fs: change test in inode_insert5 for adding to the sb list
Once that's in, it should clear the way for us to start merging the rest of the pile (though probably not all at once).
There are also some mysterious failures that have cropped up in testing with teuthology, particularly with thrash testing. We'll need to get to the bottom of that before we can merge the bulk of these patches.
I've updated the wip-fscrypt branch in the ceph tree with this pile for now. Help with testing would be welcome!
Jeff Layton (48):
libceph: add spinlock around osd->o_requests
libceph: define struct ceph_sparse_extent and add some helpers
libceph: add sparse read support to msgr2 crc state machine
libceph: add sparse read support to OSD client
libceph: support sparse reads on msgr2 secure codepath
libceph: add sparse read support to msgr1
ceph: add new mount option to enable sparse reads
fs: change test in inode_insert5 for adding to the sb list
fscrypt: export fscrypt_fname_encrypt and fscrypt_fname_encrypted_size
fscrypt: add fscrypt_context_for_new_inode
ceph: preallocate inode for ops that may create one
ceph: fscrypt_auth handling for ceph
ceph: ensure that we accept a new context from MDS for new inodes
ceph: add support for fscrypt_auth/fscrypt_file to cap messages
ceph: implement -o test_dummy_encryption mount option
ceph: decode alternate_name in lease info
ceph: add fscrypt ioctls
ceph: make ceph_msdc_build_path use ref-walk
ceph: add encrypted fname handling to ceph_mdsc_build_path
ceph: send altname in MClientRequest
ceph: encode encrypted name in dentry release
ceph: properly set DCACHE_NOKEY_NAME flag in lookup
ceph: set DCACHE_NOKEY_NAME in atomic open
ceph: make d_revalidate call fscrypt revalidator for encrypted
dentries
ceph: add helpers for converting names for userland presentation
ceph: add fscrypt support to ceph_fill_trace
ceph: create symlinks with encrypted and base64-encoded targets
ceph: make ceph_get_name decrypt filenames
ceph: add a new ceph.fscrypt.auth vxattr
ceph: add some fscrypt guardrails
libceph: add CEPH_OSD_OP_ASSERT_VER support
ceph: size handling for encrypted inodes in cap updates
ceph: fscrypt_file field handling in MClientRequest messages
ceph: get file size from fscrypt_file when present in inode traces
ceph: handle fscrypt fields in cap messages from MDS
ceph: update WARN_ON message to pr_warn
ceph: add infrastructure for file encryption and decryption
libceph: allow ceph_osdc_new_request to accept a multi-op read
ceph: disable fallocate for encrypted inodes
ceph: disable copy offload on encrypted inodes
ceph: don't use special DIO path for encrypted inodes
ceph: align data in pages in ceph_sync_write
ceph: add read/modify/write to ceph_sync_write
ceph: plumb in decryption during sync reads
ceph: add fscrypt decryption support to ceph_netfs_issue_op
ceph: set i_blkbits to crypto block size for encrypted inodes
ceph: add encryption support to writepage
ceph: fscrypt support for writepages
Luís Henriques (7):
ceph: add base64 endcoding routines for encrypted names
ceph: don't allow changing layout on encrypted files/directories
ceph: invalidate pages when doing direct/sync writes
ceph: add support for encrypted snapshot names
ceph: add support for handling encrypted snapshot names
ceph: update documentation regarding snapshot naming limitations
ceph: prevent snapshots to be created in encrypted locked directories
Xiubo Li (9):
ceph: make the ioctl cmd more readable in debug log
ceph: fix base64 encoded name's length check in ceph_fname_to_usr()
ceph: pass the request to parse_reply_info_readdir()
ceph: add ceph_encode_encrypted_dname() helper
ceph: add support to readdir for encrypted filenames
ceph: add __ceph_get_caps helper support
ceph: add __ceph_sync_read helper support
ceph: add object version support for sync read
ceph: add truncate size handling support for fscrypt
Documentation/filesystems/ceph.rst | 10 + fs/ceph/Makefile | 1 + fs/ceph/acl.c | 4 +- fs/ceph/addr.c | 136 ++++-- fs/ceph/caps.c | 219 ++++++++-- fs/ceph/crypto.c | 638 +++++++++++++++++++++++++++++ fs/ceph/crypto.h | 264 ++++++++++++ fs/ceph/dir.c | 187 +++++++-- fs/ceph/export.c | 44 +- fs/ceph/file.c | 598 ++++++++++++++++++++++----- fs/ceph/inode.c | 579 +++++++++++++++++++++++--- fs/ceph/ioctl.c | 126 +++++- fs/ceph/mds_client.c | 465 +++++++++++++++++---- fs/ceph/mds_client.h | 24 +- fs/ceph/super.c | 107 ++++- fs/ceph/super.h | 43 +- fs/ceph/xattr.c | 29 ++ fs/crypto/fname.c | 36 +- fs/crypto/fscrypt_private.h | 9 +- fs/crypto/hooks.c | 6 +- fs/crypto/policy.c | 35 +- fs/inode.c | 11 +- include/linux/ceph/ceph_fs.h | 21 +- include/linux/ceph/messenger.h | 32 ++ include/linux/ceph/osd_client.h | 89 +++- include/linux/ceph/rados.h | 4 + include/linux/fscrypt.h | 5 + net/ceph/messenger.c | 1 + net/ceph/messenger_v1.c | 98 ++++- net/ceph/messenger_v2.c | 287 ++++++++++++- net/ceph/osd_client.c | 306 +++++++++++++- 31 files changed, 4009 insertions(+), 405 deletions(-) create mode 100644 fs/ceph/crypto.c create mode 100644 fs/ceph/crypto.h
Signature · 1 line
2.35.1
PATCH v14 1/64 libceph: add spinlock around osd->o_requests
In a later patch, we're going to need to search for a request in the rbtree, but taking the o_mutex is inconvenient as we already hold the con mutex at the point where we need it.
Add a new spinlock that we take when inserting and erasing entries from the o_requests tree. Search of the rbtree can be done with either the mutex or the spinlock, but insertion and removal requires both.
--- include/linux/ceph/osd_client.h | 8 +++++++- net/ceph/osd_client.c | 5 +++++ 2 files changed, 12 insertions(+), 1 deletion(-)
include/linux/ceph/osd_client.h +7 -1
@@ -29,7 +29,12 @@ typedef void (*ceph_osdc_callback_t)(struct ceph_osd_request *);#define CEPH_HOMELESS_OSD -1 -/* a given osd we're communicating with */ +/* +* A given osd we're communicating with. +* +* Note that the o_requests tree can be searched while holding the "lock" mutex +* or the "o_requests_lock" spinlock. Insertion or removal requires both! +*/ struct ceph_osd {refcount_t o_ref;struct ceph_osd_client *o_osdc;
@@ -37,6 +42,7 @@ struct ceph_osd {int o_incarnation;struct rb_node o_node;struct ceph_connection o_con; +spinlock_t o_requests_lock;struct rb_root o_requests;struct rb_root o_linger_requests;struct rb_root o_backoff_mappings;
net/ceph/osd_client.c +5 -0
@@ -1198,6 +1198,7 @@ static void osd_init(struct ceph_osd *osd) {refcount_set (&osd->o_ref,1 );RB_CLEAR_NODE (&osd->o_node); +spin_lock_init (&osd->o_requests_lock); osd->o_requests =RB_ROOT ; osd->o_linger_requests =RB_ROOT ; osd->o_backoff_mappings =RB_ROOT ;
@@ -1427,7 +1428,9 @@ static void link_request(struct ceph_osd *osd, struct ceph_osd_request *req)atomic_inc (&osd->o_osdc->num_homeless);get_osd (osd); +spin_lock (&osd->o_requests_lock);insert_request (&osd->o_requests, req); +spin_unlock (&osd->o_requests_lock); req->r_osd = osd; }
@@ -1439,7 +1442,9 @@ static void unlink_request(struct ceph_osd *osd, struct ceph_osd_request *req) req, req->r_tid); req->r_osd =NULL ; +spin_lock (&osd->o_requests_lock);erase_request (&osd->o_requests, req); +spin_unlock (&osd->o_requests_lock);put_osd (osd);if (!osd_homeless (osd))
Signature · 1 line
2.35.1
PATCH v14 2/64 libceph: define struct ceph_sparse_extent and add some helpers
In reply to Jeff LaytonWhen the OSD sends back a sparse read reply, it contains an array of these structures. Define the structure and add a couple of helpers for dealing with them.
Also add a place in struct ceph_osd_req_op to store the extent buffer, and code to free it if it's populated when the req is torn down.
--- include/linux/ceph/osd_client.h | 43 ++++++++++++++++++++++++++++++++- net/ceph/osd_client.c | 13 ++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-)
include/linux/ceph/osd_client.h +42 -1
@@ -29,6 +29,17 @@ typedef void (*ceph_osdc_callback_t)(struct ceph_osd_request *);#define CEPH_HOMELESS_OSD -1 +/* +* A single extent in a SPARSE_READ reply. +* +* Note that these come from the OSD as little-endian values. On BE arches, +* we convert them in-place after receipt. +*/ +struct ceph_sparse_extent { +u64 off; +u64 len; +} __packed; +/* * A given osd we're communicating with. *
@@ -104,6 +115,8 @@ struct ceph_osd_req_op {u64 offset, length;u64 truncate_size;u32 truncate_seq; +int sparse_ext_cnt; +struct ceph_sparse_extent *sparse_ext;struct ceph_osd_data osd_data; } extent;struct {
@@ -507,6 +520,20 @@ extern struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *, u32 truncate_seq, u64 truncate_size, bool use_mempool); +int __ceph_alloc_sparse_ext_map (struct ceph_osd_req_op *op,int cnt); + +/* +* How big an extent array should we preallocate for a sparse read? This is +* just a starting value. If we get more than this back from the OSD, the +* receiver will reallocate. +*/ +#define CEPH_SPARSE_EXT_ARRAY_INITIAL 16 + +static inline int ceph_alloc_sparse_ext_map (struct ceph_osd_req_op *op) +{ +return __ceph_alloc_sparse_ext_map (op,CEPH_SPARSE_EXT_ARRAY_INITIAL ); +} +extern void ceph_osdc_get_request (struct ceph_osd_request *req);extern void ceph_osdc_put_request (struct ceph_osd_request *req);
@@ -562,5 +589,19 @@ int ceph_osdc_list_watchers(struct ceph_osd_client *osdc,struct ceph_object_locator *oloc, struct ceph_watch_item **watchers, u32 *num_watchers); -#endif +/* Find offset into the buffer of the end of the extent map */ +static inline u64 ceph_sparse_ext_map_end (struct ceph_osd_req_op *op) +{ +struct ceph_sparse_extent *ext; + +/* No extents? No data */ +if (op->extent.sparse_ext_cnt ==0 ) +return 0 ; + + ext = &op->extent.sparse_ext[op->extent.sparse_ext_cnt -1 ]; + +return ext->off + ext->len - op->extent.offset; +} + +#endif
net/ceph/osd_client.c +13 -0
@@ -378,6 +378,7 @@ static void osd_req_op_data_release(struct ceph_osd_request *osd_req,case CEPH_OSD_OP_READ :case CEPH_OSD_OP_WRITE :case CEPH_OSD_OP_WRITEFULL : +kfree (op->extent.sparse_ext);ceph_osd_data_release (&op->extent.osd_data);break ;case CEPH_OSD_OP_CALL :
@@ -1141,6 +1142,18 @@ struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *osdc, }EXPORT_SYMBOL (ceph_osdc_new_request); +int __ceph_alloc_sparse_ext_map (struct ceph_osd_req_op *op,int cnt) +{ + op->extent.sparse_ext_cnt = cnt; + op->extent.sparse_ext =kmalloc_array (cnt, +sizeof (*op->extent.sparse_ext), +GFP_NOFS ); +if (!op->extent.sparse_ext) +return -ENOMEM ; +return 0 ; +} +EXPORT_SYMBOL (__ceph_alloc_sparse_ext_map); +/* * We keep osd requests in an rbtree, sorted by ->r_tid. */
Signature · 1 line
2.35.1
PATCH v14 3/64 libceph: add sparse read support to msgr2 crc state machine
In reply to Jeff LaytonAdd support for a new sparse_read ceph_connection operation. The idea is that the client driver can define this operation use it to do special handling for incoming reads.
The alloc_msg routine will look at the request and determine whether the reply is expected to be sparse. If it is, then we'll dispatch to a different set of state machine states that will repeatedly call the driver's sparse_read op to get length and placement info for reading the extent map, and the extents themselves.
This necessitates adding some new field to some other structs:
- The msg gets a new bool to track whether it's a sparse_read request.
- A new field is added to the cursor to track the amount remaining in the current extent. This is used to cap the read from the socket into the msg_data
- Handing a revoke with all of this is particularly difficult, so I've added a new data_len_remain field to the v2 connection info, and then use that to skip that much on a revoke. We may want to expand the use of that to the normal read path as well, just for consistency's sake.
--- include/linux/ceph/messenger.h | 28 ++++++ net/ceph/messenger.c | 1 + net/ceph/messenger_v2.c | 168 +++++++++++++++++++++++++++++++-- 3 files changed, 188 insertions(+), 9 deletions(-)
include/linux/ceph/messenger.h +28 -0
@@ -17,6 +17,7 @@struct ceph_msg ;struct ceph_connection ; +struct ceph_msg_data_cursor ;/* * Ceph defines these callbacks for handling connection events.
@@ -70,6 +71,30 @@ struct ceph_connection_operations { int used_proto, int result, const int *allowed_protos, int proto_cnt, const int *allowed_modes, int mode_cnt); + +/** +* sparse_read: read sparse data +* @con: connection we're reading from +* @cursor: data cursor for reading extents +* @buf: optional buffer to read into +* +* This should be called more than once, each time setting up to +* receive an extent into the current cursor position, and zeroing +* the holes between them. +* +* Returns amount of data to be read (in bytes), 0 if reading is +* complete, or -errno if there was an error. +* +* If @buf is set on a >0 return, then the data should be read into +* the provided buffer. Otherwise, it should be read into the cursor. +* +* The sparse read operation is expected to initialize the cursor +* with a length covering up to the end of the last extent. +*/ +int (*sparse_read)(struct ceph_connection *con, +struct ceph_msg_data_cursor *cursor, +char **buf); + };/* use format string %s%lld */
@@ -207,6 +232,7 @@ struct ceph_msg_data_cursor {struct ceph_msg_data *data;/* current data item */ size_t resid;/* bytes not yet consumed */ +int sr_resid;/* residual sparse_read len */ bool last_piece;/* current is last piece */ bool need_crc;/* crc update needed */ union {
@@ -252,6 +278,7 @@ struct ceph_msg {struct kref kref;bool more_to_follow;bool needs_out_seq; +bool sparse_read;int front_alloc_len;struct ceph_msgpool *pool;
@@ -396,6 +423,7 @@ struct ceph_connection_v2_info {void *conn_bufs[16 ];int conn_buf_cnt; +int data_len_remain;struct kvec in_sign_kvecs[8 ];struct kvec out_sign_kvecs[8 ];
net/ceph/messenger.c +1 -0
@@ -1034,6 +1034,7 @@ void ceph_msg_data_cursor_init(struct ceph_msg_data_cursor *cursor, cursor->total_resid = length; cursor->data = msg->data; + cursor->sr_resid =0 ;__ceph_msg_data_cursor_init (cursor); }
net/ceph/messenger_v2.c +159 -9
210 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 4/64 libceph: add sparse read support to OSD client
In reply to Jeff LaytonHave get_reply check for the presence of sparse read ops in the request and set the sparse_read boolean in the msg. That will queue the messenger layer to use the sparse read codepath instead of the normal data receive.
Add a new sparse_read operation for the OSD client, driven by its own state machine. The messenger will repeatedly call the sparse_read operation, and it will pass back the necessary info to set up to read the next extent of data, while zero-filling the sparse regions.
The state machine will stop at the end of the last extent, and will attach the extent map buffer to the ceph_osd_req_op so that the caller can use it.
--- include/linux/ceph/osd_client.h | 32 ++++ net/ceph/osd_client.c | 256 +++++++++++++++++++++++++++++++- 2 files changed, 284 insertions(+), 4 deletions(-)
include/linux/ceph/osd_client.h +32 -0
@@ -40,6 +40,36 @@ struct ceph_sparse_extent {u64 len; } __packed; +/* Sparse read state machine state values */ +enum ceph_sparse_read_state { +CEPH_SPARSE_READ_HDR =0 , +CEPH_SPARSE_READ_EXTENTS , +CEPH_SPARSE_READ_DATA_LEN , +CEPH_SPARSE_READ_DATA , +}; + +/* +* A SPARSE_READ reply is a 32-bit count of extents, followed by an array of +* 64-bit offset/length pairs, and then all of the actual file data +* concatenated after it (sans holes). +* +* Unfortunately, we don't know how long the extent array is until we've +* started reading the data section of the reply. The caller should send down +* a destination buffer for the array, but we'll alloc one if it's too small +* or if the caller doesn't. +*/ +struct ceph_sparse_read { +enum ceph_sparse_read_state sr_state;/* state machine state */ +u64 sr_req_off;/* orig request offset */ +u64 sr_req_len;/* orig request length */ +u64 sr_pos;/* current pos in buffer */ +int sr_index;/* current extent index */ +__le32 sr_datalen;/* length of actual data */ +u32 sr_count;/* extent count in reply */ +int sr_ext_len;/* length of extent array */ +struct ceph_sparse_extent *sr_extent;/* extent array */ +}; +/* * A given osd we're communicating with. *
@@ -48,6 +78,7 @@ struct ceph_sparse_extent {*/ struct ceph_osd {refcount_t o_ref; +int o_sparse_op_idx;struct ceph_osd_client *o_osdc;int o_osd;int o_incarnation;
@@ -63,6 +94,7 @@ struct ceph_osd {unsigned long lru_ttl;struct list_head o_keepalive_item;struct mutex lock; +struct ceph_sparse_read o_sparse_read; };#define CEPH_OSD_SLAB_OPS 2
net/ceph/osd_client.c +252 -4
350 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 5/64 libceph: support sparse reads on msgr2 secure codepath
In reply to Jeff LaytonAdd a new init_sgs_pages helper that populates the scatterlist from an arbitrary point in an array of pages.
Change setup_message_sgs to take an optional pointer to an array of pages. If that's set, then the scatterlist will be set using that array instead of the cursor.
When given a sparse read on a secure connection, decrypt the data in-place rather than into the final destination, by passing it the in_enc_pages array.
After decrypting, run the sparse_read state machine in a loop, copying data from the decrypted pages until it's complete.
--- net/ceph/messenger_v2.c | 119 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 109 insertions(+), 10 deletions(-)
net/ceph/messenger_v2.c +109 -10
@@ -963,12 +963,48 @@ static void init_sgs_cursor(struct scatterlist **sg, } } +/** +* init_sgs_pages: set up scatterlist on an array of page pointers +* @sg: scatterlist to populate +* @pages: pointer to page array +* @dpos: position in the array to start (bytes) +* @dlen: len to add to sg (bytes) +* @pad: pointer to pad destination (if any) +* +* Populate the scatterlist from the page array, starting at an arbitrary +* byte in the array and running for a specified length. +*/ +static void init_sgs_pages (struct scatterlist **sg,struct page **pages, +int dpos,int dlen,u8 *pad) +{ +int idx = dpos >>PAGE_SHIFT ; +int off =offset_in_page (dpos); +int resid = dlen; + +do { +int len =min (resid, (int )PAGE_SIZE - off); + +sg_set_page (*sg, pages[idx], len, off); + *sg =sg_next (*sg); + off =0 ; + ++idx; + resid -= len; + }while (resid); + +if (need_padding (dlen)) { +sg_set_buf (*sg, pad,padding_len (dlen)); + *sg =sg_next (*sg); + } +} +static int setup_message_sgs (struct sg_table *sgt,struct ceph_msg *msg,u8 *front_pad,u8 *middle_pad,u8 *data_pad, -void *epilogue,bool add_tag) +void *epilogue,struct page **pages,int dpos, +bool add_tag) {struct ceph_msg_data_cursor cursor;struct scatterlist *cur_sg; +int dlen =data_len (msg);int sg_cnt;int ret;
@@ -982,9 +1018,15 @@ static int setup_message_sgs(struct sg_table *sgt, struct ceph_msg *msg,if (middle_len (msg)) sg_cnt +=calc_sg_cnt (msg->middle->vec.iov_base,middle_len (msg)); -if (data_len (msg)) { -ceph_msg_data_cursor_init (&cursor, msg,data_len (msg)); - sg_cnt +=calc_sg_cnt_cursor (&cursor); +if (dlen) { +if (pages) { + sg_cnt +=calc_pages_for (dpos, dlen); +if (need_padding (dlen)) + sg_cnt++; + }else { +ceph_msg_data_cursor_init (&cursor, msg, dlen); + sg_cnt +=calc_sg_cnt_cursor (&cursor); + } } ret =sg_alloc_table (sgt, sg_cnt,GFP_NOIO );
@@ -998,9 +1040,13 @@ static int setup_message_sgs(struct sg_table *sgt, struct ceph_msg *msg,if (middle_len (msg))init_sgs (&cur_sg, msg->middle->vec.iov_base,middle_len (msg), middle_pad); -if (data_len (msg)) { -ceph_msg_data_cursor_init (&cursor, msg,data_len (msg)); -init_sgs_cursor (&cur_sg, &cursor, data_pad); +if (dlen) { +if (pages) { +init_sgs_pages (&cur_sg, pages, dpos, dlen, data_pad); + }else { +ceph_msg_data_cursor_init (&cursor, msg, dlen); +init_sgs_cursor (&cur_sg, &cursor, data_pad); + } }WARN_ON (!sg_is_last (cur_sg));
@@ -1035,10 +1081,52 @@ static int decrypt_control_remainder(struct ceph_connection *con) padded_len(rem_len) + CEPH_GCM_TAG_LEN); } +/* Process sparse read data that lives in a buffer */ +static int process_v2_sparse_read (struct ceph_connection *con,struct page **pages,int spos) +{ +struct ceph_msg_data_cursor *cursor = &con->v2.in_cursor; +int ret; + +for (;;) { +char *buf =NULL ; + + ret = con->ops->sparse_read (con, cursor, &buf); +if (ret <=0 ) +return ret; + +dout ("%s: sparse_read return %x buf %p\n" , __func__, ret, buf); + +do { +int idx = spos >>PAGE_SHIFT ; +int soff =offset_in_page (spos); +struct page *spage = con->v2.in_enc_pages[idx]; +int len =min_t (int, ret,PAGE_SIZE - soff); + +if (buf) { +memcpy_from_page (buf, spage, soff, len); + buf += len; + }else { +struct bio_vec bv; + +get_bvec_at (cursor, &bv); + len =min_t (int, len, bv.bv_len); +memcpy_page (bv.bv_page, bv.bv_offset, + spage, soff, len); +ceph_msg_data_advance (cursor, len); + } + spos += len; + ret -= len; + }while (ret); + } +} +static int decrypt_tail (struct ceph_connection *con) {struct sg_table enc_sgt = {};struct sg_table sgt = {}; +struct page **pages =NULL ; +bool sparse = con->in_msg->sparse_read; +int dpos =0 ;int tail_len;int ret;
@@ -1049,9 +1137,14 @@ static int decrypt_tail(struct ceph_connection *con)if (ret) goto out; +if (sparse) { + dpos =padded_len (front_len (con->in_msg) +padded_len (middle_len (con->in_msg))); + pages = con->v2.in_enc_pages; + } + ret =setup_message_sgs (&sgt, con->in_msg,FRONT_PAD (con->v2.in_buf), -MIDDLE_PAD (con->v2.in_buf),DATA_PAD (con->v2.in_buf), - con->v2.in_buf, true); +MIDDLE_PAD (con->v2.in_buf),DATA_PAD (con->v2.in_buf), + con->v2.in_buf, pages, dpos, true);if (ret) goto out;
@@ -1061,6 +1154,12 @@ static int decrypt_tail(struct ceph_connection *con)if (ret) goto out; +if (sparse &&data_len (con->in_msg)) { + ret =process_v2_sparse_read (con, con->v2.in_enc_pages, dpos); +if (ret) + goto out; + } +WARN_ON (!con->v2.in_enc_page_cnt);ceph_release_page_vector (con->v2.in_enc_pages, con->v2.in_enc_page_cnt);
@@ -1584,7 +1683,7 @@ static int prepare_message_secure(struct ceph_connection *con)encode_epilogue_secure (con, false); ret =setup_message_sgs (&sgt, con->out_msg, zerop, zerop, zerop, - &con->v2.out_epil, false); + &con->v2.out_epil,NULL ,0 , false);if (ret) goto out;
Signature · 1 line
2.35.1
PATCH v14 6/64 libceph: add sparse read support to msgr1
In reply to Jeff LaytonAdd 2 new fields to ceph_connection_v1_info to track the necessary info in sparse reads. Skip initializing the cursor for a sparse read.
Break out read_partial_message_section into a wrapper around a new read_partial_message_chunk function that doesn't zero out the crc first.
Add new helper functions to drive receiving into the destinations provided by the sparse_read state machine.
--- include/linux/ceph/messenger.h | 4 ++ net/ceph/messenger_v1.c | 98 +++++++++++++++++++++++++++++++--- 2 files changed, 94 insertions(+), 8 deletions(-)
include/linux/ceph/messenger.h +4 -0
@@ -337,6 +337,10 @@ struct ceph_connection_v1_info {int in_base_pos;/* bytes read */ +/* sparse reads */ +struct kvec in_sr_kvec;/* current location to receive into */ +u64 in_sr_len;/* amount of data in this extent */ +/* message in temps */ u8 in_tag;/* protocol control byte */ struct ceph_msg_header in_hdr;
net/ceph/messenger_v1.c +90 -8
126 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 7/64 ceph: add new mount option to enable sparse reads
In reply to Jeff LaytonAdd a new mount option that has the client issue sparse reads instead of normal ones. The callers now preallocate an sparse extent buffer that the libceph receive code can populate and hand back after the operation completes.
After a successful sparse read, we can't use the req->r_result value to determine the amount of data "read", so instead we set the received length to be from the end of the last extent in the buffer. Any interstitial holes will have been filled by the receive code.
--- fs/ceph/addr.c | 17 +++++++++++++++-- fs/ceph/file.c | 51 +++++++++++++++++++++++++++++++++++++++++-------- fs/ceph/super.c | 16 +++++++++++++++- fs/ceph/super.h | 1 + 4 files changed, 74 insertions(+), 11 deletions(-)
fs/ceph/addr.c +15 -2
@@ -219,8 +219,10 @@ static void finish_netfs_read(struct ceph_osd_request *req)struct ceph_fs_client *fsc =ceph_inode_to_client (req->r_inode);struct ceph_osd_data *osd_data =osd_req_op_extent_osd_data (req,0 );struct netfs_io_subrequest *subreq = req->r_priv; +struct ceph_osd_req_op *op = &req->r_ops[0 ];int num_pages;int err = req->r_result; +bool sparse = (op->op ==CEPH_OSD_OP_SPARSE_READ );ceph_update_read_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, osd_data->length, err);
@@ -229,7 +231,9 @@ static void finish_netfs_read(struct ceph_osd_request *req) subreq->len, i_size_read(req->r_inode));/* no object means success but no data */ -if (err == -ENOENT ) +if (sparse && err >=0 ) + err =ceph_sparse_ext_map_end (op); +else if (err == -ENOENT ) err =0 ;else if (err == -EBLOCKLISTED ) fsc->blocklisted = true;
@@ -312,13 +316,14 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)size_t page_off;int err =0 ;u64 len = subreq->len; +bool sparse =ceph_test_mount_opt (fsc,SPARSEREAD );if (ci->i_inline_version !=CEPH_INLINE_NONE &&ceph_netfs_issue_op_inline (subreq))return ; req =ceph_osdc_new_request (&fsc->client->osdc, &ci->i_layout, vino, subreq->start, &len, -0 ,1 ,CEPH_OSD_OP_READ , +0 ,1 , sparse ?CEPH_OSD_OP_SPARSE_READ :CEPH_OSD_OP_READ ,CEPH_OSD_FLAG_READ | fsc->client->osdc.client->options->read_from_replica,NULL , ci->i_truncate_seq, ci->i_truncate_size, false);if (IS_ERR (req)) {
@@ -327,6 +332,14 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) goto out; } +if (sparse) { + err =ceph_alloc_sparse_ext_map (&req->r_ops[0 ]); +if (err) { +ceph_osdc_put_request (req); + goto out; + } + } +dout ("%s: pos=%llu orig_len=%zu len=%llu\n" , __func__, subreq->start, subreq->len, len);iov_iter_xarray (&iter,READ , &rreq->mapping->i_pages, subreq->start, len); err =iov_iter_get_pages_alloc (&iter, &pages, len, &page_off);
fs/ceph/file.c +43 -8
132 lines not on this page · raw message
fs/ceph/super.c +15 -1
@@ -163,6 +163,7 @@ enum { Opt_copyfrom, Opt_wsync, Opt_pagecache, + Opt_sparseread, }; enum ceph_recover_session_mode {
@@ -205,6 +206,7 @@ static const struct fs_parameter_spec ceph_mount_parameters[] = {fsparam_u32 ("wsize" , Opt_wsize),fsparam_flag_no ("wsync" , Opt_wsync),fsparam_flag_no ("pagecache" , Opt_pagecache), + fsparam_flag_no ("sparseread", Opt_sparseread), {} };
@@ -574,6 +576,12 @@ static int ceph_parse_mount_param(struct fs_context *fc,else fsopt->flags &= ~CEPH_MOUNT_OPT_NOPAGECACHE ;break ; +case Opt_sparseread: +if (result.negated) + fsopt->flags &= ~CEPH_MOUNT_OPT_SPARSEREAD ; +else + fsopt->flags |=CEPH_MOUNT_OPT_SPARSEREAD ; +break ;default :BUG (); }
@@ -708,9 +716,10 @@ static int ceph_show_options(struct seq_file *m, struct dentry *root)if (!(fsopt->flags &CEPH_MOUNT_OPT_ASYNC_DIROPS ))seq_puts (m,",wsync" ); -if (fsopt->flags &CEPH_MOUNT_OPT_NOPAGECACHE )seq_puts (m,",nopagecache" ); +if (fsopt->flags &CEPH_MOUNT_OPT_SPARSEREAD ) +seq_puts (m,",sparseread" );if (fsopt->wsize !=CEPH_MAX_WRITE_SIZE )seq_printf (m,",wsize=%u" , fsopt->wsize);
@@ -1291,6 +1300,11 @@ static int ceph_reconfigure_fc(struct fs_context *fc)else ceph_clear_mount_opt (fsc,ASYNC_DIROPS ); +if (fsopt->flags &CEPH_MOUNT_OPT_SPARSEREAD ) +ceph_set_mount_opt (fsc,SPARSEREAD ); +else +ceph_clear_mount_opt (fsc,SPARSEREAD ); +if (strcmp_null (fsc->mount_options->mon_addr, fsopt->mon_addr)) {kfree (fsc->mount_options->mon_addr); fsc->mount_options->mon_addr = fsopt->mon_addr;
fs/ceph/super.h +1 -0
@@ -41,6 +41,7 @@#define CEPH_MOUNT_OPT_NOCOPYFROM (1<<14)/* don't use RADOS 'copy-from' op */ #define CEPH_MOUNT_OPT_ASYNC_DIROPS (1<<15)/* allow async directory ops */ #define CEPH_MOUNT_OPT_NOPAGECACHE (1<<16)/* bypass pagecache altogether */ +#define CEPH_MOUNT_OPT_SPARSEREAD (1<<17)/* always do sparse reads */ #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \
Signature · 1 line
2.35.1
PATCH v14 8/64 fs: change test in inode_insert5 for adding to the sb list
In reply to Jeff LaytonThe inode_insert5 currently looks at I_CREATING to decide whether to insert the inode into the sb list. This test is a bit ambiguous though as I_CREATING state is not directly related to that list.
This test is also problematic for some upcoming ceph changes to add fscrypt support. We need to be able to allocate an inode using new_inode and insert it into the hash later if we end up using it, and doing that now means that we double add it and corrupt the list.
What we really want to know in this test is whether the inode is already in its superblock list, and then add it if it isn't. Have it test for list_empty instead and ensure that we always initialize the list by doing it in inode_init_once. It's only ever removed from the list with list_del_init, so that should be sufficient.
--- fs/inode.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
fs/inode.c +8 -3
@@ -422,6 +422,7 @@ void inode_init_once(struct inode *inode)INIT_LIST_HEAD (&inode->i_io_list);INIT_LIST_HEAD (&inode->i_wb_list);INIT_LIST_HEAD (&inode->i_lru); +INIT_LIST_HEAD (&inode->i_sb_list);__address_space_init_once (&inode->i_data);i_size_ordered_init (inode); }
@@ -1021,7 +1022,6 @@ struct inode *new_inode_pseudo(struct super_block *sb)spin_lock (&inode->i_lock); inode->i_state =0 ;spin_unlock (&inode->i_lock); -INIT_LIST_HEAD (&inode->i_sb_list); }return inode; }
@@ -1165,7 +1165,6 @@ struct inode *inode_insert5(struct inode *inode, unsigned long hashval, {struct hlist_head *head = inode_hashtable +hash (inode->i_sb, hashval);struct inode *old; -bool creating = inode->i_state &I_CREATING ; again:spin_lock (&inode_hash_lock);
@@ -1199,7 +1198,13 @@ struct inode *inode_insert5(struct inode *inode, unsigned long hashval, inode->i_state |=I_NEW ;hlist_add_head_rcu (&inode->i_hash, head);spin_unlock (&inode->i_lock); -if (!creating) + +/* +* Add it to the list if it wasn't already in, +* e.g. new_inode. We hold I_NEW at this point, so +* we should be safe to test i_sb_list locklessly. +*/ +if (list_empty (&inode->i_sb_list))inode_sb_list_add (inode); unlock:spin_unlock (&inode_hash_lock);
Signature · 1 line
2.35.1
PATCH v14 9/64 fscrypt: export fscrypt_fname_encrypt and fscrypt_fname_encrypted_size
In reply to Jeff LaytonFor ceph, we want to use our own scheme for handling filenames that are are longer than NAME_MAX after encryption and Base64 encoding. This allows us to have a consistent view of the encrypted filenames for clients that don't support fscrypt and clients that do but that don't have the key.
Currently, fs/crypto only supports encrypting filenames using fscrypt_setup_filename, but that also handles encoding nokey names. Ceph can't use that because it handles nokey names in a different way.
Export fscrypt_fname_encrypt. Rename fscrypt_fname_encrypted_size to __fscrypt_fname_encrypted_size and add a new wrapper called fscrypt_fname_encrypted_size that takes an inode argument rather than a pointer to a fscrypt_policy union.
--- fs/crypto/fname.c | 36 ++++++++++++++++++++++++++++++------ fs/crypto/fscrypt_private.h | 9 +++------ fs/crypto/hooks.c | 6 +++--- include/linux/fscrypt.h | 4 ++++ 4 files changed, 40 insertions(+), 15 deletions(-)
fs/crypto/fname.c +30 -6
@@ -79,7 +79,8 @@ static inline bool fscrypt_is_dot_dotdot(const struct qstr *str)/** * fscrypt_fname_encrypt() - encrypt a filename * @inode: inode of the parent directory (for regular filenames) -* or of the symlink (for symlink targets) +* or of the symlink (for symlink targets). Key must already be +* set up. * @iname: the filename to encrypt * @out: (output) the encrypted filename * @olen: size of the encrypted filename. It must be at least @iname->len.
@@ -130,6 +131,7 @@ int fscrypt_fname_encrypt(const struct inode *inode, const struct qstr *iname,return 0 ; } +EXPORT_SYMBOL_GPL (fscrypt_fname_encrypt);/** * fname_decrypt() - decrypt a filename
@@ -257,9 +259,9 @@ static int fscrypt_base64url_decode(const char *src, int srclen, u8 *dst)return bp - dst; } -bool fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, -u32 orig_len,u32 max_len, -u32 *encrypted_len_ret) +bool __fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, +u32 orig_len,u32 max_len, +u32 *encrypted_len_ret) {int padding =4 << (fscrypt_policy_flags (policy) &FSCRYPT_POLICY_FLAGS_PAD_MASK );
@@ -273,6 +275,29 @@ bool fscrypt_fname_encrypted_size(const union fscrypt_policy *policy,return true; } +/** +* fscrypt_fname_encrypted_size() - calculate length of encrypted filename +* @inode: parent inode of dentry name being encrypted. Key must +* already be set up. +* @orig_len: length of the original filename +* @max_len: maximum length to return +* @encrypted_len_ret: where calculated length should be returned (on success) +* +* Filenames that are shorter than the maximum length may have their lengths +* increased slightly by encryption, due to padding that is applied. +* +* Return: false if the orig_len is greater than max_len. Otherwise, true and +* fill out encrypted_len_ret with the length (up to max_len). +*/ +bool fscrypt_fname_encrypted_size (const struct inode *inode,u32 orig_len, +u32 max_len,u32 *encrypted_len_ret) +{ +return __fscrypt_fname_encrypted_size (&inode->i_crypt_info->ci_policy, + orig_len, max_len, + encrypted_len_ret); +} +EXPORT_SYMBOL_GPL (fscrypt_fname_encrypted_size); +/** * fscrypt_fname_alloc_buffer() - allocate a buffer for presented filenames * @max_encrypted_len: maximum length of encrypted filenames the buffer will be
@@ -428,8 +453,7 @@ int fscrypt_setup_filename(struct inode *dir, const struct qstr *iname,return ret;if (fscrypt_has_encryption_key (dir)) { -if (!fscrypt_fname_encrypted_size (&dir->i_crypt_info->ci_policy, - iname->len,NAME_MAX , +if (!fscrypt_fname_encrypted_size (dir, iname->len,NAME_MAX , &fname->crypto_buf.len))return -ENAMETOOLONG ; fname->crypto_buf.name = kmalloc(fname->crypto_buf.len,
fs/crypto/fscrypt_private.h +3 -6
@@ -297,14 +297,11 @@ void fscrypt_generate_iv(union fscrypt_iv *iv, u64 lblk_num, const struct fscrypt_info *ci);/* fname.c */ -int fscrypt_fname_encrypt (const struct inode *inode,const struct qstr *iname, -u8 *out,unsigned int olen); -bool fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, -u32 orig_len,u32 max_len, -u32 *encrypted_len_ret); +bool __fscrypt_fname_encrypted_size (const union fscrypt_policy *policy, +u32 orig_len,u32 max_len, +u32 *encrypted_len_ret);/* hkdf.c */ -struct fscrypt_hkdf {struct crypto_shash *hmac_tfm; };
fs/crypto/hooks.c +3 -3
@@ -228,9 +228,9 @@ int fscrypt_prepare_symlink(struct inode *dir, const char *target,* counting it (even though it is meaningless for ciphertext) is simpler * for now since filesystems will assume it is there and subtract it. */ -if (!fscrypt_fname_encrypted_size (policy, len, - max_len -sizeof (struct fscrypt_symlink_data ), - &disk_link->len)) +if (!__fscrypt_fname_encrypted_size (policy, len, + max_len -sizeof (struct fscrypt_symlink_data ), + &disk_link->len))return -ENAMETOOLONG ; disk_link->len +=sizeof (struct fscrypt_symlink_data );
include/linux/fscrypt.h +4 -0
@@ -305,6 +305,10 @@ void fscrypt_free_inode(struct inode *inode);int fscrypt_drop_inode (struct inode *inode);/* fname.c */ +int fscrypt_fname_encrypt (const struct inode *inode,const struct qstr *iname, +u8 *out,unsigned int olen); +bool fscrypt_fname_encrypted_size (const struct inode *inode,u32 orig_len, +u32 max_len,u32 *encrypted_len_ret);int fscrypt_setup_filename (struct inode *inode,const struct qstr *iname,int lookup,struct fscrypt_name *fname);
Signature · 1 line
2.35.1
PATCH v14 10/64 fscrypt: add fscrypt_context_for_new_inode
In reply to Jeff LaytonMost filesystems just call fscrypt_set_context on new inodes, which usually causes a setxattr. That's a bit late for ceph, which can send along a full set of attributes with the create request.
Doing so allows it to avoid race windows that where the new inode could be seen by other clients without the crypto context attached. It also avoids the separate round trip to the server.
Refactor the fscrypt code a bit to allow us to create a new crypto context, attach it to the inode, and write it to the buffer, but without calling set_context on it. ceph can later use this to marshal the context into the attributes we send along with the create request.
--- fs/crypto/policy.c | 35 +++++++++++++++++++++++++++++------ include/linux/fscrypt.h | 1 + 2 files changed, 30 insertions(+), 6 deletions(-)
fs/crypto/policy.c +29 -6
@@ -664,6 +664,32 @@ const union fscrypt_policy *fscrypt_policy_to_inherit(struct inode *dir)return fscrypt_get_dummy_policy (dir->i_sb); } +/** +* fscrypt_context_for_new_inode() - create an encryption context for a new inode +* @ctx: where context should be written +* @inode: inode from which to fetch policy and nonce +* +* Given an in-core "prepared" (via fscrypt_prepare_new_inode) inode, +* generate a new context and write it to ctx. ctx _must_ be at least +* FSCRYPT_SET_CONTEXT_MAX_SIZE bytes. +* +* Return: size of the resulting context or a negative error code. +*/ +int fscrypt_context_for_new_inode (void *ctx,struct inode *inode) +{ +struct fscrypt_info *ci = inode->i_crypt_info; + +BUILD_BUG_ON (sizeof (union fscrypt_context ) != +FSCRYPT_SET_CONTEXT_MAX_SIZE ); + +/* fscrypt_prepare_new_inode() should have set up the key already. */ +if (WARN_ON_ONCE (!ci)) +return -ENOKEY ; + +return fscrypt_new_context (ctx, &ci->ci_policy, ci->ci_nonce); +} +EXPORT_SYMBOL_GPL (fscrypt_context_for_new_inode); +/** * fscrypt_set_context() - Set the fscrypt context of a new inode * @inode: a new inode
@@ -680,12 +706,9 @@ int fscrypt_set_context(struct inode *inode, void *fs_data)union fscrypt_context ctx;int ctxsize; -/* fscrypt_prepare_new_inode() should have set up the key already. */ -if (WARN_ON_ONCE (!ci)) -return -ENOKEY ; - -BUILD_BUG_ON (sizeof (ctx) !=FSCRYPT_SET_CONTEXT_MAX_SIZE ); - ctxsize =fscrypt_new_context (&ctx, &ci->ci_policy, ci->ci_nonce); + ctxsize =fscrypt_context_for_new_inode (&ctx, inode); +if (ctxsize <0 ) +return ctxsize;/* * This may be the first time the inode number is available, so do any
include/linux/fscrypt.h +1 -0
@@ -273,6 +273,7 @@ int fscrypt_ioctl_get_policy(struct file *filp, void __user *arg);int fscrypt_ioctl_get_policy_ex (struct file *filp,void __user *arg);int fscrypt_ioctl_get_nonce (struct file *filp,void __user *arg);int fscrypt_has_permitted_context (struct inode *parent,struct inode *child); +int fscrypt_context_for_new_inode (void *ctx,struct inode *inode);int fscrypt_set_context (struct inode *inode,void *fs_data);struct fscrypt_dummy_policy {
Signature · 1 line
2.35.1
PATCH v14 11/64 ceph: preallocate inode for ops that may create one
In reply to Jeff LaytonWhen creating a new inode, we need to determine the crypto context before we can transmit the RPC. The fscrypt API has a routine for getting a crypto context before a create occurs, but it requires an inode.
Change the ceph code to preallocate an inode in advance of a create of any sort (open(), mknod(), symlink(), etc). Move the existing code that generates the ACL and SELinux blobs into this routine since that's mostly common across all the different codepaths.
In most cases, we just want to allow ceph_fill_trace to use that inode after the reply comes in, so add a new field to the MDS request for it (r_new_inode).
The async create codepath is a bit different though. In that case, we want to hash the inode in advance of the RPC so that it can be used before the reply comes in. If the call subsequently fails with -EJUKEBOX, then just put the references and clean up the as_ctx. Note that with this change, we now need to regenerate the as_ctx when this occurs, but it's quite rare for it to happen.
--- fs/ceph/dir.c | 70 ++++++++++++++++++++----------------- fs/ceph/file.c | 62 ++++++++++++++++++++------------- fs/ceph/inode.c | 82 ++++++++++++++++++++++++++++++++++++++++---- fs/ceph/mds_client.c | 13 +++++-- fs/ceph/mds_client.h | 1 + fs/ceph/super.h | 7 +++- 6 files changed, 169 insertions(+), 66 deletions(-)
fs/ceph/dir.c +38 -32
@@ -861,13 +861,6 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - err =ceph_pre_init_acls (dir, &mode, &as_ctx); -if (err <0 ) - goto out; - err =ceph_security_init_secctx (dentry, mode, &as_ctx); -if (err <0 ) - goto out; -dout ("mknod in dir %p dentry %p mode 0%ho rdev %d\n" , dir, dentry, mode, rdev); req =ceph_mdsc_create_request (mdsc,CEPH_MDS_OP_MKNOD ,USE_AUTH_MDS );
@@ -875,6 +868,14 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, err =PTR_ERR (req); goto out; } + + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_dentry =dget (dentry); req->r_num_caps =2 ; req->r_parent = dir;
@@ -884,13 +885,13 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, req->r_args.mknod.rdev =cpu_to_le32 (rdev); req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (!err)
@@ -913,6 +914,7 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir,struct ceph_mds_client *mdsc =ceph_sb_to_mdsc (dir->i_sb);struct ceph_mds_request *req;struct ceph_acl_sec_ctx as_ctx = {}; +umode_t mode =S_IFLNK |0777 ;int err;if (ceph_snap (dir) !=CEPH_NOSNAP )
@@ -923,21 +925,24 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - err =ceph_security_init_secctx (dentry,S_IFLNK |0777 , &as_ctx); -if (err <0 ) - goto out; -dout ("symlink in dir %p dentry %p to '%s'\n" , dir, dentry, dest); req =ceph_mdsc_create_request (mdsc,CEPH_MDS_OP_SYMLINK ,USE_AUTH_MDS );if (IS_ERR (req)) { err =PTR_ERR (req); goto out; } + + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_path2 =kstrdup (dest,GFP_KERNEL );if (!req->r_path2) { err = -ENOMEM ; -ceph_mdsc_put_request (req); - goto out; + goto out_req; } req->r_parent = dir;ihold (dir);
@@ -947,13 +952,13 @@ static int ceph_symlink(struct user_namespace *mnt_userns, struct inode *dir, req->r_num_caps =2 ; req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (err)
@@ -989,13 +994,6 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, goto out; } - mode |=S_IFDIR ; - err =ceph_pre_init_acls (dir, &mode, &as_ctx); -if (err <0 ) - goto out; - err =ceph_security_init_secctx (dentry, mode, &as_ctx); -if (err <0 ) - goto out; req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS );if (IS_ERR (req)) {
@@ -1003,6 +1001,14 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, goto out; } + mode |=S_IFDIR ; + req->r_new_inode =ceph_new_inode (dir, dentry, &mode, &as_ctx); +if (IS_ERR (req->r_new_inode)) { + err =PTR_ERR (req->r_new_inode); + req->r_new_inode =NULL ; + goto out_req; + } + req->r_dentry =dget (dentry); req->r_num_caps =2 ; req->r_parent = dir;
@@ -1011,15 +1017,15 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, req->r_args.mkdir.mode =cpu_to_le32 (mode); req->r_dentry_drop =CEPH_CAP_FILE_SHARED |CEPH_CAP_AUTH_EXCL ; req->r_dentry_unless =CEPH_CAP_FILE_EXCL ; -if (as_ctx.pagelist) { - req->r_pagelist = as_ctx.pagelist; - as_ctx.pagelist =NULL ; - } + +ceph_as_ctx_to_req (req, &as_ctx); + err =ceph_mdsc_do_request (mdsc, dir, req);if (!err && !req->r_reply_info.head->is_target && !req->r_reply_info.head->is_dentry) err =ceph_handle_notrace_create (dir, dentry); +out_req:ceph_mdsc_put_request (req); out:if (!err)
fs/ceph/file.c +38 -24
136 lines not on this page · raw message
fs/ceph/inode.c +75 -7
113 lines not on this page · raw message
fs/ceph/mds_client.c +11 -2
@@ -868,6 +868,7 @@ void ceph_mdsc_release_request(struct kref *kref)iput (req->r_parent); }iput (req->r_target_inode); +iput (req->r_new_inode);if (req->r_dentry)dput (req->r_dentry);if (req->r_old_dentry)
@@ -3208,13 +3209,21 @@ static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg)/* Must find target inode outside of mutexes to avoid deadlocks */ if ((err >=0 ) && rinfo->head->is_target) { -struct inode *in; +struct inode *in =xchg (&req->r_new_inode,NULL );struct ceph_vino tvino = { .ino =le64_to_cpu (rinfo->targeti.in->ino), .snap =le64_to_cpu (rinfo->targeti.in->snapid) }; - in =ceph_get_inode (mdsc->fsc->sb, tvino); +/* If we ended up opening an existing inode, discard r_new_inode */ +if (req->r_op ==CEPH_MDS_OP_CREATE && !req->r_reply_info.has_create_ino) { +/* This should never happen on an async create */ +WARN_ON_ONCE (req->r_deleg_ino); +iput (in); + in =NULL ; + } + + in =ceph_get_inode (mdsc->fsc->sb, tvino, in);if (IS_ERR (in)) { err =PTR_ERR (in);mutex_lock (&session->s_mutex);
fs/ceph/mds_client.h +1 -0
@@ -265,6 +265,7 @@ struct ceph_mds_request {struct inode *r_parent;/* parent dir inode */ struct inode *r_target_inode;/* resulting inode */ +struct inode *r_new_inode;/* new inode (for creates) */ #define CEPH_MDS_R_DIRECT_IS_HASH (1)/* r_direct_hash is valid */ #define CEPH_MDS_R_ABORTED (2)/* call was aborted */
fs/ceph/super.h +6 -1
@@ -967,6 +967,7 @@ static inline bool __ceph_have_pending_cap_snap(struct ceph_inode_info *ci)/* inode.c */ struct ceph_mds_reply_info_in ;struct ceph_mds_reply_dirfrag ; +struct ceph_acl_sec_ctx ;extern const struct inode_operations ceph_file_iops;
@@ -974,8 +975,12 @@ extern struct inode *ceph_alloc_inode(struct super_block *sb);extern void ceph_evict_inode (struct inode *inode);extern void ceph_free_inode (struct inode *inode); +struct inode *ceph_new_inode (struct inode *dir,struct dentry *dentry, +umode_t *mode,struct ceph_acl_sec_ctx *as_ctx); +void ceph_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as_ctx); +extern struct inode *ceph_get_inode (struct super_block *sb, -struct ceph_vino vino); +struct ceph_vino vino,struct inode *newino);extern struct inode *ceph_get_snapdir (struct inode *parent);extern int ceph_fill_file_size (struct inode *inode,int issued,u32 truncate_seq,u64 truncate_size,u64 size);
Signature · 1 line
2.35.1
PATCH v14 12/64 ceph: fscrypt_auth handling for ceph
In reply to Jeff LaytonMost fscrypt-enabled filesystems store the crypto context in an xattr, but that's problematic for ceph as xatts are governed by the XATTR cap, but we really want the crypto context as part of the AUTH cap.
Because of this, the MDS has added two new inode metadata fields:fscrypt_auth and fscrypt_file. The former is used to hold the crypto context, and the latter is used to track the real file size.
Parse new fscrypt_auth and fscrypt_file fields in inode traces. For now, we don't use fscrypt_file, but fscrypt_auth is used to hold the fscrypt context.
Allow the client to use a setattr request for setting the fscrypt_auth field. Since this is not a standard setattr request from the VFS, we add a new field to __ceph_setattr that carries ceph-specific inode attrs.
Have the set_context op do a setattr that sets the fscrypt_auth value, and get_context just return the contents of that field (since it should always be available).
--- fs/ceph/Makefile | 1 + fs/ceph/acl.c | 4 +- fs/ceph/crypto.c | 76 ++++++++++++++++++++++++ fs/ceph/crypto.h | 36 ++++++++++++ fs/ceph/inode.c | 62 ++++++++++++++++++- fs/ceph/mds_client.c | 111 ++++++++++++++++++++++++++++++++--- fs/ceph/mds_client.h | 7 +++ fs/ceph/super.c | 3 + fs/ceph/super.h | 14 ++++- include/linux/ceph/ceph_fs.h | 21 ++++--- 10 files changed, 313 insertions(+), 22 deletions(-) create mode 100644 fs/ceph/crypto.c create mode 100644 fs/ceph/crypto.h
fs/ceph/Makefile +1 -0
@@ -12,3 +12,4 @@ ceph-y := super.o inode.o dir.o file.o locks.o addr.o ioctl.o \ ceph-$(CONFIG_CEPH_FSCACHE) += cache.o ceph-$(CONFIG_CEPH_FS_POSIX_ACL) += acl.o +ceph-$(CONFIG_FS_ENCRYPTION) += crypto.o
fs/ceph/acl.c +2 -2
@@ -139,7 +139,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime =current_time (inode); newattrs.ia_mode = new_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; - ret =__ceph_setattr (inode, &newattrs); + ret =__ceph_setattr (inode, &newattrs,NULL );if (ret) goto out_free; }
@@ -150,7 +150,7 @@ int ceph_set_acl(struct user_namespace *mnt_userns, struct inode *inode, newattrs.ia_ctime = old_ctime; newattrs.ia_mode = old_mode; newattrs.ia_valid =ATTR_MODE |ATTR_CTIME ; -__ceph_setattr (inode, &newattrs); +__ceph_setattr (inode, &newattrs,NULL ); } goto out_free; }
fs/ceph/crypto.c new file +76 -0
76 lines not on this page · raw message
fs/ceph/crypto.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Ceph fscrypt functionality +*/ + +#ifndef _CEPH_CRYPTO_H +#define _CEPH_CRYPTO_H + +#include <linux/fscrypt.h> + +struct ceph_fscrypt_auth { +__le32 cfa_version; +__le32 cfa_blob_len; +u8 cfa_blob[FSCRYPT_SET_CONTEXT_MAX_SIZE ]; +} __packed; + +#define CEPH_FSCRYPT_AUTH_VERSION 1 +static inline u32 ceph_fscrypt_auth_len (struct ceph_fscrypt_auth *fa) +{ +u32 ctxsize =le32_to_cpu (fa->cfa_blob_len); + +return offsetof(struct ceph_fscrypt_auth , cfa_blob) + ctxsize; +} + +#ifdef CONFIG_FS_ENCRYPTION +void ceph_fscrypt_set_ops (struct super_block *sb); + +#else /* CONFIG_FS_ENCRYPTION */ + +static inline void ceph_fscrypt_set_ops (struct super_block *sb) +{ +} + +#endif /* CONFIG_FS_ENCRYPTION */ + +#endif
fs/ceph/inode.c +60 -2
120 lines not on this page · raw message
fs/ceph/mds_client.c +102 -9
183 lines not on this page · raw message
fs/ceph/mds_client.h +7 -0
@@ -88,6 +88,10 @@ struct ceph_mds_reply_info_in {s32 dir_pin;struct ceph_timespec btime;struct ceph_timespec snap_btime; +u8 *fscrypt_auth; +u8 *fscrypt_file; +u32 fscrypt_auth_len; +u32 fscrypt_file_len;u64 rsnaps;u64 change_attr; };
@@ -280,6 +284,9 @@ struct ceph_mds_request {struct mutex r_fill_mutex;union ceph_mds_request_args r_args; + +struct ceph_fscrypt_auth *r_fscrypt_auth; +int r_fmode;/* file mode, if expecting cap */ int r_request_release_offset;const struct cred *r_cred;
fs/ceph/super.c +3 -0
@@ -20,6 +20,7 @@#include "super.h" #include "mds_client.h" #include "cache.h" +#include "crypto.h" #include <linux/ceph/ceph_features.h> #include <linux/ceph/decode.h>
@@ -1130,6 +1131,8 @@ static int ceph_set_super(struct super_block *s, struct fs_context *fc) s->s_time_max =U32_MAX ; s->s_flags |=SB_NODIRATIME |SB_NOATIME ; +ceph_fscrypt_set_ops (s); + ret =set_anon_super_fc (s, fc);if (ret !=0 ) fsc->sb =NULL ;
fs/ceph/super.h +13 -1
@@ -434,6 +434,13 @@ struct ceph_inode_info {struct work_struct i_work;unsigned long i_work_mask; + +#ifdef CONFIG_FS_ENCRYPTION +u32 fscrypt_auth_len; +u32 fscrypt_file_len; +u8 *fscrypt_auth; +u8 *fscrypt_file; +#endif }; static inline struct ceph_inode_info *
@@ -1039,7 +1046,12 @@ static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) }extern int ceph_permission (struct user_namespace *mnt_userns,struct inode *inode,int mask); -extern int __ceph_setattr (struct inode *inode,struct iattr *attr); + +struct ceph_iattr { +struct ceph_fscrypt_auth *fscrypt_auth; +}; + +extern int __ceph_setattr (struct inode *inode,struct iattr *attr,struct ceph_iattr *cia);extern int ceph_setattr (struct user_namespace *mnt_userns,struct dentry *dentry,struct iattr *attr); extern int ceph_getattr(struct user_namespace *mnt_userns,
include/linux/ceph/ceph_fs.h +13 -8
@@ -359,14 +359,19 @@ enum {extern const char *ceph_mds_op_name (int op); - -#define CEPH_SETATTR_MODE 1 -#define CEPH_SETATTR_UID 2 -#define CEPH_SETATTR_GID 4 -#define CEPH_SETATTR_MTIME 8 -#define CEPH_SETATTR_ATIME 16 -#define CEPH_SETATTR_SIZE 32 -#define CEPH_SETATTR_CTIME 64 +#define CEPH_SETATTR_MODE (1 << 0) +#define CEPH_SETATTR_UID (1 << 1) +#define CEPH_SETATTR_GID (1 << 2) +#define CEPH_SETATTR_MTIME (1 << 3) +#define CEPH_SETATTR_ATIME (1 << 4) +#define CEPH_SETATTR_SIZE (1 << 5) +#define CEPH_SETATTR_CTIME (1 << 6) +#define CEPH_SETATTR_MTIME_NOW (1 << 7) +#define CEPH_SETATTR_ATIME_NOW (1 << 8) +#define CEPH_SETATTR_BTIME (1 << 9) +#define CEPH_SETATTR_KILL_SGUID (1 << 10) +#define CEPH_SETATTR_FSCRYPT_AUTH (1 << 11) +#define CEPH_SETATTR_FSCRYPT_FILE (1 << 12)/* * Ceph setxattr request flags.
Signature · 1 line
2.35.1
PATCH v14 13/64 ceph: ensure that we accept a new context from MDS for new inodes
In reply to Jeff Layton--- fs/ceph/inode.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-)
fs/ceph/inode.c +11 -10
@@ -944,6 +944,17 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page,__ceph_update_quota (ci, iinfo->max_bytes, iinfo->max_files); +#ifdef CONFIG_FS_ENCRYPTION +if (iinfo->fscrypt_auth_len && (inode->i_state &I_NEW )) { +kfree (ci->fscrypt_auth); + ci->fscrypt_auth_len = iinfo->fscrypt_auth_len; + ci->fscrypt_auth = iinfo->fscrypt_auth; + iinfo->fscrypt_auth =NULL ; + iinfo->fscrypt_auth_len =0 ; +inode_set_flags (inode,S_ENCRYPTED ,S_ENCRYPTED ); + } +#endif +if ((new_version || (new_issued &CEPH_CAP_AUTH_SHARED )) && (issued &CEPH_CAP_AUTH_EXCL ) ==0 ) { inode->i_mode = mode;
@@ -1033,16 +1044,6 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page, xattr_blob =NULL ; } -#ifdef CONFIG_FS_ENCRYPTION -if (iinfo->fscrypt_auth_len && !ci->fscrypt_auth) { - ci->fscrypt_auth_len = iinfo->fscrypt_auth_len; - ci->fscrypt_auth = iinfo->fscrypt_auth; - iinfo->fscrypt_auth =NULL ; - iinfo->fscrypt_auth_len =0 ; -inode_set_flags (inode,S_ENCRYPTED ,S_ENCRYPTED ); - } -#endif -/* finally update i_version */ if (le64_to_cpu (info->version) > ci->i_version) ci->i_version =le64_to_cpu (info->version);
Signature · 1 line
2.35.1
PATCH v14 14/64 ceph: add support for fscrypt_auth/fscrypt_file to cap messages
In reply to Jeff LaytonAdd support for new version 12 cap messages that carry the new fscrypt_auth and fscrypt_file fields from the inode.
--- fs/ceph/caps.c | 76 +++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 63 insertions(+), 13 deletions(-)
fs/ceph/caps.c +63 -13
@@ -13,6 +13,7 @@#include "super.h" #include "mds_client.h" #include "cache.h" +#include "crypto.h" #include <linux/ceph/decode.h> #include <linux/ceph/messenger.h>
@@ -1214,15 +1215,12 @@ struct cap_msg_args {umode_t mode;bool inline_data;bool wake; +u32 fscrypt_auth_len; +u32 fscrypt_file_len; +u8 fscrypt_auth[sizeof (struct ceph_fscrypt_auth )];// for context +u8 fscrypt_file[sizeof (u64)];// for size }; -/* -* cap struct size + flock buffer size + inline version + inline data size + -* osd_epoch_barrier + oldest_flush_tid -*/ -#define CAP_MSG_SIZE (sizeof(struct ceph_mds_caps) + \ - 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4) -/* Marshal up the cap msg to the MDS */ static void encode_cap_msg (struct ceph_msg *msg,struct cap_msg_args *arg) {
@@ -1238,7 +1236,7 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg) arg->size, arg->max_size, arg->xattr_version, arg->xattr_buf ? (int)arg->xattr_buf->vec.iov_len : 0); - msg->hdr.version =cpu_to_le16 (10 ); + msg->hdr.version =cpu_to_le16 (12 ); msg->hdr.tid =cpu_to_le64 (arg->flush_tid); fc = msg->front.iov_base;
@@ -1309,6 +1307,21 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg)/* Advisory flags (version 10) */ ceph_encode_32 (&p, arg->flags); + +/* dirstats (version 11) - these are r/o on the client */ +ceph_encode_64 (&p,0 ); +ceph_encode_64 (&p,0 ); + +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +/* fscrypt_auth and fscrypt_file (version 12) */ +ceph_encode_32 (&p, arg->fscrypt_auth_len); +ceph_encode_copy (&p, arg->fscrypt_auth, arg->fscrypt_auth_len); +ceph_encode_32 (&p, arg->fscrypt_file_len); +ceph_encode_copy (&p, arg->fscrypt_file, arg->fscrypt_file_len); +#else /* CONFIG_FS_ENCRYPTION */ +ceph_encode_32 (&p,0 ); +ceph_encode_32 (&p,0 ); +#endif /* CONFIG_FS_ENCRYPTION */ }/*
@@ -1430,8 +1443,37 @@ static void __prep_cap(struct cap_msg_args *arg, struct ceph_cap *cap, } } arg->flags = flags; +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +if (ci->fscrypt_auth_len && +WARN_ON_ONCE (ci->fscrypt_auth_len >sizeof (struct ceph_fscrypt_auth ))) { +/* Don't set this if it's too big */ + arg->fscrypt_auth_len =0 ; + }else { + arg->fscrypt_auth_len = ci->fscrypt_auth_len; +memcpy (arg->fscrypt_auth, ci->fscrypt_auth, +min_t (size_t, ci->fscrypt_auth_len,sizeof (arg->fscrypt_auth))); + } +/* FIXME: use this to track "real" size */ + arg->fscrypt_file_len =0 ; +#endif /* CONFIG_FS_ENCRYPTION */ } +#define CAP_MSG_FIXED_FIELDS (sizeof(struct ceph_mds_caps) + \ + 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4 + 8 + 8 + 4 + 4) + +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +static inline int cap_msg_size (struct cap_msg_args *arg) +{ +return CAP_MSG_FIXED_FIELDS + arg->fscrypt_auth_len + + arg->fscrypt_file_len; +} +#else +static inline int cap_msg_size (struct cap_msg_args *arg) +{ +return CAP_MSG_FIXED_FIELDS ; +} +#endif /* CONFIG_FS_ENCRYPTION */ +/* * Send a cap msg on the given inode. *
@@ -1442,7 +1484,7 @@ static void __send_cap(struct cap_msg_args *arg, struct ceph_inode_info *ci)struct ceph_msg *msg;struct inode *inode = &ci->vfs_inode; - msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,CAP_MSG_SIZE ,GFP_NOFS , false); + msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,cap_msg_size (arg),GFP_NOFS , false);if (!msg) { pr_err("error allocating cap msg: ino (%llx.%llx) flushing %s tid %llu, requeuing cap.\n", ceph_vinop(inode), ceph_cap_string(arg->dirty),
@@ -1468,10 +1510,6 @@ static inline int __send_flush_snap(struct inode *inode,struct cap_msg_args arg;struct ceph_msg *msg; - msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,CAP_MSG_SIZE ,GFP_NOFS , false); -if (!msg) -return -ENOMEM ; - arg.session = session; arg.ino =ceph_vino (inode).ino; arg.cid =0 ;
@@ -1509,6 +1547,18 @@ static inline int __send_flush_snap(struct inode *inode, arg.flags =0 ; arg.wake = false; +/* +* No fscrypt_auth changes from a capsnap. It will need +* to update fscrypt_file on size changes (TODO). +*/ + arg.fscrypt_auth_len =0 ; + arg.fscrypt_file_len =0 ; + + msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,cap_msg_size (&arg), +GFP_NOFS , false); +if (!msg) +return -ENOMEM ; +encode_cap_msg (msg, &arg);ceph_con_send (&arg.session->s_con, msg);return 0 ;
Signature · 1 line
2.35.1
PATCH v14 15/64 ceph: implement -o test_dummy_encryption mount option
In reply to Jeff LaytonAdd support for the test_dummy_encryption mount option. This allows us to test the encrypted codepaths in ceph without having to manually set keys, etc.
--- fs/ceph/crypto.c | 53 ++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 26 ++++++++++++++++ fs/ceph/inode.c | 10 ++++-- fs/ceph/super.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++-- fs/ceph/super.h | 10 +++++- fs/ceph/xattr.c | 3 ++ 6 files changed, 176 insertions(+), 6 deletions(-)
fs/ceph/crypto.c +53 -0
@@ -4,6 +4,7 @@#include <linux/fscrypt.h> #include "super.h" +#include "mds_client.h" #include "crypto.h" static int ceph_crypt_get_context (struct inode *inode,void *ctx,size_t len)
@@ -64,9 +65,15 @@ static bool ceph_crypt_empty_dir(struct inode *inode)return ci->i_rsubdirs + ci->i_rfiles ==1 ; } +static const union fscrypt_policy *ceph_get_dummy_policy (struct super_block *sb) +{ +return ceph_sb_to_client (sb)->dummy_enc_policy.policy; +} +static struct fscrypt_operations ceph_fscrypt_ops = { .get_context = ceph_crypt_get_context, .set_context = ceph_crypt_set_context, + .get_dummy_policy = ceph_get_dummy_policy, .empty_dir = ceph_crypt_empty_dir, };
@@ -74,3 +81,49 @@ void ceph_fscrypt_set_ops(struct super_block *sb) {fscrypt_set_ops (sb, &ceph_fscrypt_ops); } + +void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc) +{ +fscrypt_free_dummy_policy (&fsc->dummy_enc_policy); +} + +int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as) +{ +int ret, ctxsize; +bool encrypted = false; +struct ceph_inode_info *ci =ceph_inode (inode); + + ret =fscrypt_prepare_new_inode (dir, inode, &encrypted); +if (ret) +return ret; +if (!encrypted) +return 0 ; + + as->fscrypt_auth =kzalloc (sizeof (*as->fscrypt_auth),GFP_KERNEL ); +if (!as->fscrypt_auth) +return -ENOMEM ; + + ctxsize =fscrypt_context_for_new_inode (as->fscrypt_auth->cfa_blob, inode); +if (ctxsize <0 ) +return ctxsize; + + as->fscrypt_auth->cfa_version =cpu_to_le32 (CEPH_FSCRYPT_AUTH_VERSION ); + as->fscrypt_auth->cfa_blob_len =cpu_to_le32 (ctxsize); + +WARN_ON_ONCE (ci->fscrypt_auth); +kfree (ci->fscrypt_auth); + ci->fscrypt_auth_len =ceph_fscrypt_auth_len (as->fscrypt_auth); + ci->fscrypt_auth =kmemdup (as->fscrypt_auth, ci->fscrypt_auth_len,GFP_KERNEL ); +if (!ci->fscrypt_auth) +return -ENOMEM ; + + inode->i_flags |=S_ENCRYPTED ; + +return 0 ; +} + +void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as) +{ +swap (req->r_fscrypt_auth, as->fscrypt_auth); +}
fs/ceph/crypto.h +26 -0
@@ -8,6 +8,10 @@#include <linux/fscrypt.h> +struct ceph_fs_client ; +struct ceph_acl_sec_ctx ; +struct ceph_mds_request ; +struct ceph_fscrypt_auth {__le32 cfa_version;__le32 cfa_blob_len;
@@ -25,12 +29,34 @@ static inline u32 ceph_fscrypt_auth_len(struct ceph_fscrypt_auth *fa)#ifdef CONFIG_FS_ENCRYPTION void ceph_fscrypt_set_ops (struct super_block *sb); +void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc); + +int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as); +void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); +#else /* CONFIG_FS_ENCRYPTION */ static inline void ceph_fscrypt_set_ops (struct super_block *sb) { } +static inline void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc) +{ +} + +static inline int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode, +struct ceph_acl_sec_ctx *as) +{ +if (IS_ENCRYPTED (dir)) +return -EOPNOTSUPP ; +return 0 ; +} + +static inline void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req, +struct ceph_acl_sec_ctx *as_ctx) +{ +}#endif /* CONFIG_FS_ENCRYPTION */ #endif
fs/ceph/inode.c +8 -2
@@ -83,12 +83,17 @@ struct inode *ceph_new_inode(struct inode *dir, struct dentry *dentry, goto out_err; } + inode->i_state =0 ; + inode->i_mode = *mode; + err =ceph_security_init_secctx (dentry, *mode, as_ctx);if (err <0 ) goto out_err; - inode->i_state =0 ; - inode->i_mode = *mode; + err =ceph_fscrypt_prepare_context (dir, inode, as_ctx); +if (err) + goto out_err; +return inode; out_err:iput (inode);
@@ -101,6 +106,7 @@ void ceph_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_sec_ctx *a req->r_pagelist = as_ctx->pagelist; as_ctx->pagelist =NULL ; } +ceph_fscrypt_as_ctx_to_req (req, as_ctx); }/**
fs/ceph/super.c +77 -3
148 lines not on this page · raw message
fs/ceph/super.h +9 -1
@@ -21,6 +21,7 @@#include <linux/fscache.h> #include <linux/ceph/libceph.h> +#include "crypto.h" /* large granularity for statfs utilization stats to facilitate * large volume sizes on 32-bit machines. */
@@ -42,6 +43,7 @@#define CEPH_MOUNT_OPT_ASYNC_DIROPS (1<<15)/* allow async directory ops */ #define CEPH_MOUNT_OPT_NOPAGECACHE (1<<16)/* bypass pagecache altogether */ #define CEPH_MOUNT_OPT_SPARSEREAD (1<<17)/* always do sparse reads */ +#define CEPH_MOUNT_OPT_TEST_DUMMY_ENC (1<<18)/* enable dummy encryption (for testing) */ #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \
@@ -98,6 +100,7 @@ struct ceph_mount_options {char *server_path;/* default NULL (means "/") */ char *fscache_uniq;/* default NULL */ char *mon_addr; +char *test_dummy_encryption;/* default NULL */ };struct ceph_fs_client {
@@ -138,9 +141,11 @@ struct ceph_fs_client {#ifdef CONFIG_CEPH_FSCACHE struct fscache_volume *fscache;#endif +#ifdef CONFIG_FS_ENCRYPTION +struct fscrypt_dummy_policy dummy_enc_policy; +#endif }; -/* * File i/o capability. This tracks shared state with the metadata * server that allows us to cache or writeback attributes or to read
@@ -1085,6 +1090,9 @@ struct ceph_acl_sec_ctx {#ifdef CONFIG_CEPH_FS_SECURITY_LABEL void *sec_ctx;u32 sec_ctxlen; +#endif +#ifdef CONFIG_FS_ENCRYPTION +struct ceph_fscrypt_auth *fscrypt_auth;#endif struct ceph_pagelist *pagelist; };
fs/ceph/xattr.c +3 -0
@@ -1397,6 +1397,9 @@ void ceph_release_acl_sec_ctx(struct ceph_acl_sec_ctx *as_ctx)#endif #ifdef CONFIG_CEPH_FS_SECURITY_LABEL security_release_secctx (as_ctx->sec_ctx, as_ctx->sec_ctxlen); +#endif +#ifdef CONFIG_FS_ENCRYPTION +kfree (as_ctx->fscrypt_auth);#endif if (as_ctx->pagelist)ceph_pagelist_release (as_ctx->pagelist);
Signature · 1 line
2.35.1
PATCH v14 16/64 ceph: decode alternate_name in lease info
In reply to Jeff LaytonCeph is a bit different from local filesystems, in that we don't want to store filenames as raw binary data, since we may also be dealing with clients that don't support fscrypt.
We could just base64-encode the encrypted filenames, but that could leave us with filenames longer than NAME_MAX. It turns out that the MDS doesn't care much about filename length, but the clients do.
To manage this, we've added a new "alternate name" field that can be optionally added to any dentry that we'll use to store the binary crypttext of the filename if its base64-encoded value will be longer than NAME_MAX. When a dentry has one of these names attached, the MDS will send it along in the lease info, which we can then store for later usage.
--- fs/ceph/mds_client.c | 43 +++++++++++++++++++++++++++++++++---------- fs/ceph/mds_client.h | 11 +++++++---- 2 files changed, 40 insertions(+), 14 deletions(-)
fs/ceph/mds_client.c +33 -10
@@ -308,27 +308,47 @@ static int parse_reply_info_dir(void **p, void *end,static int parse_reply_info_lease (void **p,void *end,struct ceph_mds_reply_lease **lease, -u64 features) +u64 features,u32 *altname_len,u8 **altname) { +u8 struct_v; +u32 struct_len; +void *lend; +if (features == (u64)-1 ) { -u8 struct_v, struct_compat; -u32 struct_len; +u8 struct_compat; +ceph_decode_8_safe (p, end, struct_v, bad);ceph_decode_8_safe (p, end, struct_compat, bad); +/* struct_v is expected to be >= 1. we only understand * encoding whose struct_compat == 1. */ if (!struct_v || struct_compat !=1 ) goto bad; +ceph_decode_32_safe (p, end, struct_len, bad); -ceph_decode_need (p, end, struct_len, bad); - end = *p + struct_len; + }else { + struct_len =sizeof (**lease); + *altname_len =0 ; + *altname =NULL ; } -ceph_decode_need (p, end,sizeof (**lease), bad); + lend = *p + struct_len; +ceph_decode_need (p, end, struct_len, bad); *lease = *p; *p +=sizeof (**lease); -if (features == (u64)-1 ) - *p = end; + +if (features == (u64)-1 ) { +if (struct_v >=2 ) { +ceph_decode_32_safe (p, end, *altname_len, bad); +ceph_decode_need (p, end, *altname_len, bad); + *altname = *p; + *p += *altname_len; + }else { + *altname =NULL ; + *altname_len =0 ; + } + } + *p = lend;return 0 ; bad:return -EIO ;
@@ -358,7 +378,8 @@ static int parse_reply_info_trace(void **p, void *end, info->dname = *p; *p += info->dname_len; - err =parse_reply_info_lease (p, end, &info->dlease, features); + err =parse_reply_info_lease (p, end, &info->dlease, features, + &info->altname_len, &info->altname);if (err <0 ) goto out_bad; }
@@ -425,9 +446,11 @@ static int parse_reply_info_readdir(void **p, void *end,dout ("parsed dir dname '%.*s'\n" , rde->name_len, rde->name);/* dentry lease */ - err =parse_reply_info_lease (p, end, &rde->lease, features); + err =parse_reply_info_lease (p, end, &rde->lease, features, + &rde->altname_len, &rde->altname);if (err) goto out_bad; +/* inode */ err =parse_reply_info_in (p, end, &rde->inode, features);if (err <0 )
fs/ceph/mds_client.h +7 -4
@@ -29,8 +29,8 @@ enum ceph_feature_type {CEPHFS_FEATURE_MULTI_RECONNECT ,CEPHFS_FEATURE_DELEG_INO ,CEPHFS_FEATURE_METRIC_COLLECT , - - CEPHFS_FEATURE_MAX = CEPHFS_FEATURE_METRIC_COLLECT, +CEPHFS_FEATURE_ALTERNATE_NAME , + CEPHFS_FEATURE_MAX = CEPHFS_FEATURE_ALTERNATE_NAME, };/*
@@ -45,8 +45,7 @@ enum ceph_feature_type {CEPHFS_FEATURE_MULTI_RECONNECT , \CEPHFS_FEATURE_DELEG_INO , \CEPHFS_FEATURE_METRIC_COLLECT , \ - \ -CEPHFS_FEATURE_MAX , \ +CEPHFS_FEATURE_ALTERNATE_NAME , \ } #define CEPHFS_FEATURES_CLIENT_REQUIRED {}
@@ -98,7 +97,9 @@ struct ceph_mds_reply_info_in {struct ceph_mds_reply_dir_entry {char *name; +u8 *altname;u32 name_len; +u32 altname_len;struct ceph_mds_reply_lease *lease;struct ceph_mds_reply_info_in inode;loff_t offset;
@@ -122,7 +123,9 @@ struct ceph_mds_reply_info_parsed {struct ceph_mds_reply_info_in diri, targeti;struct ceph_mds_reply_dirfrag *dirfrag;char *dname; +u8 *altname;u32 dname_len; +u32 altname_len;struct ceph_mds_reply_lease *dlease;struct ceph_mds_reply_xattr xattr_info;
Signature · 1 line
2.35.1
PATCH v14 17/64 ceph: add fscrypt ioctls
In reply to Jeff LaytonWe gate most of the ioctls on MDS feature support. The exception is the key removal and status functions that we still want to work if the MDS's were to (inexplicably) lose the feature.
For the set_policy ioctl, we take Fs caps to ensure that nothing can create files in the directory while the ioctl is running. That should be enough to ensure that the "empty_dir" check is reliable.
--- fs/ceph/ioctl.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
fs/ceph/ioctl.c +83 -0
@@ -6,6 +6,7 @@#include "mds_client.h" #include "ioctl.h" #include <linux/ceph/striper.h> +#include <linux/fscrypt.h> /* * ioctls
@@ -268,8 +269,54 @@ static long ceph_ioctl_syncio(struct file *file)return 0 ; } +static int vet_mds_for_fscrypt (struct file *file) +{ +int i, ret = -EOPNOTSUPP ; +struct ceph_mds_client *mdsc =ceph_sb_to_mdsc (file_inode (file)->i_sb); + +mutex_lock (&mdsc->mutex); +for (i =0 ; i < mdsc->max_sessions; i++) { +struct ceph_mds_session *s = mdsc->sessions[i]; + +if (!s) +continue ; +if (test_bit (CEPHFS_FEATURE_ALTERNATE_NAME , &s->s_features)) + ret =0 ; +break ; + } +mutex_unlock (&mdsc->mutex); +return ret; +} + +static long ceph_set_encryption_policy (struct file *file,unsigned long arg) +{ +int ret, got =0 ; +struct inode *inode =file_inode (file); +struct ceph_inode_info *ci =ceph_inode (inode); + + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; + +/* +* Ensure we hold these caps so that we _know_ that the rstats check +* in the empty_dir check is reliable. +*/ + ret =ceph_get_caps (file,CEPH_CAP_FILE_SHARED ,0 ,-1 , &got); +if (ret) +return ret; + + ret =fscrypt_ioctl_set_policy (file, (const void __user *)arg); +if (got) +ceph_put_cap_refs (ci, got); + +return ret; +} +long ceph_ioctl (struct file *file,unsigned int cmd,unsigned long arg) { +int ret; +dout ("ioctl file %p cmd %u arg %lu\n" , file, cmd, arg);switch (cmd) {case CEPH_IOC_GET_LAYOUT :
@@ -289,6 +336,42 @@ long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg)case CEPH_IOC_SYNCIO :return ceph_ioctl_syncio (file); + +case FS_IOC_SET_ENCRYPTION_POLICY : +return ceph_set_encryption_policy (file, arg); + +case FS_IOC_GET_ENCRYPTION_POLICY : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_policy (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_POLICY_EX : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_policy_ex (file, (void __user *)arg); + +case FS_IOC_ADD_ENCRYPTION_KEY : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_add_key (file, (void __user *)arg); + +case FS_IOC_REMOVE_ENCRYPTION_KEY : +return fscrypt_ioctl_remove_key (file, (void __user *)arg); + +case FS_IOC_REMOVE_ENCRYPTION_KEY_ALL_USERS : +return fscrypt_ioctl_remove_key_all_users (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_KEY_STATUS : +return fscrypt_ioctl_get_key_status (file, (void __user *)arg); + +case FS_IOC_GET_ENCRYPTION_NONCE : + ret =vet_mds_for_fscrypt (file); +if (ret) +return ret; +return fscrypt_ioctl_get_nonce (file, (void __user *)arg); }return -ENOTTY ;
Signature · 1 line
2.35.1
PATCH v14 18/64 ceph: make the ioctl cmd more readable in debug log
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
ioctl file 0000000004e6b054 cmd 2148296211 arg 824635143532
The numerical cmd valye in the ioctl debug log message is too hard to understand even when you look at it in the code. Make it more readable.
--- fs/ceph/ioctl.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)
fs/ceph/ioctl.c +38 -1
@@ -313,11 +313,48 @@ static long ceph_set_encryption_policy(struct file *file, unsigned long arg)return ret; } +static const char *ceph_ioctl_cmd_name (const unsigned int cmd) +{ +switch (cmd) { +case CEPH_IOC_GET_LAYOUT : +return "get_layout" ; +case CEPH_IOC_SET_LAYOUT : +return "set_layout" ; +case CEPH_IOC_SET_LAYOUT_POLICY : +return "set_layout_policy" ; +case CEPH_IOC_GET_DATALOC : +return "get_dataloc" ; +case CEPH_IOC_LAZYIO : +return "lazyio" ; +case CEPH_IOC_SYNCIO : +return "syncio" ; +case FS_IOC_SET_ENCRYPTION_POLICY : +return "set encryption_policy" ; +case FS_IOC_GET_ENCRYPTION_POLICY : +return "get_encryption_policy" ; +case FS_IOC_GET_ENCRYPTION_POLICY_EX : +return "get_encryption_policy_ex" ; +case FS_IOC_ADD_ENCRYPTION_KEY : +return "add_encryption_key" ; +case FS_IOC_REMOVE_ENCRYPTION_KEY : +return "remove_encryption_key" ; +case FS_IOC_REMOVE_ENCRYPTION_KEY_ALL_USERS : +return "remove_encryption_key_all_users" ; +case FS_IOC_GET_ENCRYPTION_KEY_STATUS : +return "get_encryption_key_status" ; +case FS_IOC_GET_ENCRYPTION_NONCE : +return "get_encryption_nonce" ; +default : +return "unknown" ; + } +} +long ceph_ioctl (struct file *file,unsigned int cmd,unsigned long arg) {int ret; -dout ("ioctl file %p cmd %u arg %lu\n" , file, cmd, arg); +dout ("ioctl file %p cmd %s arg %lu\n" , file, +ceph_ioctl_cmd_name (cmd), arg);switch (cmd) {case CEPH_IOC_GET_LAYOUT :return ceph_ioctl_get_layout (file, (void __user *)arg);
Signature · 1 line
2.35.1
PATCH v14 19/64 ceph: make ceph_msdc_build_path use ref-walk
In reply to Jeff LaytonEncryption potentially requires allocation, at which point we'll need to be in a non-atomic context. Convert ceph_msdc_build_path to take dentry spinlocks and references instead of using rcu_read_lock to walk the path.
This is slightly less efficient, and we may want to eventually allow using RCU when the leaf dentry isn't encrypted.
--- fs/ceph/mds_client.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)
fs/ceph/mds_client.c +19 -16
@@ -2398,7 +2398,8 @@ static inline u64 __get_oldest_tid(struct ceph_mds_client *mdsc)char *ceph_mdsc_build_path (struct dentry *dentry,int *plen,u64 *pbase,int stop_on_nosnap) { -struct dentry *temp; +struct dentry *cur; +struct inode *inode;char *path;int pos;unsigned seq;
@@ -2415,34 +2416,35 @@ char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *pbase, path[pos] ='\0' ; seq =read_seqbegin (&rename_lock); -rcu_read_lock (); - temp = dentry; + cur =dget (dentry);for (;;) { -struct inode *inode; +struct dentry *temp; -spin_lock (&temp->d_lock); - inode =d_inode (temp); +spin_lock (&cur->d_lock); + inode =d_inode (cur);if (inode &&ceph_snap (inode) ==CEPH_SNAPDIR ) {dout ("build_path path+%d: %p SNAPDIR\n" , - pos, temp); - }else if (stop_on_nosnap && inode && dentry != temp && + pos, cur); + }else if (stop_on_nosnap && inode && dentry != cur &&ceph_snap (inode) ==CEPH_NOSNAP ) { -spin_unlock (&temp->d_lock); +spin_unlock (&cur->d_lock); pos++;/* get rid of any prepended '/' */ break ; }else { - pos -= temp->d_name.len; + pos -= cur->d_name.len;if (pos <0 ) { -spin_unlock (&temp->d_lock); +spin_unlock (&cur->d_lock);break ; } -memcpy (path + pos, temp->d_name.name, temp->d_name.len); +memcpy (path + pos, cur->d_name.name, cur->d_name.len); } + temp = cur;spin_unlock (&temp->d_lock); - temp =READ_ONCE (temp->d_parent); + cur =dget_parent (temp); +dput (temp);/* Are we at the root? */ -if (IS_ROOT (temp)) +if (IS_ROOT (cur))break ;/* Are we out of buffer? */
@@ -2451,8 +2453,9 @@ char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *pbase, path[pos] ='/' ; } - base =ceph_ino (d_inode (temp)); -rcu_read_unlock (); + inode =d_inode (cur); + base = inode ?ceph_ino (inode) :0 ; +dput (cur);if (read_seqretry (&rename_lock, seq)) goto retry;
Signature · 1 line
2.35.1
PATCH v14 20/64 ceph: add base64 endcoding routines for encrypted names
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
The base64url encoding used by fscrypt includes the '_' character, which may cause problems in snapshot names (if the name starts with '_'). Thus, use the base64 encoding defined for IMAP mailbox names (RFC 3501), which uses '+' and ',' instead of '-' and '_'.
--- fs/ceph/crypto.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 32 ++++++++++++++++++++++++++ 2 files changed, 92 insertions(+)
fs/ceph/crypto.c +60 -0
@@ -1,4 +1,11 @@// SPDX-License-Identifier: GPL-2.0 +/* +* The base64 encode/decode code was copied from fscrypt: +* Copyright (C) 2015, Google, Inc. +* Copyright (C) 2015, Motorola Mobility +* Written by Uday Savagaonkar, 2014. +* Modified by Jaegeuk Kim, 2015. +*/ #include <linux/ceph/ceph_debug.h> #include <linux/xattr.h> #include <linux/fscrypt.h>
@@ -7,6 +14,59 @@#include "mds_client.h" #include "crypto.h" +/* +* The base64url encoding used by fscrypt includes the '_' character, which may +* cause problems in snapshot names (which can not starts with '_'). Thus, we +* used the base64 encoding defined for IMAP mailbox names (RFC 3501) instead, +* which replaces '-' and '_' by '+' and ','. +*/ +static const char base64_table[65 ] = +"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+," ; + +int ceph_base64_encode (const u8 *src,int srclen,char *dst) +{ +u32 ac =0 ; +int bits =0 ; +int i; +char *cp = dst; + +for (i =0 ; i < srclen; i++) { + ac = (ac <<8 ) | src[i]; + bits +=8 ; +do { + bits -=6 ; + *cp++ = base64_table[(ac >> bits) &0x3f ]; + }while (bits >=6 ); + } +if (bits) + *cp++ = base64_table[(ac << (6 - bits)) &0x3f ]; +return cp - dst; +} + +int ceph_base64_decode (const char *src,int srclen,u8 *dst) +{ +u32 ac =0 ; +int bits =0 ; +int i; +u8 *bp = dst; + +for (i =0 ; i < srclen; i++) { +const char *p =strchr (base64_table, src[i]); + +if (p ==NULL || src[i] ==0 ) +return -1 ; + ac = (ac <<6 ) | (p - base64_table); + bits +=6 ; +if (bits >=8 ) { + bits -=8 ; + *bp++ = (u8)(ac >> bits); + } + } +if (ac & ((1 << bits) -1 )) +return -1 ; +return bp - dst; +} +static int ceph_crypt_get_context (struct inode *inode,void *ctx,size_t len) {struct ceph_inode_info *ci =ceph_inode (inode);
fs/ceph/crypto.h +32 -0
@@ -27,6 +27,38 @@ static inline u32 ceph_fscrypt_auth_len(struct ceph_fscrypt_auth *fa) }#ifdef CONFIG_FS_ENCRYPTION +/* +* We want to encrypt filenames when creating them, but the encrypted +* versions of those names may have illegal characters in them. To mitigate +* that, we base64 encode them, but that gives us a result that can exceed +* NAME_MAX. +* +* Follow a similar scheme to fscrypt itself, and cap the filename to a +* smaller size. If the ciphertext name is longer than the value below, then +* sha256 hash the remaining bytes. +* +* For the fscrypt_nokey_name struct the dirhash[2] member is useless in ceph +* so the corresponding struct will be: +* +* struct fscrypt_ceph_nokey_name { +* u8 bytes[157]; +* u8 sha256[SHA256_DIGEST_SIZE]; +* }; // 180 bytes => 240 bytes base64-encoded, which is <= NAME_MAX (255) +* +* (240 bytes is the maximum size allowed for snapshot names to take into +* account the format: '_<SNAPSHOT-NAME>_<INODE-NUMBER>'.) +* +* Note that for long names that end up having their tail portion hashed, we +* must also store the full encrypted name (in the dentry's alternate_name +* field). +*/ +#define CEPH_NOHASH_NAME_MAX (180 - SHA256_DIGEST_SIZE) + +#define CEPH_BASE64_CHARS (nbytes) DIV_ROUND_UP((nbytes) * 4, 3) + +int ceph_base64_encode (const u8 *src,int srclen,char *dst); +int ceph_base64_decode (const char *src,int srclen,u8 *dst); +void ceph_fscrypt_set_ops (struct super_block *sb);void ceph_fscrypt_free_dummy_policy (struct ceph_fs_client *fsc);
Signature · 1 line
2.35.1
PATCH v14 21/64 ceph: add encrypted fname handling to ceph_mdsc_build_path
In reply to Jeff LaytonAllow ceph_mdsc_build_path to encrypt and base64 encode the filename when the parent is encrypted and we're sending the path to the MDS.
In most cases, we just encrypt the filenames and base64 encode them, but when the name is longer than CEPH_NOHASH_NAME_MAX, we use a similar scheme to fscrypt proper, and hash the remaning bits with sha256.
When doing this, we then send along the full crypttext of the name in the new alternate_name field of the MClientRequest. The MDS can then send that along in readdir responses and traces.
--- fs/ceph/crypto.c | 47 ++++++++++++++++++++++++++ fs/ceph/crypto.h | 8 +++++ fs/ceph/mds_client.c | 80 ++++++++++++++++++++++++++++++++++---------- 3 files changed, 117 insertions(+), 18 deletions(-)
fs/ceph/crypto.c +47 -0
@@ -187,3 +187,50 @@ void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_se {swap (req->r_fscrypt_auth, as->fscrypt_auth); } + +int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) +{ +u32 len; +int elen; +int ret; +u8 *cryptbuf; + +WARN_ON_ONCE (!fscrypt_has_encryption_key (parent)); + +/* +* Convert cleartext d_name to ciphertext. If result is longer than +* CEPH_NOHASH_NAME_MAX, sha256 the remaining bytes +* +* See: fscrypt_setup_filename +*/ +if (!fscrypt_fname_encrypted_size (parent, dentry->d_name.len,NAME_MAX , &len)) +return -ENAMETOOLONG ; + +/* Allocate a buffer appropriate to hold the result */ + cryptbuf =kmalloc (len >CEPH_NOHASH_NAME_MAX ?NAME_MAX : len,GFP_KERNEL ); +if (!cryptbuf) +return -ENOMEM ; + + ret =fscrypt_fname_encrypt (parent, &dentry->d_name, cryptbuf, len); +if (ret) { +kfree (cryptbuf); +return ret; + } + +/* hash the end if the name is long enough */ +if (len >CEPH_NOHASH_NAME_MAX ) { +u8 hash[SHA256_DIGEST_SIZE ]; +u8 *extra = cryptbuf +CEPH_NOHASH_NAME_MAX ; + +/* hash the extra bytes and overwrite crypttext beyond that point with it */ +sha256 (extra, len -CEPH_NOHASH_NAME_MAX , hash); +memcpy (extra, hash,SHA256_DIGEST_SIZE ); + len =CEPH_NOHASH_NAME_MAX +SHA256_DIGEST_SIZE ; + } + +/* base64 encode the encrypted name */ + elen =ceph_base64_encode (cryptbuf, len, buf); +kfree (cryptbuf); +dout ("base64-encoded ciphertext name = %.*s\n" , elen, buf); +return elen; +}
fs/ceph/crypto.h +8 -0
@@ -6,6 +6,7 @@#ifndef _CEPH_CRYPTO_H#define _CEPH_CRYPTO_H +#include <crypto/sha2.h> #include <linux/fscrypt.h> struct ceph_fs_client ;
@@ -66,6 +67,7 @@ void ceph_fscrypt_free_dummy_policy(struct ceph_fs_client *fsc);int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode,struct ceph_acl_sec_ctx *as);void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); +int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf);#else /* CONFIG_FS_ENCRYPTION */
@@ -89,6 +91,12 @@ static inline void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_sec_ctx *as_ctx) { } + +static inline int ceph_encode_encrypted_fname (const struct inode *parent, +struct dentry *dentry,char *buf) +{ +return -EOPNOTSUPP ; +}#endif /* CONFIG_FS_ENCRYPTION */ #endif
fs/ceph/mds_client.c +62 -18
131 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 22/64 ceph: send altname in MClientRequest
In reply to Jeff LaytonIn the event that we have a filename longer than CEPH_NOHASH_NAME_MAX, we'll need to hash the tail of the filename. The client however will still need to know the full name of the file if it has a key.
To support this, the MClientRequest field has grown a new alternate_name field that we populate with the full (binary) crypttext of the filename. This is then transmitted to the clients in readdir or traces as part of the dentry lease.
Add support for populating this field when the filenames are very long.
--- fs/ceph/mds_client.c | 75 +++++++++++++++++++++++++++++++++++++++++--- fs/ceph/mds_client.h | 3 ++ 2 files changed, 73 insertions(+), 5 deletions(-)
fs/ceph/mds_client.c +70 -5
@@ -972,6 +972,7 @@ void ceph_mdsc_release_request(struct kref *kref)if (req->r_pagelist)ceph_pagelist_release (req->r_pagelist);kfree (req->r_fscrypt_auth); +kfree (req->r_altname);put_request_session (req);ceph_unreserve_caps (req->r_mdsc, &req->r_caps_reservation);WARN_ON_ONCE (!list_empty (&req->r_wait));
@@ -2386,6 +2387,63 @@ static inline u64 __get_oldest_tid(struct ceph_mds_client *mdsc)return mdsc->oldest_tid; } +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +static u8 *get_fscrypt_altname (const struct ceph_mds_request *req,u32 *plen) +{ +struct inode *dir = req->r_parent; +struct dentry *dentry = req->r_dentry; +u8 *cryptbuf =NULL ; +u32 len =0 ; +int ret =0 ; + +/* only encode if we have parent and dentry */ +if (!dir || !dentry) + goto success; + +/* No-op unless this is encrypted */ +if (!IS_ENCRYPTED (dir)) + goto success; + + ret =__fscrypt_prepare_readdir (dir); +if (ret) +return ERR_PTR (ret); + +/* No key? Just ignore it. */ +if (!fscrypt_has_encryption_key (dir)) + goto success; + +if (!fscrypt_fname_encrypted_size (dir, dentry->d_name.len,NAME_MAX , &len)) { +WARN_ON_ONCE (1 ); +return ERR_PTR (-ENAMETOOLONG ); + } + +/* No need to append altname if name is short enough */ +if (len <=CEPH_NOHASH_NAME_MAX ) { + len =0 ; + goto success; + } + + cryptbuf =kmalloc (len,GFP_KERNEL ); +if (!cryptbuf) +return ERR_PTR (-ENOMEM ); + + ret =fscrypt_fname_encrypt (dir, &dentry->d_name, cryptbuf, len); +if (ret) { +kfree (cryptbuf); +return ERR_PTR (ret); + } +success: + *plen = len; +return cryptbuf; +} +#else +static u8 *get_fscrypt_altname (const struct ceph_mds_request *req,u32 *plen) +{ + *plen =0 ; +return NULL ; +} +#endif +/** * ceph_mdsc_build_path - build a path string to a given dentry * @dentry: dentry to which path should be built
@@ -2606,14 +2664,15 @@ static void encode_mclientrequest_tail(void **p, const struct ceph_mds_request *ceph_encode_timespec64 (&ts, &req->r_stamp);ceph_encode_copy (p, &ts,sizeof (ts)); -/* gid_list */ +/* v4: gid_list */ ceph_encode_32 (p, req->r_cred->group_info->ngroups);for (i =0 ; i < req->r_cred->group_info->ngroups; i++)ceph_encode_64 (p,from_kgid (&init_user_ns, req->r_cred->group_info->gid[i])); -/* v5: altname (TODO: skip for now) */ -ceph_encode_32 (p,0 ); +/* v5: altname */ +ceph_encode_32 (p, req->r_altname_len); +ceph_encode_copy (p, req->r_altname, req->r_altname_len);/* v6: fscrypt_auth and fscrypt_file */ if (req->r_fscrypt_auth) {
@@ -2669,7 +2728,13 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, goto out_free1; } -/* head */ + req->r_altname =get_fscrypt_altname (req, &req->r_altname_len); +if (IS_ERR (req->r_altname)) { + msg =ERR_CAST (req->r_altname); + req->r_altname =NULL ; + goto out_free2; + } + len = legacy ?sizeof (*head) :sizeof (struct ceph_mds_request_head );/* filepaths */
@@ -2695,7 +2760,7 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, len +=sizeof (u32) + (sizeof (u64) * req->r_cred->group_info->ngroups);/* alternate name */ - len +=sizeof (u32);// TODO + len +=sizeof (u32) + req->r_altname_len;/* fscrypt_auth */ len +=sizeof (u32);// fscrypt_auth
fs/ceph/mds_client.h +3 -0
@@ -290,6 +290,9 @@ struct ceph_mds_request {struct ceph_fscrypt_auth *r_fscrypt_auth; +u8 *r_altname;/* fscrypt binary crypttext for long filenames */ +u32 r_altname_len;/* length of r_altname */ +int r_fmode;/* file mode, if expecting cap */ int r_request_release_offset;const struct cred *r_cred;
Signature · 1 line
2.35.1
PATCH v14 23/64 ceph: encode encrypted name in dentry release
In reply to Jeff LaytonEncode encrypted dentry names when sending a dentry release request. Also add a more helpful comment over ceph_encode_dentry_release.
--- fs/ceph/caps.c | 32 ++++++++++++++++++++++++++++---- fs/ceph/mds_client.c | 20 ++++++++++++++++---- 2 files changed, 44 insertions(+), 8 deletions(-)
fs/ceph/caps.c +28 -4
@@ -4622,6 +4622,18 @@ int ceph_encode_inode_release(void **p, struct inode *inode,return ret; } +/** +* ceph_encode_dentry_release - encode a dentry release into an outgoing request +* @p: outgoing request buffer +* @dentry: dentry to release +* @dir: dir to release it from +* @mds: mds that we're speaking to +* @drop: caps being dropped +* @unless: unless we have these caps +* +* Encode a dentry release into an outgoing request buffer. Returns 1 if the +* thing was released, or a negative error code otherwise. +*/ int ceph_encode_dentry_release (void **p,struct dentry *dentry,struct inode *dir,int mds,int drop,int unless)
@@ -4654,13 +4666,25 @@ int ceph_encode_dentry_release(void **p, struct dentry *dentry,if (ret && di->lease_session && di->lease_session->s_mds == mds) {dout ("encode_dentry_release %p mds%d seq %d\n" , dentry, mds, (int )di->lease_seq); - rel->dname_len =cpu_to_le32 (dentry->d_name.len); -memcpy (*p, dentry->d_name.name, dentry->d_name.len); - *p += dentry->d_name.len; rel->dname_seq =cpu_to_le32 (di->lease_seq);__ceph_mdsc_drop_dentry_lease (dentry); +spin_unlock (&dentry->d_lock); +if (IS_ENCRYPTED (dir) &&fscrypt_has_encryption_key (dir)) { +int ret2 =ceph_encode_encrypted_fname (dir, dentry, *p); + +if (ret2 <0 ) +return ret2; + + rel->dname_len =cpu_to_le32 (ret2); + *p += ret2; + }else { + rel->dname_len =cpu_to_le32 (dentry->d_name.len); +memcpy (*p, dentry->d_name.name, dentry->d_name.len); + *p += dentry->d_name.len; + } + }else { +spin_unlock (&dentry->d_lock); } -spin_unlock (&dentry->d_lock);return ret; }
fs/ceph/mds_client.c +16 -4
@@ -2819,15 +2819,23 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session, req->r_inode ? req->r_inode :d_inode (req->r_dentry), mds, req->r_inode_drop, req->r_inode_unless, req->r_op == CEPH_MDS_OP_READDIR); -if (req->r_dentry_drop) - releases +=ceph_encode_dentry_release (&p, req->r_dentry, +if (req->r_dentry_drop) { + ret =ceph_encode_dentry_release (&p, req->r_dentry, req->r_parent, mds, req->r_dentry_drop, req->r_dentry_unless); -if (req->r_old_dentry_drop) - releases +=ceph_encode_dentry_release (&p, req->r_old_dentry, +if (ret <0 ) + goto out_err; + releases += ret; + } +if (req->r_old_dentry_drop) { + ret =ceph_encode_dentry_release (&p, req->r_old_dentry, req->r_old_dentry_dir, mds, req->r_old_dentry_drop, req->r_old_dentry_unless); +if (ret <0 ) + goto out_err; + releases += ret; + }if (req->r_old_inode_drop) releases += ceph_encode_inode_release(&p, d_inode(req->r_old_dentry),
@@ -2869,6 +2877,10 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,ceph_mdsc_free_path ((char *)path1, pathlen1); out:return msg; +out_err: +ceph_msg_put (msg); + msg =ERR_PTR (ret); + goto out_free2; }/*
Signature · 1 line
2.35.1
PATCH v14 24/64 ceph: properly set DCACHE_NOKEY_NAME flag in lookup
In reply to Jeff LaytonThis is required so that we know to invalidate these dentries when the directory is unlocked.
--- fs/ceph/dir.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
fs/ceph/dir.c +11 -0
@@ -760,6 +760,17 @@ static struct dentry *ceph_lookup(struct inode *dir, struct dentry *dentry,if (dentry->d_name.len >NAME_MAX )return ERR_PTR (-ENAMETOOLONG ); +if (IS_ENCRYPTED (dir)) { + err =__fscrypt_prepare_readdir (dir); +if (err) +return ERR_PTR (err); +if (!fscrypt_has_encryption_key (dir)) { +spin_lock (&dentry->d_lock); + dentry->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dentry->d_lock); + } + } +/* can we conclude ENOENT locally? */ if (d_really_is_negative (dentry)) {struct ceph_inode_info *ci =ceph_inode (dir);
Signature · 1 line
2.35.1
PATCH v14 25/64 ceph: set DCACHE_NOKEY_NAME in atomic open
In reply to Jeff LaytonAtomic open can act as a lookup if handed a dentry that is negative on the MDS. Ensure that we set DCACHE_NOKEY_NAME on the dentry in atomic_open, if we don't have the key for the parent. Otherwise, we can end up validating the dentry inappropriately if someone later adds a key.
--- fs/ceph/file.c | 7 +++++++ 1 file changed, 7 insertions(+)
fs/ceph/file.c +7 -0
@@ -768,6 +768,13 @@ int ceph_atomic_open(struct inode *dir, struct dentry *dentry, req->r_args.open.mask =cpu_to_le32 (mask); req->r_parent = dir;ihold (dir); +if (IS_ENCRYPTED (dir)) { +if (!fscrypt_has_encryption_key (dir)) { +spin_lock (&dentry->d_lock); + dentry->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dentry->d_lock); + } + }if (flags &O_CREAT ) {struct ceph_file_layout lo;
Signature · 1 line
2.35.1
PATCH v14 26/64 ceph: make d_revalidate call fscrypt revalidator for encrypted dentries
In reply to Jeff LaytonIf we have a dentry which represents a no-key name, then we need to test whether the parent directory's encryption key has since been added. Do that before we test anything else about the dentry.
--- fs/ceph/dir.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
fs/ceph/dir.c +6 -2
@@ -1709,6 +1709,10 @@ static int ceph_d_revalidate(struct dentry *dentry, unsigned int flags)struct inode *dir, *inode;struct ceph_mds_client *mdsc; + valid =fscrypt_d_revalidate (dentry, flags); +if (valid <=0 ) +return valid; +if (flags &LOOKUP_RCU ) { parent =READ_ONCE (dentry->d_parent); dir =d_inode_rcu (parent);
@@ -1721,8 +1725,8 @@ static int ceph_d_revalidate(struct dentry *dentry, unsigned int flags) inode =d_inode (dentry); } -dout ("d_revalidate %p '%pd' inode %p offset 0x%llx\n" , dentry, - dentry, inode,ceph_dentry (dentry)->offset); +dout ("d_revalidate %p '%pd' inode %p offset 0x%llx nokey %d\n" , dentry, + dentry, inode,ceph_dentry (dentry)->offset, !!(dentry->d_flags &DCACHE_NOKEY_NAME )); mdsc =ceph_sb_to_client (dir->i_sb)->mdsc;
Signature · 1 line
2.35.1
PATCH v14 27/64 ceph: add helpers for converting names for userland presentation
In reply to Jeff LaytonDefine a new ceph_fname struct that we can use to carry information about encrypted dentry names. Add helpers for working with these objects, including ceph_fname_to_usr which formats an encrypted filename for userland presentation.
--- fs/ceph/crypto.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 41 ++++++++++++++++++++++++++ 2 files changed, 117 insertions(+)
fs/ceph/crypto.c +76 -0
@@ -234,3 +234,79 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentrdout ("base64-encoded ciphertext name = %.*s\n" , elen, buf);return elen; } + +/** +* ceph_fname_to_usr - convert a filename for userland presentation +* @fname: ceph_fname to be converted +* @tname: temporary name buffer to use for conversion (may be NULL) +* @oname: where converted name should be placed +* @is_nokey: set to true if key wasn't available during conversion (may be NULL) +* +* Given a filename (usually from the MDS), format it for presentation to +* userland. If @parent is not encrypted, just pass it back as-is. +* +* Otherwise, base64 decode the string, and then ask fscrypt to format it +* for userland presentation. +* +* Returns 0 on success or negative error code on error. +*/ +int ceph_fname_to_usr (const struct ceph_fname *fname,struct fscrypt_str *tname, +struct fscrypt_str *oname,bool *is_nokey) +{ +int ret; +struct fscrypt_str _tname =FSTR_INIT (NULL ,0 ); +struct fscrypt_str iname; + +if (!IS_ENCRYPTED (fname->dir)) { + oname->name = fname->name; + oname->len = fname->name_len; +return 0 ; + } + +/* Sanity check that the resulting name will fit in the buffer */ +if (fname->name_len >FSCRYPT_BASE64URL_CHARS (NAME_MAX )) +return -EIO ; + + ret =__fscrypt_prepare_readdir (fname->dir); +if (ret) +return ret; + +/* +* Use the raw dentry name as sent by the MDS instead of +* generating a nokey name via fscrypt. +*/ +if (!fscrypt_has_encryption_key (fname->dir)) { +memcpy (oname->name, fname->name, fname->name_len); + oname->len = fname->name_len; +if (is_nokey) + *is_nokey = true; +return 0 ; + } + +if (fname->ctext_len ==0 ) { +int declen; + +if (!tname) { + ret =fscrypt_fname_alloc_buffer (NAME_MAX , &_tname); +if (ret) +return ret; + tname = &_tname; + } + + declen =fscrypt_base64url_decode (fname->name, fname->name_len, tname->name); +if (declen <=0 ) { + ret = -EIO ; + goto out; + } + iname.name = tname->name; + iname.len = declen; + }else { + iname.name = fname->ctext; + iname.len = fname->ctext_len; + } + + ret =fscrypt_fname_disk_to_usr (fname->dir,0 ,0 , &iname, oname); +out: +fscrypt_fname_free_buffer (&_tname); +return ret; +}
fs/ceph/crypto.h +41 -0
59 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 28/64 ceph: fix base64 encoded name's length check in ceph_fname_to_usr()
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
The fname->name is based64_encoded names and the max long shouldn't exceed the NAME_MAX.
The FSCRYPT_BASE64URL_CHARS(NAME_MAX) will be 255 * 4 / 3.
--- fs/ceph/crypto.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/ceph/crypto.c +1 -1
@@ -264,7 +264,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, }/* Sanity check that the resulting name will fit in the buffer */ -if (fname->name_len >FSCRYPT_BASE64URL_CHARS (NAME_MAX )) +if (fname->name_len >NAME_MAX || fname->ctext_len >NAME_MAX )return -EIO ; ret =__fscrypt_prepare_readdir (fname->dir);
Signature · 1 line
2.35.1
PATCH v14 29/64 ceph: add fscrypt support to ceph_fill_trace
In reply to Jeff LaytonWhen we get a dentry in a trace, decrypt the name so we can properly instantiate the dentry.
--- fs/ceph/inode.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-)
fs/ceph/inode.c +28 -2
@@ -1405,8 +1405,15 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req)if (dir && req->r_op ==CEPH_MDS_OP_LOOKUPNAME &&test_bit (CEPH_MDS_R_PARENT_LOCKED , &req->r_req_flags) && !test_bit (CEPH_MDS_R_ABORTED , &req->r_req_flags)) { +bool is_nokey = false;struct qstr dname;struct dentry *dn, *parent; +struct fscrypt_str oname =FSTR_INIT (NULL ,0 ); +struct ceph_fname fname = { .dir = dir, + .name = rinfo->dname, + .ctext = rinfo->altname, + .name_len = rinfo->dname_len, + .ctext_len = rinfo->altname_len };BUG_ON (!rinfo->head->is_target);BUG_ON (req->r_dentry);
@@ -1414,8 +1421,20 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req) parent =d_find_any_alias (dir);BUG_ON (!parent); - dname.name = rinfo->dname; - dname.len = rinfo->dname_len; + err =ceph_fname_alloc_buffer (dir, &oname); +if (err <0 ) { +dput (parent); + goto done; + } + + err =ceph_fname_to_usr (&fname,NULL , &oname, &is_nokey); +if (err <0 ) { +dput (parent); +ceph_fname_free_buffer (dir, &oname); + goto done; + } + dname.name = oname.name; + dname.len = oname.len; dname.hash =full_name_hash (parent, dname.name, dname.len); tvino.ino =le64_to_cpu (rinfo->targeti.in->ino); tvino.snap =le64_to_cpu (rinfo->targeti.in->snapid);
@@ -1430,9 +1449,15 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req) dname.len, dname.name, dn);if (!dn) {dput (parent); +ceph_fname_free_buffer (dir, &oname); err = -ENOMEM ; goto done; } +if (is_nokey) { +spin_lock (&dn->d_lock); + dn->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dn->d_lock); + } err =0 ; } else if (d_really_is_positive(dn) && (ceph_ino(d_inode(dn)) != tvino.ino ||
@@ -1444,6 +1469,7 @@ int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req)dput (dn); goto retry_lookup; } +ceph_fname_free_buffer (dir, &oname); req->r_dentry = dn;dput (parent);
Signature · 1 line
2.35.1
PATCH v14 30/64 ceph: pass the request to parse_reply_info_readdir()
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Instead of passing just the r_reply_info to the readdir reply parser, pass the request pointer directly instead. This will facilitate implementing readdir on fscrypted directories.
--- fs/ceph/mds_client.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
fs/ceph/mds_client.c +12 -10
@@ -406,9 +406,10 @@ static int parse_reply_info_trace(void **p, void *end,* parse readdir results */ static int parse_reply_info_readdir (void **p,void *end, -struct ceph_mds_reply_info_parsed *info, -u64 features) +struct ceph_mds_request *req, +u64 features) { +struct ceph_mds_reply_info_parsed *info = &req->r_reply_info;u32 num, i =0 ;int err;
@@ -650,15 +651,16 @@ static int parse_reply_info_getvxattr(void **p, void *end,* parse extra results */ static int parse_reply_info_extra (void **p,void *end, -struct ceph_mds_reply_info_parsed *info, +struct ceph_mds_request *req,u64 features,struct ceph_mds_session *s) { +struct ceph_mds_reply_info_parsed *info = &req->r_reply_info;u32 op =le32_to_cpu (info->head->op);if (op ==CEPH_MDS_OP_GETFILELOCK )return parse_reply_info_filelock (p, end, info, features);else if (op ==CEPH_MDS_OP_READDIR || op ==CEPH_MDS_OP_LSSNAP ) -return parse_reply_info_readdir (p, end, info, features); +return parse_reply_info_readdir (p, end, req, features);else if (op ==CEPH_MDS_OP_CREATE )return parse_reply_info_create (p, end, info, features, s);else if (op ==CEPH_MDS_OP_GETVXATTR )
@@ -671,9 +673,9 @@ static int parse_reply_info_extra(void **p, void *end,* parse entire mds reply */ static int parse_reply_info (struct ceph_mds_session *s,struct ceph_msg *msg, -struct ceph_mds_reply_info_parsed *info, -u64 features) +struct ceph_mds_request *req,u64 features) { +struct ceph_mds_reply_info_parsed *info = &req->r_reply_info;void *p, *end;u32 len;int err;
@@ -695,7 +697,7 @@ static int parse_reply_info(struct ceph_mds_session *s, struct ceph_msg *msg,ceph_decode_32_safe (&p, end, len, bad);if (len >0 ) {ceph_decode_need (&p, end, len, bad); - err =parse_reply_info_extra (&p, p+len, info, features, s); + err =parse_reply_info_extra (&p, p+len, req, features, s);if (err <0 ) goto out_bad; }
@@ -3440,14 +3442,14 @@ static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg) }dout ("handle_reply tid %lld result %d\n" , tid, result); - rinfo = &req->r_reply_info;if (test_bit (CEPHFS_FEATURE_REPLY_ENCODING , &session->s_features)) - err =parse_reply_info (session, msg, rinfo, (u64)-1 ); + err =parse_reply_info (session, msg, req, (u64)-1 );else - err =parse_reply_info (session, msg, rinfo, session->s_con.peer_features); + err =parse_reply_info (session, msg, req, session->s_con.peer_features);mutex_unlock (&mdsc->mutex);/* Must find target inode outside of mutexes to avoid deadlocks */ + rinfo = &req->r_reply_info;if ((err >=0 ) && rinfo->head->is_target) {struct inode *in =xchg (&req->r_new_inode,NULL );struct ceph_vino tvino = {
Signature · 1 line
2.35.1
PATCH v14 31/64 ceph: add ceph_encode_encrypted_dname() helper
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Add a new helper that basically calls ceph_encode_encrypted_fname, but with a qstr pointer instead of a dentry pointer. This will make it simpler to decrypt names in a readdir reply, before we have a dentry.
--- fs/ceph/crypto.c | 11 ++++++++--- fs/ceph/crypto.h | 8 ++++++++ 2 files changed, 16 insertions(+), 3 deletions(-)
fs/ceph/crypto.c +8 -3
@@ -188,7 +188,7 @@ void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_seswap (req->r_fscrypt_auth, as->fscrypt_auth); } -int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) +int ceph_encode_encrypted_dname (const struct inode *parent,struct qstr *d_name,char *buf) {u32 len;int elen;
@@ -203,7 +203,7 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentr* * See: fscrypt_setup_filename */ -if (!fscrypt_fname_encrypted_size (parent, dentry->d_name.len,NAME_MAX , &len)) +if (!fscrypt_fname_encrypted_size (parent, d_name->len,NAME_MAX , &len))return -ENAMETOOLONG ;/* Allocate a buffer appropriate to hold the result */
@@ -211,7 +211,7 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentrif (!cryptbuf)return -ENOMEM ; - ret =fscrypt_fname_encrypt (parent, &dentry->d_name, cryptbuf, len); + ret =fscrypt_fname_encrypt (parent, d_name, cryptbuf, len);if (ret) {kfree (cryptbuf);return ret;
@@ -235,6 +235,11 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentrreturn elen; } +int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) +{ +return ceph_encode_encrypted_dname (parent, &dentry->d_name, buf); +} +/** * ceph_fname_to_usr - convert a filename for userland presentation * @fname: ceph_fname to be converted
fs/ceph/crypto.h +8 -0
@@ -75,6 +75,7 @@ void ceph_fscrypt_free_dummy_policy(struct ceph_fs_client *fsc);int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode,struct ceph_acl_sec_ctx *as);void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); +int ceph_encode_encrypted_dname (const struct inode *parent,struct qstr *d_name,char *buf);int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf);static inline int ceph_fname_alloc_buffer (struct inode *parent,struct fscrypt_str *fname)
@@ -116,6 +117,13 @@ static inline void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, { } +static inline int ceph_encode_encrypted_dname (const struct inode *parent, +struct qstr *d_name,char *buf) +{ +memcpy (buf, d_name->name, d_name->len); +return d_name->len; +} +static inline int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) {
Signature · 1 line
2.35.1
PATCH v14 32/64 ceph: add support to readdir for encrypted filenames
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Once we've decrypted the names in a readdir reply, we no longer need the crypttext, so overwrite them in ceph_mds_reply_dir_entry with the unencrypted names. Then in both ceph_readdir_prepopulate() and ceph_readdir() we will use the dencrypted name directly.
[ jlayton: convert some BUG_ONs into error returns ]
--- fs/ceph/crypto.c | 12 +++++-- fs/ceph/crypto.h | 1 + fs/ceph/dir.c | 35 +++++++++++++++---- fs/ceph/inode.c | 12 ++++--- fs/ceph/mds_client.c | 81 ++++++++++++++++++++++++++++++++++++++++---- fs/ceph/mds_client.h | 4 +-- 6 files changed, 124 insertions(+), 21 deletions(-)
fs/ceph/crypto.c +10 -2
@@ -195,7 +195,10 @@ int ceph_encode_encrypted_dname(const struct inode *parent, struct qstr *d_name,int ret;u8 *cryptbuf; -WARN_ON_ONCE (!fscrypt_has_encryption_key (parent)); +if (!fscrypt_has_encryption_key (parent)) { +memcpy (buf, d_name->name, d_name->len); +return d_name->len; + }/* * Convert cleartext d_name to ciphertext. If result is longer than
@@ -237,6 +240,8 @@ int ceph_encode_encrypted_dname(const struct inode *parent, struct qstr *d_name,int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) { +WARN_ON_ONCE (!fscrypt_has_encryption_key (parent)); +return ceph_encode_encrypted_dname (parent, &dentry->d_name, buf); }
@@ -281,7 +286,10 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,* generating a nokey name via fscrypt. */ if (!fscrypt_has_encryption_key (fname->dir)) { -memcpy (oname->name, fname->name, fname->name_len); +if (fname->no_copy) + oname->name = fname->name; +else +memcpy (oname->name, fname->name, fname->name_len); oname->len = fname->name_len;if (is_nokey) *is_nokey = true;
fs/ceph/crypto.h +1 -0
@@ -19,6 +19,7 @@ struct ceph_fname {unsigned char *ctext;// binary crypttext (if any) u32 name_len;// length of name buffer u32 ctext_len;// length of crypttext +bool no_copy; };struct ceph_fscrypt_auth {
fs/ceph/dir.c +29 -6
@@ -9,6 +9,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" /* * Directory operations: readdir, lookup, create, link, unlink,
@@ -241,7 +242,9 @@ static int __dcache_readdir(struct file *file, struct dir_context *ctx, di =ceph_dentry (dentry);if (d_unhashed (dentry) ||d_really_is_negative (dentry) || - di->lease_shared_gen != shared_gen) { + di->lease_shared_gen != shared_gen || + ((dentry->d_flags &DCACHE_NOKEY_NAME ) && +fscrypt_has_encryption_key (dir))) {spin_unlock (&dentry->d_lock);dput (dentry); err = -EAGAIN ;
@@ -340,6 +343,10 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) ctx->pos =2 ; } + err =fscrypt_prepare_readdir (inode); +if (err) +return err; +spin_lock (&ci->i_ceph_lock);/* request Fx cap. if have Fx, we don't need to release Fs cap * for later create/unlink. */
@@ -389,6 +396,7 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS );if (IS_ERR (req))return PTR_ERR (req); + err =ceph_alloc_readdir_reply_buffer (req, inode);if (err) {ceph_mdsc_put_request (req);
@@ -402,11 +410,20 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) req->r_inode_drop =CEPH_CAP_FILE_EXCL ; }if (dfi->last_name) { - req->r_path2 =kstrdup (dfi->last_name,GFP_KERNEL ); +struct qstr d_name = { .name = dfi->last_name, + .len =strlen (dfi->last_name) }; + + req->r_path2 =kzalloc (NAME_MAX +1 ,GFP_KERNEL );if (!req->r_path2) {ceph_mdsc_put_request (req);return -ENOMEM ; } + + err =ceph_encode_encrypted_dname (inode, &d_name, req->r_path2); +if (err <0 ) { +ceph_mdsc_put_request (req); +return err; + } }else if (is_hash_order (ctx->pos)) { req->r_args.readdir.offset_hash =cpu_to_le32 (fpos_hash (ctx->pos));
@@ -511,15 +528,20 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx)for (; i < rinfo->dir_nr; i++) {struct ceph_mds_reply_dir_entry *rde = rinfo->dir_entries + i; -BUG_ON (rde->offset < ctx->pos); +if (rde->offset < ctx->pos) { +pr_warn ("%s: rde->offset 0x%llx ctx->pos 0x%llx\n" , + __func__, rde->offset, ctx->pos); +return -EIO ; + } + +if (WARN_ON_ONCE (!rde->inode.in)) +return -EIO ; ctx->pos = rde->offset;dout ("readdir (%d/%d) -> %llx '%.*s' %p\n" , i, rinfo->dir_nr, ctx->pos, rde->name_len, rde->name, &rde->inode.in); -BUG_ON (!rde->inode.in); -if (!dir_emit (ctx, rde->name, rde->name_len,ceph_present_ino (inode->i_sb,le64_to_cpu (rde->inode.in->ino)),le32_to_cpu (rde->inode.in->mode) >>12 )) {
@@ -532,6 +554,8 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx)dout ("filldir stopping us...\n" );return 0 ; } + +/* Reset the lengths to their original allocated vals */ ctx->pos++; }
@@ -586,7 +610,6 @@ static int ceph_readdir(struct file *file, struct dir_context *ctx) dfi->dir_ordered_count);spin_unlock (&ci->i_ceph_lock); } -dout ("readdir %p file %p done.\n" , inode, file);return 0 ; }
fs/ceph/inode.c +8 -4
@@ -1751,7 +1751,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, struct ceph_mds_session *session) {struct dentry *parent = req->r_dentry; -struct ceph_inode_info *ci =ceph_inode (d_inode (parent)); +struct inode *inode =d_inode (parent); +struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;struct qstr dname;struct dentry *dn;
@@ -1825,9 +1826,7 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, tvino.snap =le64_to_cpu (rde->inode.in->snapid);if (rinfo->hash_order) { -u32 hash =ceph_str_hash (ci->i_dir_layout.dl_dir_hash, - rde->name, rde->name_len); - hash =ceph_frag_value (hash); +u32 hash =ceph_frag_value (rde->raw_hash);if (hash != last_hash) fpos_offset =2 ; last_hash = hash;
@@ -1850,6 +1849,11 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req, err = -ENOMEM ; goto out; } +if (rde->is_nokey) { +spin_lock (&dn->d_lock); + dn->d_flags |=DCACHE_NOKEY_NAME ; +spin_unlock (&dn->d_lock); + } }else if (d_really_is_positive (dn) && (ceph_ino (d_inode (dn)) != tvino.ino ||ceph_snap (d_inode (dn)) != tvino.snap)) {
fs/ceph/mds_client.c +74 -7
101 lines not on this page · raw message
fs/ceph/mds_client.h +2 -2
@@ -96,10 +96,10 @@ struct ceph_mds_reply_info_in { };struct ceph_mds_reply_dir_entry { +bool is_nokey;char *name; -u8 *altname;u32 name_len; -u32 altname_len; +u32 raw_hash;struct ceph_mds_reply_lease *lease;struct ceph_mds_reply_info_in inode;loff_t offset;
Signature · 1 line
2.35.1
PATCH v14 33/64 ceph: create symlinks with encrypted and base64-encoded targets
In reply to Jeff LaytonWhen creating symlinks in encrypted directories, encrypt and base64-encode the target with the new inode's key before sending to the MDS.
When filling a symlinked inode, base64-decode it into a buffer that we'll keep in ci->i_symlink. When get_link is called, decrypt the buffer into a new one that will hang off i_link.
--- fs/ceph/crypto.c | 2 +- fs/ceph/dir.c | 51 +++++++++++++++++++--- fs/ceph/inode.c | 107 +++++++++++++++++++++++++++++++++++++++++------ 3 files changed, 142 insertions(+), 18 deletions(-)
fs/ceph/crypto.c +1 -1
@@ -306,7 +306,7 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, tname = &_tname; } - declen =fscrypt_base64url_decode (fname->name, fname->name_len, tname->name); + declen =ceph_base64_decode (fname->name, fname->name_len, tname->name);if (declen <=0 ) { ret = -EIO ; goto out;
fs/ceph/dir.c +46 -5
66 lines not on this page · raw message
fs/ceph/inode.c +95 -12
158 lines not on this page · raw message
Signature · 1 line
2.35.1
PATCH v14 34/64 ceph: make ceph_get_name decrypt filenames
In reply to Jeff LaytonWhen we do a lookupino to the MDS, we get a filename in the trace. ceph_get_name uses that name directly, so we must properly decrypt it before copying it to the name buffer.
--- fs/ceph/export.c | 44 ++++++++++++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 12 deletions(-)
fs/ceph/export.c +32 -12
@@ -7,6 +7,7 @@#include "super.h" #include "mds_client.h" +#include "crypto.h" /* * Basic fh
@@ -534,7 +535,9 @@ static int ceph_get_name(struct dentry *parent, char *name, {struct ceph_mds_client *mdsc;struct ceph_mds_request *req; +struct inode *dir =d_inode (parent);struct inode *inode =d_inode (child); +struct ceph_mds_reply_info_parsed *rinfo;int err;if (ceph_snap (inode) !=CEPH_NOSNAP )
@@ -546,30 +549,47 @@ static int ceph_get_name(struct dentry *parent, char *name,if (IS_ERR (req))return PTR_ERR (req); -inode_lock (d_inode (parent)); - +inode_lock (dir); req->r_inode = inode;ihold (inode); req->r_ino2 =ceph_vino (d_inode (parent)); - req->r_parent =d_inode (parent); -ihold (req->r_parent); + req->r_parent = dir; +ihold (dir);set_bit (CEPH_MDS_R_PARENT_LOCKED , &req->r_req_flags); req->r_num_caps =2 ; err =ceph_mdsc_do_request (mdsc,NULL , req); +inode_unlock (dir); -inode_unlock (d_inode (parent)); +if (err) + goto out; -if (!err) { -struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info; + rinfo = &req->r_reply_info; +if (!IS_ENCRYPTED (dir)) {memcpy (name, rinfo->dname, rinfo->dname_len); name[rinfo->dname_len] =0 ; -dout ("get_name %p ino %llx.%llx name %s\n" , - child,ceph_vinop (inode), name); }else { -dout ("get_name %p ino %llx.%llx err %d\n" , - child,ceph_vinop (inode), err); - } +struct fscrypt_str oname =FSTR_INIT (NULL ,0 ); +struct ceph_fname fname = { .dir = dir, + .name = rinfo->dname, + .ctext = rinfo->altname, + .name_len = rinfo->dname_len, + .ctext_len = rinfo->altname_len }; + + err =ceph_fname_alloc_buffer (dir, &oname); +if (err <0 ) + goto out; + err =ceph_fname_to_usr (&fname,NULL , &oname,NULL ); +if (!err) { +memcpy (name, oname.name, oname.len); + name[oname.len] =0 ; + } +ceph_fname_free_buffer (dir, &oname); + } +out: +dout ("get_name %p ino %llx.%llx err %d %s%s\n" , + child,ceph_vinop (inode), err, + err ?"" :"name " , err ?"" : name);ceph_mdsc_put_request (req);return err; }
Signature · 1 line
2.35.1
PATCH v14 35/64 ceph: add a new ceph.fscrypt.auth vxattr
In reply to Jeff LaytonGive the client a way to get at the xattr from userland, mostly for future debugging purposes.
--- fs/ceph/xattr.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
fs/ceph/xattr.c +26 -0
@@ -352,6 +352,23 @@ static ssize_t ceph_vxattrcb_auth_mds(struct ceph_inode_info *ci,return ret; } +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +static bool ceph_vxattrcb_fscrypt_auth_exists (struct ceph_inode_info *ci) +{ +return ci->fscrypt_auth_len; +} + +static ssize_t ceph_vxattrcb_fscrypt_auth (struct ceph_inode_info *ci,char *val,size_t size) +{ +if (size) { +if (size < ci->fscrypt_auth_len) +return -ERANGE ; +memcpy (val, ci->fscrypt_auth, ci->fscrypt_auth_len); + } +return ci->fscrypt_auth_len; +} +#endif /* CONFIG_FS_ENCRYPTION */ +#define CEPH_XATTR_NAME (_type, _name) XATTR_CEPH_PREFIX #_type "." #_name#define CEPH_XATTR_NAME2 (_type, _name, _name2) \ XATTR_CEPH_PREFIX #_type "." #_name "." #_name2
@@ -500,6 +517,15 @@ static struct ceph_vxattr ceph_common_vxattrs[] = { .exists_cb = NULL, .flags = VXATTR_FLAG_READONLY, }, +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) + { + .name = "ceph.fscrypt.auth", + .name_size = sizeof("ceph.fscrypt.auth"), + .getxattr_cb = ceph_vxattrcb_fscrypt_auth, + .exists_cb = ceph_vxattrcb_fscrypt_auth_exists, + .flags = VXATTR_FLAG_READONLY, + }, +#endif /* CONFIG_FS_ENCRYPTION */ { .name = NULL, 0 } /* Required table terminator */ };
Signature · 1 line
2.35.1
PATCH v14 36/64 ceph: add some fscrypt guardrails
In reply to Jeff LaytonAdd the appropriate calls into fscrypt for various actions, including link, rename, setattr, and the open codepaths.
--- fs/ceph/dir.c | 8 ++++++++ fs/ceph/file.c | 14 +++++++++++++- fs/ceph/inode.c | 4 ++++ 3 files changed, 25 insertions(+), 1 deletion(-)
fs/ceph/dir.c +8 -0
@@ -1121,6 +1121,10 @@ static int ceph_link(struct dentry *old_dentry, struct inode *dir,if (ceph_snap (dir) !=CEPH_NOSNAP )return -EROFS ; + err =fscrypt_prepare_link (old_dentry, dir, dentry); +if (err) +return err; +dout ("link in dir %p old_dentry %p dentry %p\n" , dir, old_dentry, dentry); req =ceph_mdsc_create_request (mdsc,CEPH_MDS_OP_LINK ,USE_AUTH_MDS );
@@ -1318,6 +1322,10 @@ static int ceph_rename(struct user_namespace *mnt_userns, struct inode *old_dir, (!ceph_quota_is_same_realm(old_dir, new_dir)))return -EXDEV ; + err =fscrypt_prepare_rename (old_dir, old_dentry, new_dir, new_dentry, flags); +if (err) +return err; +dout ("rename dir %p dentry %p to dir %p dentry %p\n" , old_dir, old_dentry, new_dir, new_dentry); req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS );
fs/ceph/file.c +13 -1
@@ -372,8 +372,13 @@ int ceph_open(struct inode *inode, struct file *file)/* filter out O_CREAT|O_EXCL; vfs did that already. yuck. */ flags = file->f_flags & ~(O_CREAT |O_EXCL ); -if (S_ISDIR (inode->i_mode)) +if (S_ISDIR (inode->i_mode)) { flags =O_DIRECTORY ;/* mds likes to know */ + }else if (S_ISREG (inode->i_mode)) { + err =fscrypt_file_open (inode, file); +if (err) +return err; + }dout ("open inode %p ino %llx.%llx file %p flags %d (%d)\n" , inode,ceph_vinop (inode), file, flags, file->f_flags);
@@ -857,6 +862,13 @@ int ceph_atomic_open(struct inode *dir, struct dentry *dentry,dout ("atomic_open finish_no_open on dn %p\n" , dn); err =finish_no_open (file, dn); }else { +if (IS_ENCRYPTED (dir) && + !fscrypt_has_permitted_context (dir,d_inode (dentry))) { +pr_warn ("Inconsistent encryption context (parent %llx:%llx child %llx:%llx)\n" , +ceph_vinop (dir),ceph_vinop (d_inode (dentry))); + goto out_req; + } +dout ("atomic_open finish_open on dn %p\n" , dn);if (req->r_op ==CEPH_MDS_OP_CREATE && req->r_reply_info.has_create_ino) {struct inode *newino =d_inode (dentry);
fs/ceph/inode.c +4 -0
@@ -2486,6 +2486,10 @@ int ceph_setattr(struct user_namespace *mnt_userns, struct dentry *dentry,if (ceph_inode_is_shutdown (inode))return -ESTALE ; + err =fscrypt_prepare_setattr (dentry, attr); +if (err) +return err; + err =setattr_prepare (&init_user_ns, dentry, attr);if (err !=0 )return err;
Signature · 1 line
2.35.1
PATCH v14 37/64 ceph: don't allow changing layout on encrypted files/directories
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
Encryption is currently only supported on files/directories with layouts where stripe_count=1. Forbid changing layouts when encryption is involved.
--- fs/ceph/ioctl.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/ceph/ioctl.c +4 -0
@@ -294,6 +294,10 @@ static long ceph_set_encryption_policy(struct file *file, unsigned long arg)struct inode *inode =file_inode (file);struct ceph_inode_info *ci =ceph_inode (inode); +/* encrypted directories can't have striped layout */ +if (ci->i_layout.stripe_count >1 ) +return -EINVAL ; + ret =vet_mds_for_fscrypt (file);if (ret)return ret;
Signature · 1 line
2.35.1
PATCH v14 38/64 libceph: add CEPH_OSD_OP_ASSERT_VER support
In reply to Jeff Layton...and record the user_version in the reply in a new field in ceph_osd_request, so we can populate the assert_ver appropriately. Shuffle the fields a bit too so that the new field fits in an existing hole on x86_64.
--- include/linux/ceph/osd_client.h | 6 +++++- include/linux/ceph/rados.h | 4 ++++ net/ceph/osd_client.c | 5 +++++ 3 files changed, 14 insertions(+), 1 deletion(-)
include/linux/ceph/osd_client.h +5 -1
@@ -196,6 +196,9 @@ struct ceph_osd_req_op {u32 src_fadvise_flags;struct ceph_osd_data osd_data; } copy_from; +struct { +u64 ver; + } assert_ver; }; };
@@ -250,6 +253,7 @@ struct ceph_osd_request {struct ceph_osd_client *r_osdc;struct kref r_kref;bool r_mempool; +bool r_linger;/* don't resend on failure */ struct completion r_completion;/* private to osd_client.c */ ceph_osdc_callback_t r_callback;
@@ -262,9 +266,9 @@ struct ceph_osd_request {struct ceph_snap_context *r_snapc;/* for writes */ struct timespec64 r_mtime;/* ditto */ u64 r_data_offset;/* ditto */ -bool r_linger;/* don't resend on failure */ /* internal */ +u64 r_version;/* data version sent in reply */ unsigned long r_stamp;/* jiffies, send or check time */ unsigned long r_start_stamp;/* jiffies */ ktime_t r_start_latency;/* ktime_t */
include/linux/ceph/rados.h +4 -0
@@ -523,6 +523,10 @@ struct ceph_osd_op {struct {__le64 cookie; } __attribute__ ((packed)) notify; +struct { +__le64 unused; +__le64 ver; + } __attribute__ ((packed)) assert_ver;struct {__le64 offset, length;__le64 src_offset;
net/ceph/osd_client.c +5 -0
@@ -1042,6 +1042,10 @@ static u32 osd_req_encode_op(struct ceph_osd_op *dst, dst->copy_from.src_fadvise_flags =cpu_to_le32 (src->copy_from.src_fadvise_flags);break ; +case CEPH_OSD_OP_ASSERT_VER : + dst->assert_ver.unused =cpu_to_le64 (0 ); + dst->assert_ver.ver =cpu_to_le64 (src->assert_ver.ver); +break ;default :pr_err ("unsupported osd opcode %s\n" ,ceph_osd_op_name (src->op));
@@ -3804,6 +3808,7 @@ static void handle_reply(struct ceph_osd *osd, struct ceph_msg *msg)* one (type of) reply back. */ WARN_ON (!(m.flags &CEPH_OSD_FLAG_ONDISK )); + req->r_version = m.user_version; req->r_result = m.result ?: data_len;finish_request (req);mutex_unlock (&osd->lock);
Signature · 1 line
2.35.1
PATCH v14 39/64 ceph: size handling for encrypted inodes in cap updates
In reply to Jeff LaytonTransmit the rounded-up size as the normal size, and fill out the fscrypt_file field with the real file size.
--- fs/ceph/caps.c | 43 +++++++++++++++++++++++++------------------ fs/ceph/crypto.h | 4 ++++ 2 files changed, 29 insertions(+), 18 deletions(-)
fs/ceph/caps.c +25 -18
@@ -1215,10 +1215,9 @@ struct cap_msg_args {umode_t mode;bool inline_data;bool wake; +bool encrypted;u32 fscrypt_auth_len; -u32 fscrypt_file_len;u8 fscrypt_auth[sizeof (struct ceph_fscrypt_auth )];// for context -u8 fscrypt_file[sizeof (u64)];// for size };/* Marshal up the cap msg to the MDS */
@@ -1253,7 +1252,12 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg) fc->ino =cpu_to_le64 (arg->ino); fc->snap_follows =cpu_to_le64 (arg->follows); - fc->size =cpu_to_le64 (arg->size); +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +if (arg->encrypted) + fc->size =cpu_to_le64 (round_up (arg->size,CEPH_FSCRYPT_BLOCK_SIZE )); + else +#endif + fc->size =cpu_to_le64 (arg->size); fc->max_size =cpu_to_le64 (arg->max_size);ceph_encode_timespec64 (&fc->mtime, &arg->mtime);ceph_encode_timespec64 (&fc->atime, &arg->atime);
@@ -1313,11 +1317,17 @@ static void encode_cap_msg(struct ceph_msg *msg, struct cap_msg_args *arg)ceph_encode_64 (&p,0 );#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) -/* fscrypt_auth and fscrypt_file (version 12) */ +/* +* fscrypt_auth and fscrypt_file (version 12) +* +* fscrypt_auth holds the crypto context (if any). fscrypt_file +* tracks the real i_size as an __le64 field (and we use a rounded-up +* i_size in * the traditional size field). +*/ ceph_encode_32 (&p, arg->fscrypt_auth_len);ceph_encode_copy (&p, arg->fscrypt_auth, arg->fscrypt_auth_len); -ceph_encode_32 (&p, arg->fscrypt_file_len); -ceph_encode_copy (&p, arg->fscrypt_file, arg->fscrypt_file_len); +ceph_encode_32 (&p,sizeof (__le64)); +ceph_encode_64 (&p, arg->size);#else /* CONFIG_FS_ENCRYPTION */ ceph_encode_32 (&p,0 );ceph_encode_32 (&p,0 );
@@ -1389,7 +1399,6 @@ static void __prep_cap(struct cap_msg_args *arg, struct ceph_cap *cap, arg->follows = flushing ? ci->i_head_snapc->seq :0 ; arg->flush_tid = flush_tid; arg->oldest_flush_tid = oldest_flush_tid; - arg->size =i_size_read (inode); ci->i_reported_size = arg->size; arg->max_size = ci->i_wanted_max_size;
@@ -1443,6 +1452,7 @@ static void __prep_cap(struct cap_msg_args *arg, struct ceph_cap *cap, } } arg->flags = flags; + arg->encrypted =IS_ENCRYPTED (inode);#if IS_ENABLED (CONFIG_FS_ENCRYPTION )if (ci->fscrypt_auth_len &&WARN_ON_ONCE (ci->fscrypt_auth_len >sizeof (struct ceph_fscrypt_auth ))) {
@@ -1453,21 +1463,21 @@ static void __prep_cap(struct cap_msg_args *arg, struct ceph_cap *cap,memcpy (arg->fscrypt_auth, ci->fscrypt_auth,min_t (size_t, ci->fscrypt_auth_len,sizeof (arg->fscrypt_auth))); } -/* FIXME: use this to track "real" size */ - arg->fscrypt_file_len =0 ;#endif /* CONFIG_FS_ENCRYPTION */ } +#if IS_ENABLED (CONFIG_FS_ENCRYPTION )#define CAP_MSG_FIXED_FIELDS (sizeof(struct ceph_mds_caps) + \ - 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4 + 8 + 8 + 4 + 4) + 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4 + 8 + 8 + 4 + 4 + 8) -#if IS_ENABLED (CONFIG_FS_ENCRYPTION )static inline int cap_msg_size (struct cap_msg_args *arg) { -return CAP_MSG_FIXED_FIELDS + arg->fscrypt_auth_len + - arg->fscrypt_file_len; +return CAP_MSG_FIXED_FIELDS + arg->fscrypt_auth_len; }#else +#define CAP_MSG_FIXED_FIELDS (sizeof(struct ceph_mds_caps) + \ + 4 + 8 + 4 + 4 + 8 + 4 + 4 + 4 + 8 + 8 + 4 + 8 + 8 + 4 + 4) +static inline int cap_msg_size (struct cap_msg_args *arg) {return CAP_MSG_FIXED_FIELDS ;
@@ -1546,13 +1556,10 @@ static inline int __send_flush_snap(struct inode *inode, arg.inline_data = capsnap->inline_data; arg.flags =0 ; arg.wake = false; + arg.encrypted =IS_ENCRYPTED (inode); -/* -* No fscrypt_auth changes from a capsnap. It will need -* to update fscrypt_file on size changes (TODO). -*/ +/* No fscrypt_auth changes from a capsnap.*/ arg.fscrypt_auth_len =0 ; - arg.fscrypt_file_len =0 ; msg =ceph_msg_new (CEPH_MSG_CLIENT_CAPS ,cap_msg_size (&arg),GFP_NOFS , false);
fs/ceph/crypto.h +4 -0
@@ -9,6 +9,10 @@#include <crypto/sha2.h> #include <linux/fscrypt.h> +#define CEPH_FSCRYPT_BLOCK_SHIFT 12 +#define CEPH_FSCRYPT_BLOCK_SIZE (_AC(1, UL) << CEPH_FSCRYPT_BLOCK_SHIFT) +#define CEPH_FSCRYPT_BLOCK_MASK (~(CEPH_FSCRYPT_BLOCK_SIZE-1)) +struct ceph_fs_client ;struct ceph_acl_sec_ctx ;struct ceph_mds_request ;
Signature · 1 line
2.35.1
PATCH v14 40/64 ceph: fscrypt_file field handling in MClientRequest messages
In reply to Jeff LaytonFor encrypted inodes, transmit a rounded-up size to the MDS as the normal file size and send the real inode size in fscrypt_file field.
Also, fix up creates and truncates to also transmit fscrypt_file.
--- fs/ceph/dir.c | 3 +++ fs/ceph/file.c | 1 + fs/ceph/inode.c | 18 ++++++++++++++++-- fs/ceph/mds_client.c | 9 ++++++++- fs/ceph/mds_client.h | 2 ++ 5 files changed, 30 insertions(+), 3 deletions(-)
fs/ceph/dir.c +3 -0
@@ -910,6 +910,9 @@ static int ceph_mknod(struct user_namespace *mnt_userns, struct inode *dir, goto out_req; } +if (S_ISREG (mode) &&IS_ENCRYPTED (dir)) +set_bit (CEPH_MDS_R_FSCRYPT_FILE , &req->r_req_flags); + req->r_dentry =dget (dentry); req->r_num_caps =2 ; req->r_parent = dir;
fs/ceph/file.c +1 -0
@@ -774,6 +774,7 @@ int ceph_atomic_open(struct inode *dir, struct dentry *dentry, req->r_parent = dir;ihold (dir);if (IS_ENCRYPTED (dir)) { +set_bit (CEPH_MDS_R_FSCRYPT_FILE , &req->r_req_flags);if (!fscrypt_has_encryption_key (dir)) {spin_lock (&dentry->d_lock); dentry->d_flags |=DCACHE_NOKEY_NAME ;
fs/ceph/inode.c +16 -2
@@ -2377,11 +2377,25 @@ int __ceph_setattr(struct inode *inode, struct iattr *attr, struct ceph_iattr *c } }else if ((issued &CEPH_CAP_FILE_SHARED ) ==0 || attr->ia_size != isize) { - req->r_args.setattr.size =cpu_to_le64 (attr->ia_size); - req->r_args.setattr.old_size =cpu_to_le64 (isize); mask |=CEPH_SETATTR_SIZE ; release |=CEPH_CAP_FILE_SHARED |CEPH_CAP_FILE_EXCL |CEPH_CAP_FILE_RD |CEPH_CAP_FILE_WR ; +if (IS_ENCRYPTED (inode) && attr->ia_size) { +set_bit (CEPH_MDS_R_FSCRYPT_FILE , &req->r_req_flags); + mask |=CEPH_SETATTR_FSCRYPT_FILE ; + req->r_args.setattr.size = +cpu_to_le64 (round_up (attr->ia_size, +CEPH_FSCRYPT_BLOCK_SIZE )); + req->r_args.setattr.old_size = +cpu_to_le64 (round_up (isize, +CEPH_FSCRYPT_BLOCK_SIZE )); + req->r_fscrypt_file = attr->ia_size; +/* FIXME: client must zero out any partial blocks! */ + }else { + req->r_args.setattr.size =cpu_to_le64 (attr->ia_size); + req->r_args.setattr.old_size =cpu_to_le64 (isize); + req->r_fscrypt_file =0 ; + } } }if (ia_valid &ATTR_MTIME ) {
fs/ceph/mds_client.c +8 -1
@@ -2752,7 +2752,12 @@ static void encode_mclientrequest_tail(void **p, const struct ceph_mds_request * }else {ceph_encode_32 (p,0 ); } -ceph_encode_32 (p,0 );// fscrypt_file for now +if (test_bit (CEPH_MDS_R_FSCRYPT_FILE , &req->r_req_flags)) { +ceph_encode_32 (p,sizeof (__le64)); +ceph_encode_64 (p, req->r_fscrypt_file); + }else { +ceph_encode_32 (p,0 ); + } }/*
@@ -2838,6 +2843,8 @@ static struct ceph_msg *create_request_message(struct ceph_mds_session *session,/* fscrypt_file */ len +=sizeof (u32); +if (test_bit (CEPH_MDS_R_FSCRYPT_FILE , &req->r_req_flags)) + len +=sizeof (__le64); msg =ceph_msg_new2 (CEPH_MSG_CLIENT_REQUEST , len,1 ,GFP_NOFS , false);if (!msg) {
fs/ceph/mds_client.h +2 -0
@@ -282,6 +282,7 @@ struct ceph_mds_request {#define CEPH_MDS_R_DID_PREPOPULATE (6)/* prepopulated readdir */ #define CEPH_MDS_R_PARENT_LOCKED (7)/* is r_parent->i_rwsem wlocked? */ #define CEPH_MDS_R_ASYNC (8)/* async request */ +#define CEPH_MDS_R_FSCRYPT_FILE (9)/* must marshal fscrypt_file field */ unsigned long r_req_flags;struct mutex r_fill_mutex;
@@ -289,6 +290,7 @@ struct ceph_mds_request {union ceph_mds_request_args r_args;struct ceph_fscrypt_auth *r_fscrypt_auth; +u64 r_fscrypt_file;u8 *r_altname;/* fscrypt binary crypttext for long filenames */ u32 r_altname_len;/* length of r_altname */
Signature · 1 line
2.35.1
PATCH v14 41/64 ceph: get file size from fscrypt_file when present in inode traces
In reply to Jeff LaytonWhen we get an inode trace from the MDS, grab the fscrypt_file field if the inode is encrypted, and use it to populate the i_size field instead of the regular inode size field.
--- fs/ceph/inode.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
fs/ceph/inode.c +15 -3
@@ -1024,6 +1024,7 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page,if (new_version || (new_issued & (CEPH_CAP_ANY_FILE_RD |CEPH_CAP_ANY_FILE_WR ))) { +u64 size =le64_to_cpu (info->size);s64 old_pool = ci->i_layout.pool_id;struct ceph_string *old_ns;
@@ -1037,10 +1038,21 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page, pool_ns = old_ns; +if (IS_ENCRYPTED (inode) && size && (iinfo->fscrypt_file_len ==sizeof (__le64))) { +u64 fsize =__le64_to_cpu (*(__le64 *)iinfo->fscrypt_file); + +if (size ==round_up (fsize,CEPH_FSCRYPT_BLOCK_SIZE )) { + size = fsize; + }else { +pr_warn ("fscrypt size mismatch: size=%llu fscrypt_file=%llu, discarding fscrypt_file size.\n" , + info->size, size); + } + } + queue_trunc =ceph_fill_file_size (inode, issued, -le32_to_cpu (info->truncate_seq), -le64_to_cpu (info->truncate_size), -le64_to_cpu (info->size)); +le32_to_cpu (info->truncate_seq), +le64_to_cpu (info->truncate_size), + size);/* only update max_size on auth cap */ if ((info->cap.flags &CEPH_CAP_FLAG_AUTH ) && ci->i_max_size !=le64_to_cpu (info->max_size)) {
Signature · 1 line
2.35.1
PATCH v14 42/64 ceph: handle fscrypt fields in cap messages from MDS
In reply to Jeff LaytonHandle the new fscrypt_file and fscrypt_auth fields in cap messages. Use them to populate new fields in cap_extra_info and update the inode with those values.
--- fs/ceph/caps.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 76 insertions(+), 2 deletions(-)
fs/ceph/caps.c +76 -2
@@ -3368,6 +3368,9 @@ struct cap_extra_info {/* currently issued */ int issued;struct timespec64 btime; +u8 *fscrypt_auth; +u32 fscrypt_auth_len; +u64 fscrypt_file_size; };/*
@@ -3400,6 +3403,14 @@ static void handle_cap_grant(struct inode *inode,bool deleted_inode = false;bool fill_inline = false; +/* +* If there is at least one crypto block then we'll trust fscrypt_file_size. +* If the real length of the file is 0, then ignore it (it has probably been +* truncated down to 0 by the MDS). +*/ +if (IS_ENCRYPTED (inode) && size) + size = extra_info->fscrypt_file_size; +dout ("handle_cap_grant inode %p cap %p mds%d seq %d %s\n" , inode, cap, session->s_mds, seq,ceph_cap_string (newcaps)); dout(" size %llu max_size %llu, i_size %llu\n", size, max_size,
@@ -3466,6 +3477,10 @@ static void handle_cap_grant(struct inode *inode,dout ("%p mode 0%o uid.gid %d.%d\n" , inode, inode->i_mode,from_kuid (&init_user_ns, inode->i_uid),from_kgid (&init_user_ns, inode->i_gid)); + +WARN_ON_ONCE (ci->fscrypt_auth_len != extra_info->fscrypt_auth_len || +memcmp (ci->fscrypt_auth, extra_info->fscrypt_auth, + ci->fscrypt_auth_len)); } if ((newcaps & CEPH_CAP_LINK_SHARED) &&
@@ -3876,7 +3891,8 @@ static void handle_cap_flushsnap_ack(struct inode *inode, u64 flush_tid,*/ static bool handle_cap_trunc (struct inode *inode,struct ceph_mds_caps *trunc, -struct ceph_mds_session *session) +struct ceph_mds_session *session, +struct cap_extra_info *extra_info) {struct ceph_inode_info *ci =ceph_inode (inode);int mds = session->s_mds;
@@ -3893,6 +3909,14 @@ static bool handle_cap_trunc(struct inode *inode, issued |= implemented | dirty; +/* +* If there is at least one crypto block then we'll trust fscrypt_file_size. +* If the real length of the file is 0, then ignore it (it has probably been +* truncated down to 0 by the MDS). +*/ +if (IS_ENCRYPTED (inode) && size) + size = extra_info->fscrypt_file_size; +dout ("handle_cap_trunc inode %p mds%d seq %d to %lld seq %d\n" , inode, mds, seq, truncate_size, truncate_seq); queue_trunc = ceph_fill_file_size(inode, issued,
@@ -4114,6 +4138,49 @@ static void handle_cap_import(struct ceph_mds_client *mdsc, *target_cap = cap; } +#ifdef CONFIG_FS_ENCRYPTION +static int parse_fscrypt_fields (void **p,void *end,struct cap_extra_info *extra) +{ +u32 len; + +ceph_decode_32_safe (p, end, extra->fscrypt_auth_len, bad); +if (extra->fscrypt_auth_len) { +ceph_decode_need (p, end, extra->fscrypt_auth_len, bad); + extra->fscrypt_auth =kmalloc (extra->fscrypt_auth_len,GFP_KERNEL ); +if (!extra->fscrypt_auth) +return -ENOMEM ; +ceph_decode_copy_safe (p, end, extra->fscrypt_auth, + extra->fscrypt_auth_len, bad); + } + +ceph_decode_32_safe (p, end, len, bad); +if (len >=sizeof (u64)) { +ceph_decode_64_safe (p, end, extra->fscrypt_file_size, bad); + len -=sizeof (u64); + } +ceph_decode_skip_n (p, end, len, bad); +return 0 ; +bad: +return -EIO ; +} +#else +static int parse_fscrypt_fields (void **p,void *end,struct cap_extra_info *extra) +{ +u32 len; + +/* Don't care about these fields unless we're encryption-capable */ +ceph_decode_32_safe (p, end, len, bad); +if (len) +ceph_decode_skip_n (p, end, len, bad); +ceph_decode_32_safe (p, end, len, bad); +if (len) +ceph_decode_skip_n (p, end, len, bad); +return 0 ; +bad: +return -EIO ; +} +#endif +/* * Handle a caps message from the MDS. *
@@ -4232,6 +4299,11 @@ void ceph_handle_caps(struct ceph_mds_session *session,ceph_decode_64_safe (&p, end, extra_info.nsubdirs, bad); } +if (msg_version >=12 ) { +if (parse_fscrypt_fields (&p, end, &extra_info)) + goto bad; + } +/* lookup ino */ inode =ceph_find_inode (mdsc->fsc->sb, vino); dout(" op %s ino %llx.%llx inode %p\n", ceph_cap_op_name(op), vino.ino,
@@ -4328,7 +4400,8 @@ void ceph_handle_caps(struct ceph_mds_session *session,break ;case CEPH_CAP_OP_TRUNC : - queue_trunc =handle_cap_trunc (inode, h, session); + queue_trunc =handle_cap_trunc (inode, h, session, + &extra_info);spin_unlock (&ci->i_ceph_lock);if (queue_trunc)ceph_queue_vmtruncate (inode);
@@ -4346,6 +4419,7 @@ void ceph_handle_caps(struct ceph_mds_session *session,iput (inode); out:ceph_put_string (extra_info.pool_ns); +kfree (extra_info.fscrypt_auth);return ; flush_cap_releases:
Signature · 1 line
2.35.1
PATCH v14 43/64 ceph: update WARN_ON message to pr_warn
In reply to Jeff LaytonGive some more helpful info
--- fs/ceph/caps.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
fs/ceph/caps.c +7 -4
@@ -3477,10 +3477,13 @@ static void handle_cap_grant(struct inode *inode,dout ("%p mode 0%o uid.gid %d.%d\n" , inode, inode->i_mode,from_kuid (&init_user_ns, inode->i_uid),from_kgid (&init_user_ns, inode->i_gid)); - -WARN_ON_ONCE (ci->fscrypt_auth_len != extra_info->fscrypt_auth_len || -memcmp (ci->fscrypt_auth, extra_info->fscrypt_auth, - ci->fscrypt_auth_len)); +#if IS_ENABLED (CONFIG_FS_ENCRYPTION ) +if (ci->fscrypt_auth_len != extra_info->fscrypt_auth_len || +memcmp (ci->fscrypt_auth, extra_info->fscrypt_auth, + ci->fscrypt_auth_len)) +pr_warn_ratelimited ("%s: cap grant attempt to change fscrypt_auth on non-I_NEW inode (old len %d new len %d)\n" , + __func__, ci->fscrypt_auth_len, extra_info->fscrypt_auth_len); +#endif } if ((newcaps & CEPH_CAP_LINK_SHARED) &&
Signature · 1 line
2.35.1
PATCH v14 44/64 ceph: add __ceph_get_caps helper support
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Break out the guts of ceph_get_caps into a helper that takes an inode and ceph_file_info instead of a file pointer.
--- fs/ceph/caps.c | 19 +++++++++++++------ fs/ceph/super.h | 2 ++ 2 files changed, 15 insertions(+), 6 deletions(-)
fs/ceph/caps.c +13 -6
@@ -2951,10 +2951,9 @@ int ceph_try_get_caps(struct inode *inode, int need, int want,* due to a small max_size, make sure we check_max_size (and possibly * ask the mds) so we don't get hung up indefinitely. */ -int ceph_get_caps (struct file *filp,int need,int want,loff_t endoff,int *got) +int __ceph_get_caps (struct inode *inode,struct ceph_file_info *fi,int need, +int want,loff_t endoff,int *got) { -struct ceph_file_info *fi = filp->private_data; -struct inode *inode =file_inode (filp);struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_fs_client *fsc =ceph_inode_to_client (inode);int ret, _got, flags;
@@ -2963,7 +2962,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *gotif (ret <0 )return ret; -if ((fi->fmode &CEPH_FILE_MODE_WR ) && +if (fi && (fi->fmode &CEPH_FILE_MODE_WR ) && fi->filp_gen !=READ_ONCE (fsc->filp_gen))return -EBADF ;
@@ -2971,7 +2970,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *gotwhile (true) { flags &=CEPH_FILE_MODE_MASK ; -if (atomic_read (&fi->num_locks)) +if (fi &&atomic_read (&fi->num_locks)) flags |=CHECK_FILELOCK ; _got =0 ; ret = try_get_cap_refs(inode, need, want, endoff,
@@ -3016,7 +3015,7 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *gotcontinue ; } -if ((fi->fmode &CEPH_FILE_MODE_WR ) && +if (fi && (fi->fmode &CEPH_FILE_MODE_WR ) && fi->filp_gen !=READ_ONCE (fsc->filp_gen)) {if (ret >=0 && _got)ceph_put_cap_refs (ci, _got);
@@ -3079,6 +3078,14 @@ int ceph_get_caps(struct file *filp, int need, int want, loff_t endoff, int *gotreturn 0 ; } +int ceph_get_caps (struct file *filp,int need,int want,loff_t endoff,int *got) +{ +struct ceph_file_info *fi = filp->private_data; +struct inode *inode =file_inode (filp); + +return __ceph_get_caps (inode, fi, need, want, endoff, got); +} +/* * Take cap refs. Caller must already know we hold at least one ref * on the caps in question or we don't know this is safe.
fs/ceph/super.h +2 -0
@@ -1230,6 +1230,8 @@ extern int ceph_encode_dentry_release(void **p, struct dentry *dn,struct inode *dir, int mds, int drop, int unless); +extern int __ceph_get_caps (struct inode *inode,struct ceph_file_info *fi, +int need,int want,loff_t endoff,int *got);extern int ceph_get_caps (struct file *filp,int need,int want,loff_t endoff,int *got); extern int ceph_try_get_caps(struct inode *inode,
Signature · 1 line
2.35.1
PATCH v14 45/64 ceph: add __ceph_sync_read helper support
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Turn the guts of ceph_sync_read into a new helper that takes an inode and an offset instead of a kiocb struct, and make ceph_sync_read call the helper as a wrapper.
--- fs/ceph/file.c | 33 +++++++++++++++++++++------------ fs/ceph/super.h | 2 ++ 2 files changed, 23 insertions(+), 12 deletions(-)
fs/ceph/file.c +21 -12
@@ -937,22 +937,19 @@ enum {* If we get a short result from the OSD, check against i_size; we need to * only return a short read to the caller if we hit EOF. */ -static ssize_t ceph_sync_read (struct kiocb *iocb,struct iov_iter *to, -int *retry_op) +ssize_t __ceph_sync_read (struct inode *inode,loff_t *ki_pos, +struct iov_iter *to,int *retry_op) { -struct file *file = iocb->ki_filp; -struct inode *inode =file_inode (file);struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_fs_client *fsc =ceph_inode_to_client (inode);struct ceph_osd_client *osdc = &fsc->client->osdc;ssize_t ret; -u64 off = iocb->ki_pos; +u64 off = *ki_pos;u64 len =iov_iter_count (to);u64 i_size =i_size_read (inode);bool sparse =ceph_test_mount_opt (fsc,SPARSEREAD ); -dout ("sync_read on file %p %llu~%u %s\n" , file, off, (unsigned )len, - (file->f_flags &O_DIRECT ) ?"O_DIRECT" :"" ); +dout ("sync_read on inode %p %llx~%llx\n" , inode, *ki_pos, len);if (!len)return 0 ;
@@ -1071,14 +1068,14 @@ static ssize_t ceph_sync_read(struct kiocb *iocb, struct iov_iter *to,break ; } -if (off > iocb->ki_pos) { +if (off > *ki_pos) {if (off >= i_size) { *retry_op =CHECK_EOF ; - ret = i_size - iocb->ki_pos; - iocb->ki_pos = i_size; + ret = i_size - *ki_pos; + *ki_pos = i_size; }else { - ret = off - iocb->ki_pos; - iocb->ki_pos = off; + ret = off - *ki_pos; + *ki_pos = off; } }
@@ -1086,6 +1083,18 @@ static ssize_t ceph_sync_read(struct kiocb *iocb, struct iov_iter *to,return ret; } +static ssize_t ceph_sync_read (struct kiocb *iocb,struct iov_iter *to, +int *retry_op) +{ +struct file *file = iocb->ki_filp; +struct inode *inode =file_inode (file); + +dout ("sync_read on file %p %llx~%zx %s\n" , file, iocb->ki_pos, +iov_iter_count (to), (file->f_flags &O_DIRECT ) ?"O_DIRECT" :"" ); + +return __ceph_sync_read (inode, &iocb->ki_pos, to, retry_op); +} +struct ceph_aio_request {struct kiocb *iocb;size_t total_len;
fs/ceph/super.h +2 -0
@@ -1259,6 +1259,8 @@ extern int ceph_renew_caps(struct inode *inode, int fmode);extern int ceph_open (struct inode *inode,struct file *file);extern int ceph_atomic_open (struct inode *dir,struct dentry *dentry,struct file *file,unsigned flags,umode_t mode); +extern ssize_t __ceph_sync_read (struct inode *inode,loff_t *ki_pos, +struct iov_iter *to,int *retry_op);extern int ceph_release (struct inode *inode,struct file *filp);extern void ceph_fill_inline_data (struct inode *inode,struct page *locked_page,char *data,size_t len);
Signature · 1 line
2.35.1
PATCH v14 46/64 ceph: add object version support for sync read
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
Always return the last object's version.
--- fs/ceph/file.c | 12 ++++++++++-- fs/ceph/super.h | 3 ++- 2 files changed, 12 insertions(+), 3 deletions(-)
fs/ceph/file.c +10 -2
@@ -938,7 +938,8 @@ enum {* only return a short read to the caller if we hit EOF. */ ssize_t __ceph_sync_read (struct inode *inode,loff_t *ki_pos, -struct iov_iter *to,int *retry_op) +struct iov_iter *to,int *retry_op, +u64 *last_objver) {struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_fs_client *fsc =ceph_inode_to_client (inode);
@@ -948,6 +949,7 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,u64 len =iov_iter_count (to);u64 i_size =i_size_read (inode);bool sparse =ceph_test_mount_opt (fsc,SPARSEREAD ); +u64 objver =0 ;dout ("sync_read on inode %p %llx~%llx\n" , inode, *ki_pos, len);
@@ -1018,6 +1020,9 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos, req->r_end_latency, len, ret); +if (ret >0 ) + objver = req->r_version; + i_size =i_size_read (inode);dout ("sync_read %llu~%llu got %zd i_size %llu%s\n" , off, len, ret, i_size, (more ?" MORE" :"" ));
@@ -1079,6 +1084,9 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos, } } +if (last_objver && ret >0 ) + *last_objver = objver; +dout ("sync_read result %zd retry_op %d\n" , ret, *retry_op);return ret; }
@@ -1092,7 +1100,7 @@ static ssize_t ceph_sync_read(struct kiocb *iocb, struct iov_iter *to,dout ("sync_read on file %p %llx~%zx %s\n" , file, iocb->ki_pos,iov_iter_count (to), (file->f_flags &O_DIRECT ) ?"O_DIRECT" :"" ); -return __ceph_sync_read (inode, &iocb->ki_pos, to, retry_op); +return __ceph_sync_read (inode, &iocb->ki_pos, to, retry_op,NULL ); }struct ceph_aio_request {
fs/ceph/super.h +2 -1
@@ -1260,7 +1260,8 @@ extern int ceph_open(struct inode *inode, struct file *file);extern int ceph_atomic_open (struct inode *dir,struct dentry *dentry,struct file *file,unsigned flags,umode_t mode);extern ssize_t __ceph_sync_read (struct inode *inode,loff_t *ki_pos, -struct iov_iter *to,int *retry_op); +struct iov_iter *to,int *retry_op, +u64 *last_objver);extern int ceph_release (struct inode *inode,struct file *filp);extern void ceph_fill_inline_data (struct inode *inode,struct page *locked_page,char *data,size_t len);
Signature · 1 line
2.35.1
PATCH v14 47/64 ceph: add infrastructure for file encryption and decryption
In reply to Jeff Layton...and allow test_dummy_encryption to bypass content encryption if mounted with test_dummy_encryption=clear.
--- fs/ceph/crypto.c | 177 +++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/crypto.h | 71 +++++++++++++++++++ fs/ceph/super.c | 8 +++ fs/ceph/super.h | 1 + 4 files changed, 257 insertions(+)
fs/ceph/crypto.c +177 -0
@@ -9,6 +9,7 @@#include <linux/ceph/ceph_debug.h> #include <linux/xattr.h> #include <linux/fscrypt.h> +#include <linux/ceph/striper.h> #include "super.h" #include "mds_client.h"
@@ -323,3 +324,179 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,fscrypt_fname_free_buffer (&_tname);return ret; } + +int ceph_fscrypt_decrypt_block_inplace (const struct inode *inode, +struct page *page,unsigned int len, +unsigned int offs,u64 lblk_num) +{ +struct ceph_mount_options *opt =ceph_inode_to_client (inode)->mount_options; + +if (opt->flags &CEPH_MOUNT_OPT_DUMMY_ENC_CLEAR ) +return 0 ; + +dout ("%s: len %u offs %u blk %llu\n" , __func__, len, offs, lblk_num); +return fscrypt_decrypt_block_inplace (inode, page, len, offs, lblk_num); +} + +int ceph_fscrypt_encrypt_block_inplace (const struct inode *inode, +struct page *page,unsigned int len, +unsigned int offs,u64 lblk_num,gfp_t gfp_flags) +{ +struct ceph_mount_options *opt =ceph_inode_to_client (inode)->mount_options; + +if (opt->flags &CEPH_MOUNT_OPT_DUMMY_ENC_CLEAR ) +return 0 ; + +dout ("%s: len %u offs %u blk %llu\n" , __func__, len, offs, lblk_num); +return fscrypt_encrypt_block_inplace (inode, page, len, offs, lblk_num, gfp_flags); +} + +/** +* ceph_fscrypt_decrypt_pages - decrypt an array of pages +* @inode: pointer to inode associated with these pages +* @page: pointer to page array +* @off: offset into the file that the read data starts +* @len: max length to decrypt +* +* Decrypt an array of fscrypt'ed pages and return the amount of +* data decrypted. Any data in the page prior to the start of the +* first complete block in the read is ignored. Any incomplete +* crypto blocks at the end of the array are ignored (and should +* probably be zeroed by the caller). +* +* Returns the length of the decrypted data or a negative errno. +*/ +int ceph_fscrypt_decrypt_pages (struct inode *inode,struct page **page,u64 off,int len) +{ +int i, num_blocks; +u64 baseblk = off >>CEPH_FSCRYPT_BLOCK_SHIFT ; +int ret =0 ; + +/* +* We can't deal with partial blocks on an encrypted file, so mask off +* the last bit. +*/ + num_blocks =ceph_fscrypt_blocks (off, len &CEPH_FSCRYPT_BLOCK_MASK ); + +/* Decrypt each block */ +for (i =0 ; i < num_blocks; ++i) { +int blkoff = i <<CEPH_FSCRYPT_BLOCK_SHIFT ; +int pgidx = blkoff >>PAGE_SHIFT ; +unsigned int pgoffs =offset_in_page (blkoff); +int fret; + + fret =ceph_fscrypt_decrypt_block_inplace (inode, page[pgidx], +CEPH_FSCRYPT_BLOCK_SIZE , pgoffs, + baseblk + i); +if (fret <0 ) { +if (ret ==0 ) + ret = fret; +break ; + } + ret +=CEPH_FSCRYPT_BLOCK_SIZE ; + } +return ret; +} + +/** +* ceph_fscrypt_decrypt_extents: decrypt received extents in given buffer +* @inode: inode associated with pages being decrypted +* @page: pointer to page array +* @off: offset into the file that the data in page[0] starts +* @map: pointer to extent array +* @ext_cnt: length of extent array +* +* Given an extent map and a page array, decrypt the received data in-place, +* skipping holes. Returns the offset into buffer of end of last decrypted +* block. +*/ +int ceph_fscrypt_decrypt_extents (struct inode *inode,struct page **page,u64 off, +struct ceph_sparse_extent *map,u32 ext_cnt) +{ +int i, ret =0 ; +struct ceph_inode_info *ci =ceph_inode (inode); +u64 objno, objoff; +u32 xlen; + +/* Nothing to do for empty array */ +if (ext_cnt ==0 ) { +dout ("%s: empty array, ret 0\n" , __func__); +return 0 ; + } + +ceph_calc_file_object_mapping (&ci->i_layout, off, map[0 ].len, + &objno, &objoff, &xlen); + +for (i =0 ; i < ext_cnt; ++i) { +struct ceph_sparse_extent *ext = &map[i]; +int pgsoff = ext->off - objoff; +int pgidx = pgsoff >>PAGE_SHIFT ; +int fret; + +if ((ext->off | ext->len) & ~CEPH_FSCRYPT_BLOCK_MASK ) { +pr_warn ("%s: bad encrypted sparse extent idx %d off %llx len %llx\n" , + __func__, i, ext->off, ext->len); +return -EIO ; + } + fret =ceph_fscrypt_decrypt_pages (inode, &page[pgidx], + off + pgsoff, ext->len); +dout ("%s: [%d] 0x%llx~0x%llx fret %d\n" , __func__, i, + ext->off, ext->len, fret); +if (fret <0 ) { +if (ret ==0 ) + ret = fret; +break ; + } + ret = pgsoff + fret; + } +dout ("%s: ret %d\n" , __func__, ret); +return ret; +} + +/** +* ceph_fscrypt_encrypt_pages - encrypt an array of pages +* @inode: pointer to inode associated with these pages +* @page: pointer to page array +* @off: offset into the file that the data starts +* @len: max length to encrypt +* @gfp: gfp flags to use for allocation +* +* Decrypt an array of cleartext pages and return the amount of +* data encrypted. Any data in the page prior to the start of the +* first complete block in the read is ignored. Any incomplete +* crypto blocks at the end of the array are ignored. +* +* Returns the length of the encrypted data or a negative errno. +*/ +int ceph_fscrypt_encrypt_pages (struct inode *inode,struct page **page,u64 off, +int len,gfp_t gfp) +{ +int i, num_blocks; +u64 baseblk = off >>CEPH_FSCRYPT_BLOCK_SHIFT ; +int ret =0 ; + +/* +* We can't deal with partial blocks on an encrypted file, so mask off +* the last bit. +*/ + num_blocks =ceph_fscrypt_blocks (off, len &CEPH_FSCRYPT_BLOCK_MASK ); + +/* Encrypt each block */ +for (i =0 ; i < num_blocks; ++i) { +int blkoff = i <<CEPH_FSCRYPT_BLOCK_SHIFT ; +int pgidx = blkoff >>PAGE_SHIFT ; +unsigned int pgoffs =offset_in_page (blkoff); +int fret; + + fret =ceph_fscrypt_encrypt_block_inplace (inode, page[pgidx], +CEPH_FSCRYPT_BLOCK_SIZE , pgoffs, + baseblk + i, gfp); +if (fret <0 ) { +if (ret ==0 ) + ret = fret; +break ; + } + ret +=CEPH_FSCRYPT_BLOCK_SIZE ; + } +return ret; +}
fs/ceph/crypto.h +71 -0
83 lines not on this page · raw message
fs/ceph/super.c +8 -0
@@ -1093,6 +1093,14 @@ static int ceph_set_test_dummy_encryption(struct super_block *sb, struct fs_contreturn -EEXIST ; } +/* HACK: allow for cleartext "encryption" in files for testing */ +if (fsc->mount_options->test_dummy_encryption && + !strcmp (fsc->mount_options->test_dummy_encryption,"clear" )) { + fsopt->flags |=CEPH_MOUNT_OPT_DUMMY_ENC_CLEAR ; +kfree (fsc->mount_options->test_dummy_encryption); + fsc->mount_options->test_dummy_encryption =NULL ; + } + err =fscrypt_set_test_dummy_encryption (sb, fsc->mount_options->test_dummy_encryption, &fsc->dummy_enc_policy);
fs/ceph/super.h +1 -0
@@ -44,6 +44,7 @@#define CEPH_MOUNT_OPT_NOPAGECACHE (1<<16)/* bypass pagecache altogether */ #define CEPH_MOUNT_OPT_SPARSEREAD (1<<17)/* always do sparse reads */ #define CEPH_MOUNT_OPT_TEST_DUMMY_ENC (1<<18)/* enable dummy encryption (for testing) */ +#define CEPH_MOUNT_OPT_DUMMY_ENC_CLEAR (1<<19)/* don't actually encrypt content */ #define CEPH_MOUNT_OPT_DEFAULT \ (CEPH_MOUNT_OPT_DCACHE | \
Signature · 1 line
2.35.1
PATCH v14 48/64 ceph: add truncate size handling support for fscrypt
In reply to Jeff LaytonFrom: Xiubo Li <[email protected]>
This will transfer the encrypted last block contents to the MDS along with the truncate request only when the new size is smaller and not aligned to the fscrypt BLOCK size. When the last block is located in the file hole, the truncate request will only contain the header.
The MDS could fail to do the truncate if there has another client or process has already updated the RADOS object which contains the last block, and will return -EAGAIN, then the kclient needs to retry it. The RMW will take around 50ms, and will let it retry 20 times for now.
--- fs/ceph/crypto.h | 21 +++++ fs/ceph/inode.c | 198 +++++++++++++++++++++++++++++++++++++++++++++-- fs/ceph/super.h | 5 ++ 3 files changed, 217 insertions(+), 7 deletions(-)
fs/ceph/crypto.h +21 -0
@@ -26,6 +26,27 @@ struct ceph_fname {bool no_copy; }; +/* +* Header for the crypted file when truncating the size, this +* will be sent to MDS, and the MDS will update the encrypted +* last block and then truncate the size. +*/ +struct ceph_fscrypt_truncate_size_header { +__u8 ver; +__u8 compat; + +/* +* It will be sizeof(assert_ver + file_offset + block_size) +* if the last block is empty when it's located in a file +* hole. Or the data_len will plus CEPH_FSCRYPT_BLOCK_SIZE. +*/ +__le32 data_len; + +__le64 change_attr; +__le64 file_offset; +__le32 block_size; +} __packed; +struct ceph_fscrypt_auth {__le32 cfa_version;__le32 cfa_blob_len;
fs/ceph/inode.c +191 -7
274 lines not on this page · raw message
fs/ceph/super.h +5 -0
@@ -410,6 +410,11 @@ struct ceph_inode_info {u32 i_truncate_seq;/* last truncate to smaller size */ u64 i_truncate_size;/* and the size we last truncated down to */ int i_truncate_pending;/* still need to call vmtruncate */ +/* +* For none fscrypt case it equals to i_truncate_size or it will +* equals to fscrypt_file_size +*/ +u64 i_truncate_pagecache_size;u64 i_max_size;/* max file size authorized by mds */ u64 i_reported_size;/* (max_)size reported to or requested of mds */
Signature · 1 line
2.35.1
PATCH v14 49/64 libceph: allow ceph_osdc_new_request to accept a multi-op read
In reply to Jeff LaytonCurrently we have some special-casing for multi-op writes, but in the case of a read, we can't really handle it. All of the current multi-op callers call it with CEPH_OSD_FLAG_WRITE set.
Have ceph_osdc_new_request check for CEPH_OSD_FLAG_READ and if it's set, allocate multiple reply ops instead of multiple request ops. If neither flag is set, return -EINVAL.
--- net/ceph/osd_client.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-)
net/ceph/osd_client.c +21 -6
@@ -1130,15 +1130,30 @@ struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *osdc,if (flags &CEPH_OSD_FLAG_WRITE ) req->r_data_offset = off; -if (num_ops >1 ) +if (num_ops >1 ) { +int num_req_ops, num_rep_ops; +/* -* This is a special case for ceph_writepages_start(), but it -* also covers ceph_uninline_data(). If more multi-op request -* use cases emerge, we will need a separate helper. +* If this is a multi-op write request, assume that we'll need +* request ops. If it's a multi-op read then assume we'll need +* reply ops. Anything else and call it -EINVAL. */ - r =__ceph_osdc_alloc_messages (req,GFP_NOFS , num_ops,0 ); -else +if (flags &CEPH_OSD_FLAG_WRITE ) { + num_req_ops = num_ops; + num_rep_ops =0 ; + }else if (flags &CEPH_OSD_FLAG_READ ) { + num_req_ops =0 ; + num_rep_ops = num_ops; + }else { + r = -EINVAL ; + goto fail; + } + + r =__ceph_osdc_alloc_messages (req,GFP_NOFS , num_req_ops, + num_rep_ops); + }else { r =ceph_osdc_alloc_messages (req,GFP_NOFS ); + }if (r) goto fail;
Signature · 1 line
2.35.1
PATCH v14 50/64 ceph: disable fallocate for encrypted inodes
In reply to Jeff Layton...hopefully, just for now.
--- fs/ceph/file.c | 3 +++ 1 file changed, 3 insertions(+)
fs/ceph/file.c +3 -0
@@ -2217,6 +2217,9 @@ static long ceph_fallocate(struct file *file, int mode,if (!S_ISREG (inode->i_mode))return -EOPNOTSUPP ; +if (IS_ENCRYPTED (inode)) +return -EOPNOTSUPP ; + prealloc_cf =ceph_alloc_cap_flush ();if (!prealloc_cf)return -ENOMEM ;
Signature · 1 line
2.35.1
PATCH v14 51/64 ceph: disable copy offload on encrypted inodes
In reply to Jeff LaytonIf we have an encrypted inode, then the client will need to re-encrypt the contents of the new object. Disable copy offload to or from encrypted inodes.
--- fs/ceph/file.c | 4 ++++ 1 file changed, 4 insertions(+)
fs/ceph/file.c +4 -0
@@ -2536,6 +2536,10 @@ static ssize_t __ceph_copy_file_range(struct file *src_file, loff_t src_off,return -EOPNOTSUPP ; } +/* Every encrypted inode gets its own key, so we can't offload them */ +if (IS_ENCRYPTED (src_inode) ||IS_ENCRYPTED (dst_inode)) +return -EOPNOTSUPP ; +if (len < src_ci->i_layout.object_size)return -EOPNOTSUPP ;/* no remote copy will be done */
Signature · 1 line
2.35.1
PATCH v14 52/64 ceph: don't use special DIO path for encrypted inodes
In reply to Jeff LaytonEventually I want to merge the synchronous and direct read codepaths, possibly via new netfs infrastructure. For now, the direct path is not crypto-enabled, so use the sync read/write paths instead.
--- fs/ceph/file.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/ceph/file.c +4 -2
@@ -1719,7 +1719,9 @@ static ssize_t ceph_read_iter(struct kiocb *iocb, struct iov_iter *to) ceph_cap_string(got));if (ci->i_inline_version ==CEPH_INLINE_NONE ) { -if (!retry_op && (iocb->ki_flags &IOCB_DIRECT )) { +if (!retry_op && + (iocb->ki_flags &IOCB_DIRECT ) && + !IS_ENCRYPTED (inode)) { ret =ceph_direct_read_write (iocb, to,NULL ,NULL );if (ret >=0 && ret < len)
@@ -1945,7 +1947,7 @@ static ssize_t ceph_write_iter(struct kiocb *iocb, struct iov_iter *from)/* we might need to revert back to that point */ data = *from; -if (iocb->ki_flags &IOCB_DIRECT ) +if ((iocb->ki_flags &IOCB_DIRECT ) && !IS_ENCRYPTED (inode)) written =ceph_direct_read_write (iocb, &data, snapc, &prealloc_cf);else
Signature · 1 line
2.35.1
PATCH v14 53/64 ceph: align data in pages in ceph_sync_write
In reply to Jeff LaytonEncrypted files will need to be dealt with in block-sized chunks and once we do that, the way that ceph_sync_write aligns the data in the bounce buffer won't be acceptable.
Change it to align the data the same way it would be aligned in the pagecache.
--- fs/ceph/file.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
fs/ceph/file.c +10 -10
@@ -1561,6 +1561,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,bool check_caps = false;struct timespec64 mtime =current_time (inode);size_t count =iov_iter_count (from); +size_t off;if (ceph_snap (file_inode (file)) !=CEPH_NOSNAP )return -EROFS ;
@@ -1598,12 +1599,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; } -/* -* write from beginning of first page, -* regardless of io alignment -*/ - num_pages = (len +PAGE_SIZE -1 ) >>PAGE_SHIFT ; - + num_pages =calc_pages_for (pos, len); pages =ceph_alloc_page_vector (num_pages,GFP_KERNEL );if (IS_ERR (pages)) { ret =PTR_ERR (pages);
@@ -1611,9 +1607,12 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, } left = len; + off =offset_in_page (pos);for (n =0 ; n < num_pages; n++) { -size_t plen =min_t (size_t, left,PAGE_SIZE ); - ret =copy_page_from_iter (pages[n],0 , plen, from); +size_t plen =min_t (size_t, left,PAGE_SIZE - off); + + ret =copy_page_from_iter (pages[n], off, plen, from); + off =0 ;if (ret != plen) { ret = -EFAULT ;break ;
@@ -1628,8 +1627,9 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, req->r_inode = inode; -osd_req_op_extent_osd_data_pages (req,0 , pages, len,0 , - false, true); +osd_req_op_extent_osd_data_pages (req,0 , pages, len, +offset_in_page (pos), + false, true); req->r_mtime = mtime; ret =ceph_osdc_start_request (&fsc->client->osdc, req, false);
Signature · 1 line
2.35.1
PATCH v14 54/64 ceph: add read/modify/write to ceph_sync_write
In reply to Jeff LaytonWhen doing a synchronous write on an encrypted inode, we have no guarantee that the caller is writing crypto block-aligned data. When that happens, we must do a read/modify/write cycle.
First, expand the range to cover complete blocks. If we had to change the original pos or length, issue a read to fill the first and/or last pages, and fetch the version of the object from the result.
We then copy data into the pages as usual, encrypt the result and issue a write prefixed by an assertion that the version hasn't changed. If it has changed then we restart the whole thing again.
If there is no object at that position in the file (-ENOENT), we prefix the write on an exclusive create of the object instead.
--- fs/ceph/file.c | 319 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 290 insertions(+), 29 deletions(-)
fs/ceph/file.c +290 -29
@@ -1550,18 +1550,16 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,struct inode *inode =file_inode (file);struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_fs_client *fsc =ceph_inode_to_client (inode); -struct ceph_vino vino; +struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;struct page **pages;u64 len;int num_pages;int written =0 ; -int flags;int ret;bool check_caps = false;struct timespec64 mtime =current_time (inode);size_t count =iov_iter_count (from); -size_t off;if (ceph_snap (file_inode (file)) !=CEPH_NOSNAP )return -EROFS ;
@@ -1581,29 +1579,236 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,if (ret <0 )dout ("invalidate_inode_pages2_range returned %d\n" , ret); - flags =/* CEPH_OSD_FLAG_ORDERSNAP | */ CEPH_OSD_FLAG_WRITE ; -while ((len =iov_iter_count (from)) >0 ) {size_t left;int n; +u64 write_pos = pos; +u64 write_len = len; +u64 objnum, objoff; +u32 xlen; +u64 assert_ver; +bool rmw; +bool first, last; +struct iov_iter saved_iter = *from; +size_t off; + +ceph_fscrypt_adjust_off_and_len (inode, &write_pos, &write_len); + +/* clamp the length to the end of first object */ +ceph_calc_file_object_mapping (&ci->i_layout, write_pos, + write_len, &objnum, &objoff, + &xlen); + write_len = xlen; + +/* adjust len downward if it goes beyond current object */ +if (pos + len > write_pos + write_len) + len = write_pos + write_len - pos; - vino =ceph_vino (inode); - req =ceph_osdc_new_request (&fsc->client->osdc, &ci->i_layout, - vino, pos, &len,0 ,1 , -CEPH_OSD_OP_WRITE , flags, snapc, - ci->i_truncate_seq, - ci->i_truncate_size, - false); -if (IS_ERR (req)) { - ret =PTR_ERR (req); -break ; - } +/* +* If we had to adjust the length or position to align with a +* crypto block, then we must do a read/modify/write cycle. We +* use a version assertion to redrive the thing if something +* changes in between. +*/ + first = pos != write_pos; + last = (pos + len) != (write_pos + write_len); + rmw = first || last; - num_pages =calc_pages_for (pos, len); +dout ("sync_write ino %llx %lld~%llu adjusted %lld~%llu -- %srmw\n" , + ci->i_vino.ino, pos, len, write_pos, write_len, rmw ?"" :"no " ); + +/* +* The data is emplaced into the page as it would be if it were in +* an array of pagecache pages. +*/ + num_pages =calc_pages_for (write_pos, write_len); pages =ceph_alloc_page_vector (num_pages,GFP_KERNEL );if (IS_ERR (pages)) { ret =PTR_ERR (pages); - goto out; +break ; + } + +/* Do we need to preload the pages? */ +if (rmw) { +u64 first_pos = write_pos; +u64 last_pos = (write_pos + write_len) -CEPH_FSCRYPT_BLOCK_SIZE ; +u64 read_len =CEPH_FSCRYPT_BLOCK_SIZE ; +struct ceph_osd_req_op *op; + +/* We should only need to do this for encrypted inodes */ +WARN_ON_ONCE (!IS_ENCRYPTED (inode)); + +/* No need to do two reads if first and last blocks are same */ +if (first && last_pos == first_pos) + last = false; + +/* +* Allocate a read request for one or two extents, depending +* on how the request was aligned. +*/ + req =ceph_osdc_new_request (osdc, &ci->i_layout, + ci->i_vino, first ? first_pos : last_pos, + &read_len,0 , (first && last) ?2 :1 , +CEPH_OSD_OP_SPARSE_READ ,CEPH_OSD_FLAG_READ , +NULL , ci->i_truncate_seq, + ci->i_truncate_size, false); +if (IS_ERR (req)) { +ceph_release_page_vector (pages, num_pages); + ret =PTR_ERR (req); +break ; + } + +/* Something is misaligned! */ +if (read_len !=CEPH_FSCRYPT_BLOCK_SIZE ) { +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); + ret = -EIO ; +break ; + } + +/* Add extent for first block? */ + op = &req->r_ops[0 ]; + +if (first) { +osd_req_op_extent_osd_data_pages (req,0 , pages, +CEPH_FSCRYPT_BLOCK_SIZE , +offset_in_page (first_pos), + false, false); +/* We only expect a single extent here */ + ret =__ceph_alloc_sparse_ext_map (op,1 ); +if (ret) { +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); +break ; + } + } + +/* Add extent for last block */ +if (last) { +/* Init the other extent if first extent has been used */ +if (first) { + op = &req->r_ops[1 ]; +osd_req_op_extent_init (req,1 ,CEPH_OSD_OP_SPARSE_READ , + last_pos,CEPH_FSCRYPT_BLOCK_SIZE , + ci->i_truncate_size, + ci->i_truncate_seq); + } + + ret =__ceph_alloc_sparse_ext_map (op,1 ); +if (ret) { +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); +break ; + } + +osd_req_op_extent_osd_data_pages (req, first ?1 :0 , + &pages[num_pages -1 ], +CEPH_FSCRYPT_BLOCK_SIZE , +offset_in_page (last_pos), + false, false); + } + + ret =ceph_osdc_start_request (osdc, req, false); +if (!ret) + ret =ceph_osdc_wait_request (osdc, req); + +/* FIXME: length field is wrong if there are 2 extents */ +ceph_update_read_metrics (&fsc->mdsc->metric, + req->r_start_latency, + req->r_end_latency, + read_len, ret); + +/* Ok if object is not already present */ +if (ret == -ENOENT ) { +/* +* If there is no object, then we can't assert +* on its version. Set it to 0, and we'll use an +* exclusive create instead. +*/ +ceph_osdc_put_request (req); + assert_ver =0 ; + ret =0 ; + +/* +* zero out the soon-to-be uncopied parts of the +* first and last pages. +*/ +if (first) +zero_user_segment (pages[0 ],0 , +offset_in_page (first_pos)); +if (last) +zero_user_segment (pages[num_pages -1 ], +offset_in_page (last_pos), +PAGE_SIZE ); + }else { +if (ret <0 ) { +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); +break ; + } + + op = &req->r_ops[0 ]; +if (op->extent.sparse_ext_cnt ==0 ) { +if (first) +zero_user_segment (pages[0 ],0 , +offset_in_page (first_pos)); +else +zero_user_segment (pages[num_pages -1 ], +offset_in_page (last_pos), +PAGE_SIZE ); + }else if (op->extent.sparse_ext_cnt !=1 || +ceph_sparse_ext_map_end (op) != +CEPH_FSCRYPT_BLOCK_SIZE ) { + ret = -EIO ; +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); +break ; + } + +if (first && last) { + op = &req->r_ops[1 ]; +if (op->extent.sparse_ext_cnt ==0 ) { +zero_user_segment (pages[num_pages -1 ], +offset_in_page (last_pos), +PAGE_SIZE ); + }else if (op->extent.sparse_ext_cnt !=1 || +ceph_sparse_ext_map_end (op) != +CEPH_FSCRYPT_BLOCK_SIZE ) { + ret = -EIO ; +ceph_osdc_put_request (req); +ceph_release_page_vector (pages, num_pages); +break ; + } + } + +/* Grab assert version. It must be non-zero. */ + assert_ver = req->r_version; +WARN_ON_ONCE (ret >0 && assert_ver ==0 ); + +ceph_osdc_put_request (req); +if (first) { + ret =ceph_fscrypt_decrypt_block_inplace (inode, + pages[0 ], +CEPH_FSCRYPT_BLOCK_SIZE , +offset_in_page (first_pos), + first_pos >>CEPH_FSCRYPT_BLOCK_SHIFT ); +if (ret <0 ) { +ceph_release_page_vector (pages, num_pages); +break ; + } + } +if (last) { + ret =ceph_fscrypt_decrypt_block_inplace (inode, + pages[num_pages -1 ], +CEPH_FSCRYPT_BLOCK_SIZE , +offset_in_page (last_pos), + last_pos >>CEPH_FSCRYPT_BLOCK_SHIFT ); +if (ret <0 ) { +ceph_release_page_vector (pages, num_pages); +break ; + } + } + } } left = len;
@@ -1611,43 +1816,98 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,for (n =0 ; n < num_pages; n++) {size_t plen =min_t (size_t, left,PAGE_SIZE - off); +/* copy the data */ ret =copy_page_from_iter (pages[n], off, plen, from); - off =0 ;if (ret != plen) { ret = -EFAULT ;break ; } + off =0 ; left -= ret; } -if (ret <0 ) { +dout ("sync_write write failed with %d\n" , ret);ceph_release_page_vector (pages, num_pages); - goto out; +break ; } - req->r_inode = inode; +if (IS_ENCRYPTED (inode)) { + ret =ceph_fscrypt_encrypt_pages (inode, pages, + write_pos, write_len, +GFP_KERNEL ); +if (ret <0 ) { +dout ("encryption failed with %d\n" , ret); +ceph_release_page_vector (pages, num_pages); +break ; + } + } -osd_req_op_extent_osd_data_pages (req,0 , pages, len, -offset_in_page (pos), - false, true); + req =ceph_osdc_new_request (osdc, &ci->i_layout, + ci->i_vino, write_pos, &write_len, + rmw ?1 :0 , rmw ?2 :1 , +CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , + snapc, ci->i_truncate_seq, + ci->i_truncate_size, false); +if (IS_ERR (req)) { + ret =PTR_ERR (req); +ceph_release_page_vector (pages, num_pages); +break ; + } +dout ("sync_write write op %lld~%llu\n" , write_pos, write_len); +osd_req_op_extent_osd_data_pages (req, rmw ?1 :0 , pages, write_len, +offset_in_page (write_pos), false, + true); + req->r_inode = inode; req->r_mtime = mtime; - ret =ceph_osdc_start_request (&fsc->client->osdc, req, false); + +/* Set up the assertion */ +if (rmw) { +/* +* Set up the assertion. If we don't have a version number, +* then the object doesn't exist yet. Use an exclusive create +* instead of a version assertion in that case. +*/ +if (assert_ver) { +osd_req_op_init (req,0 ,CEPH_OSD_OP_ASSERT_VER ,0 ); + req->r_ops[0 ].assert_ver.ver = assert_ver; + }else { +osd_req_op_init (req,0 ,CEPH_OSD_OP_CREATE , +CEPH_OSD_OP_FLAG_EXCL ); + } + } + + ret =ceph_osdc_start_request (osdc, req, false);if (!ret) - ret =ceph_osdc_wait_request (&fsc->client->osdc, req); + ret =ceph_osdc_wait_request (osdc, req);ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, ret); -out:ceph_osdc_put_request (req);if (ret !=0 ) { +dout ("sync_write osd write returned %d\n" , ret); +/* Version changed! Must re-do the rmw cycle */ +if ((assert_ver && (ret == -ERANGE || ret == -EOVERFLOW )) || + (!assert_ver && ret == -EEXIST )) { +/* We should only ever see this on a rmw */ +WARN_ON_ONCE (!rmw); + +/* The version should never go backward */ +WARN_ON_ONCE (ret == -EOVERFLOW ); + + *from = saved_iter; + +/* FIXME: limit number of times we loop? */ +continue ; + }ceph_set_error_write (ci);break ; } -ceph_clear_error_write (ci); pos += len; written += len; +dout ("sync_write written %d\n" , written);if (pos >i_size_read (inode)) { check_caps =ceph_inode_set_size (inode, pos);if (check_caps)
@@ -1662,6 +1922,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos, ret = written; iocb->ki_pos = pos; } +dout ("sync_write returning %d\n" , ret);return ret; }
Signature · 1 line
2.35.1
PATCH v14 55/64 ceph: plumb in decryption during sync reads
In reply to Jeff LaytonSwitch to using sparse reads when the inode is encrypted.
Note that the crypto block may be smaller than a page, but the reverse cannot be true.
--- fs/ceph/file.c | 89 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 65 insertions(+), 24 deletions(-)
fs/ceph/file.c +65 -24
@@ -948,7 +948,7 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,u64 off = *ki_pos;u64 len =iov_iter_count (to);u64 i_size =i_size_read (inode); -bool sparse =ceph_test_mount_opt (fsc,SPARSEREAD ); +bool sparse =IS_ENCRYPTED (inode) ||ceph_test_mount_opt (fsc,SPARSEREAD );u64 objver =0 ;dout ("sync_read on inode %p %llx~%llx\n" , inode, *ki_pos, len);
@@ -976,10 +976,19 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,int idx;size_t left;struct ceph_osd_req_op *op; +u64 read_off = off; +u64 read_len = len; + +/* determine new offset/length if encrypted */ +ceph_fscrypt_adjust_off_and_len (inode, &read_off, &read_len); + +dout ("sync_read orig %llu~%llu reading %llu~%llu" , + off, len, read_off, read_len); req =ceph_osdc_new_request (osdc, &ci->i_layout, - ci->i_vino, off, &len,0 ,1 , - sparse ?CEPH_OSD_OP_SPARSE_READ :CEPH_OSD_OP_READ , + ci->i_vino, read_off, &read_len,0 ,1 , + sparse ?CEPH_OSD_OP_SPARSE_READ : +CEPH_OSD_OP_READ ,CEPH_OSD_FLAG_READ ,NULL , ci->i_truncate_seq, ci->i_truncate_size, false);
@@ -988,10 +997,13 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,break ; } +/* adjust len downward if the request truncated the len */ +if (off + len > read_off + read_len) + len = read_off + read_len - off; more = len <iov_iter_count (to); - num_pages =calc_pages_for (off, len); - page_off = off & ~PAGE_MASK ; + num_pages =calc_pages_for (read_off, read_len); + page_off =offset_in_page (off); pages =ceph_alloc_page_vector (num_pages,GFP_KERNEL );if (IS_ERR (pages)) {ceph_osdc_put_request (req);
@@ -999,7 +1011,8 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,break ; } -osd_req_op_extent_osd_data_pages (req,0 , pages, len, page_off, +osd_req_op_extent_osd_data_pages (req,0 , pages, read_len, +offset_in_page (read_off), false, false); op = &req->r_ops[0 ];
@@ -1018,7 +1031,7 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,ceph_update_read_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, - len, ret); + read_len, ret);if (ret >0 ) objver = req->r_version;
@@ -1033,8 +1046,34 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,else if (ret == -ENOENT ) ret =0 ; +if (ret >0 &&IS_ENCRYPTED (inode)) { +int fret; + + fret =ceph_fscrypt_decrypt_extents (inode, pages, read_off, + op->extent.sparse_ext, op->extent.sparse_ext_cnt); +if (fret <0 ) { + ret = fret; +ceph_osdc_put_request (req); +break ; + } + +/* account for any partial block at the beginning */ + fret -= (off - read_off); + +/* +* Short read after big offset adjustment? +* Nothing is usable, just call it a zero +* len read. +*/ + fret =max (fret,0 ); + +/* account for partial block at the end */ + ret =min_t (ssize_t, fret, len); + } +ceph_osdc_put_request (req); +/* Short read but not EOF? Zero out the remainder. */ if (ret >=0 && ret < len && (off + ret < i_size)) {int zlen =min (len - ret, i_size - off - ret);int zoff = page_off + ret;
@@ -1048,15 +1087,16 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos, idx =0 ; left = ret >0 ? ret :0 ;while (left >0 ) { -size_t len, copied; - page_off = off & ~PAGE_MASK ; - len =min_t (size_t, left,PAGE_SIZE - page_off); +size_t plen, copied; + + plen =min_t (size_t, left,PAGE_SIZE - page_off);SetPageUptodate (pages[idx]); copied =copy_page_to_iter (pages[idx++], - page_off, len, to); + page_off, plen, to); off += copied; left -= copied; -if (copied < len) { + page_off =0 ; +if (copied < plen) { ret = -EFAULT ;break ; }
@@ -1073,20 +1113,21 @@ ssize_t __ceph_sync_read(struct inode *inode, loff_t *ki_pos,break ; } -if (off > *ki_pos) { -if (off >= i_size) { - *retry_op =CHECK_EOF ; - ret = i_size - *ki_pos; - *ki_pos = i_size; - }else { - ret = off - *ki_pos; - *ki_pos = off; +if (ret >0 ) { +if (off > *ki_pos) { +if (off >= i_size) { + *retry_op =CHECK_EOF ; + ret = i_size - *ki_pos; + *ki_pos = i_size; + }else { + ret = off - *ki_pos; + *ki_pos = off; + } } - } - -if (last_objver && ret >0 ) - *last_objver = objver; +if (last_objver) + *last_objver = objver; + }dout ("sync_read result %zd retry_op %d\n" , ret, *retry_op);return ret; }
Signature · 1 line
2.35.1
PATCH v14 56/64 ceph: add fscrypt decryption support to ceph_netfs_issue_op
In reply to Jeff LaytonForce the use of sparse reads when the inode is encrypted, and add the appropriate code to decrypt the extent map after receiving.
--- fs/ceph/addr.c | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-)
fs/ceph/addr.c +23 -9
@@ -18,6 +18,7 @@#include "mds_client.h" #include "cache.h" #include "metric.h" +#include "crypto.h" #include <linux/ceph/osd_client.h> #include <linux/ceph/striper.h>
@@ -216,7 +217,8 @@ static bool ceph_netfs_clamp_length(struct netfs_io_subrequest *subreq)static void finish_netfs_read (struct ceph_osd_request *req) { -struct ceph_fs_client *fsc =ceph_inode_to_client (req->r_inode); +struct inode *inode = req->r_inode; +struct ceph_fs_client *fsc =ceph_inode_to_client (inode);struct ceph_osd_data *osd_data =osd_req_op_extent_osd_data (req,0 );struct netfs_io_subrequest *subreq = req->r_priv;struct ceph_osd_req_op *op = &req->r_ops[0 ];
@@ -231,15 +233,24 @@ static void finish_netfs_read(struct ceph_osd_request *req) subreq->len, i_size_read(req->r_inode));/* no object means success but no data */ -if (sparse && err >=0 ) - err =ceph_sparse_ext_map_end (op); -else if (err == -ENOENT ) +if (err == -ENOENT ) err =0 ;else if (err == -EBLOCKLISTED ) fsc->blocklisted = true; -if (err >=0 && err < subreq->len) -__set_bit (NETFS_SREQ_CLEAR_TAIL , &subreq->flags); +if (err >=0 ) { +if (sparse && err >0 ) + err =ceph_sparse_ext_map_end (op); +if (err < subreq->len) +__set_bit (NETFS_SREQ_CLEAR_TAIL , &subreq->flags); +if (IS_ENCRYPTED (inode) && err >0 ) { + err =ceph_fscrypt_decrypt_extents (inode, osd_data->pages, + subreq->start, op->extent.sparse_ext, + op->extent.sparse_ext_cnt); +if (err > subreq->len) + err = subreq->len; + } + }netfs_subreq_terminated (subreq, err, true);
@@ -316,13 +327,16 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)size_t page_off;int err =0 ;u64 len = subreq->len; -bool sparse =ceph_test_mount_opt (fsc,SPARSEREAD ); +bool sparse =IS_ENCRYPTED (inode) ||ceph_test_mount_opt (fsc,SPARSEREAD ); +u64 off = subreq->start;if (ci->i_inline_version !=CEPH_INLINE_NONE &&ceph_netfs_issue_op_inline (subreq))return ; - req =ceph_osdc_new_request (&fsc->client->osdc, &ci->i_layout, vino, subreq->start, &len, +ceph_fscrypt_adjust_off_and_len (inode, &off, &len); + + req =ceph_osdc_new_request (&fsc->client->osdc, &ci->i_layout, vino, off, &len,0 ,1 , sparse ?CEPH_OSD_OP_SPARSE_READ :CEPH_OSD_OP_READ ,CEPH_OSD_FLAG_READ | fsc->client->osdc.client->options->read_from_replica,NULL , ci->i_truncate_seq, ci->i_truncate_size, false);
@@ -341,7 +355,7 @@ static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq) }dout ("%s: pos=%llu orig_len=%zu len=%llu\n" , __func__, subreq->start, subreq->len, len); -iov_iter_xarray (&iter,READ , &rreq->mapping->i_pages, subreq->start, len); +iov_iter_xarray (&iter,READ , &rreq->mapping->i_pages, off, len); err =iov_iter_get_pages_alloc (&iter, &pages, len, &page_off);if (err <0 ) {dout ("%s: iov_ter_get_pages_alloc returned %d\n" , __func__, err);
Signature · 1 line
2.35.1
PATCH v14 57/64 ceph: set i_blkbits to crypto block size for encrypted inodes
In reply to Jeff LaytonSome of the underlying infrastructure for fscrypt relies on i_blkbits being aligned to the crypto blocksize.
--- fs/ceph/inode.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
fs/ceph/inode.c +9 -7
@@ -976,13 +976,6 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page, issued |=__ceph_caps_dirty (ci); new_issued = ~issued & info_caps; -/* directories have fl_stripe_unit set to zero */ -if (le32_to_cpu (info->layout.fl_stripe_unit)) - inode->i_blkbits = -fls (le32_to_cpu (info->layout.fl_stripe_unit)) -1 ; -else - inode->i_blkbits =CEPH_BLOCK_SHIFT ; -__ceph_update_quota (ci, iinfo->max_bytes, iinfo->max_files);#ifdef CONFIG_FS_ENCRYPTION
@@ -1008,6 +1001,15 @@ int ceph_fill_inode(struct inode *inode, struct page *locked_page,ceph_decode_timespec64 (&ci->i_snap_btime, &iinfo->snap_btime); } +/* directories have fl_stripe_unit set to zero */ +if (IS_ENCRYPTED (inode)) + inode->i_blkbits =CEPH_FSCRYPT_BLOCK_SHIFT ; +else if (le32_to_cpu (info->layout.fl_stripe_unit)) + inode->i_blkbits = +fls (le32_to_cpu (info->layout.fl_stripe_unit)) -1 ; +else + inode->i_blkbits =CEPH_BLOCK_SHIFT ; +if ((new_version || (new_issued &CEPH_CAP_LINK_SHARED )) && (issued &CEPH_CAP_LINK_EXCL ) ==0 )set_nlink (inode,le32_to_cpu (info->nlink));
Signature · 1 line
2.35.1
PATCH v14 58/64 ceph: add encryption support to writepage
In reply to Jeff LaytonAllow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)
fs/ceph/addr.c +27 -7
@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);
@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);
@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);
@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Signature · 1 line
2.35.1
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 59 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +12
-4@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 661
Quoted 10 lines
fs/ceph/addr.c +8
-0ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }
Here IMO we should redirty the page instead of detaching the page's private data in 'out:' ?
-- Xiubo
Quoted 22 lines
fs/ceph/addr.c +7
-3/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 74 lines · Xiubo Li
Quoted 59 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +12
-4@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 661
Quoted 10 lines
fs/ceph/addr.c +8
-0ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }Here IMO we should redirty the page instead of detaching the page's private data in 'out:' ?
Quoted 3 lines
-- Xiubo
Good catch. I think you're right. I'll fold the following delta into this patch:
fs/ceph/addr.c +4 -2
@@ -655,10 +655,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out; +redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +return PTR_ERR (bounce_page); } } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,
Quoted 23 lines
Quoted 22 lines
fs/ceph/addr.c +7
-3/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 85 lines · Jeff Layton
Quoted 73 lines · Xiubo Li
Quoted 59 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +12
-4@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 661
Quoted 10 lines
fs/ceph/addr.c +8
-0ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }Here IMO we should redirty the page instead of detaching the page's private data in 'out:' ?
Quoted 3 lines
-- Xiubo
Good catch. I think you're right. I'll fold the following delta into this patch:
fs/ceph/addr.c +0
-0
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 660
Quoted 8 lines
fs/ceph/addr.c +3
-2@@ -655,10 +655,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out; +redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +return PTR_ERR (bounce_page);
You should also call ceph_osdc_put_request() too ?
-- Xiubo
Quoted 30 lines
fs/ceph/addr.c +1
-0} } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,Quoted 24 lines
Quoted 24 lines
fs/ceph/addr.c +7
-3/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 97 lines · Xiubo Li
Quoted 85 lines · Jeff Layton
Quoted 73 lines · Xiubo Li
Quoted 59 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +12
-4@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 661
Quoted 10 lines
fs/ceph/addr.c +8
-0ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }Here IMO we should redirty the page instead of detaching the page's private data in 'out:' ?
Quoted 3 lines
-- Xiubo
Good catch. I think you're right. I'll fold the following delta into this patch:
fs/ceph/addr.c +0
-0fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 660
Quoted 8 lines
fs/ceph/addr.c +3
-2@@ -655,10 +655,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out; +redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +return PTR_ERR (bounce_page);You should also call
ceph_osdc_put_request()too ?
Quoted 2 lines
-- Xiubo
Definitely:
fs/ceph/addr.c +5 -2
@@ -655,10 +655,13 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out; +redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +ceph_osdc_put_request (req); +return PTR_ERR (bounce_page); } } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,
Quoted 31 lines
Quoted 30 lines
fs/ceph/addr.c +1
-0} } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,Quoted 24 lines
Quoted 24 lines
fs/ceph/addr.c +7
-3/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 111 lines · Jeff Layton
Quoted 96 lines · Xiubo Li
Quoted 85 lines · Jeff Layton
Quoted 73 lines · Xiubo Li
Quoted 59 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +12
-4@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 661
Quoted 10 lines
fs/ceph/addr.c +8
-0ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }Here IMO we should redirty the page instead of detaching the page's private data in 'out:' ?
Quoted 3 lines
-- Xiubo
Good catch. I think you're right. I'll fold the following delta into this patch:
fs/ceph/addr.c +0
-0fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 660
Quoted 8 lines
fs/ceph/addr.c +3
-2@@ -655,10 +655,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out; +redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +return PTR_ERR (bounce_page);You should also call
ceph_osdc_put_request()too ?Quoted 2 lines
-- Xiubo
Definitely:
fs/ceph/addr.c +0
-2@@ -655,10 +655,13 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE ,0 ,GFP_NOFS );if (IS_ERR (bounce_page)) { - err =PTR_ERR (bounce_page); - goto out;
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 667
Quoted 10 lines
fs/ceph/addr.c +5
-0+redirty_page_for_writepage (wbc, page); +end_page_writeback (page); +ceph_osdc_put_request (req); +return PTR_ERR (bounce_page); } } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,
Please remove the 'out' tag, it's not needed any more.
The others LGTM.
-- Xiubo
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
fs/ceph/addr.c +1
-0} } +/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); osd_req_op_extent_osd_data_pages(req, 0,Quoted 24 lines
Quoted 24 lines
fs/ceph/addr.c +7
-3/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
In reply to Jeff LaytonQuoted 27 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +2
-0@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 628
Quoted 6 lines
fs/ceph/addr.c +2
-0@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );
The above is buggy. We're only setting "wlen" in the encrypted case. You would think that the compiler would catch that, but next usage of wlen just passes a pointer to it to another function and that cloaks the warning.
Fixed in the wip-fscrypt branch, but I'll hold off on re-posting the series in case any other new bugs turn up.
Quoted 58 lines
fs/ceph/addr.c +23
-7dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 29 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +2
-0@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
Quoted 10 lines
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 628
Quoted 5 lines
fs/ceph/addr.c +2
-0if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );The above is buggy. We're only setting "wlen" in the encrypted case. You would think that the compiler would catch that, but next usage of wlen just passes a pointer to it to another function and that cloaks the warning.
Yikes! That's indeed the sort of things we got used to have compilers complaining about. That must have been fun to figure this out. Nice ;-)
Cheers,
Signature · 1 line
Luís
Quoted 65 lines
Fixed in the wip-fscrypt branch, but I'll hold off on re-posting the series in case any other new bugs turn up.
Quoted 58 lines
fs/ceph/addr.c +23
-7dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 42 lines · Luís Henriques
Quoted 29 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +2
-0@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)Quoted 10 lines
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 628
Quoted 5 lines
fs/ceph/addr.c +2
-0if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );The above is buggy. We're only setting "wlen" in the encrypted case. You would think that the compiler would catch that, but next usage of wlen just passes a pointer to it to another function and that cloaks the warning.
Quoted 3 lines
Yikes! That's indeed the sort of things we got used to have compilers complaining about. That must have been fun to figure this out. Nice ;-)
Yeah. I remember that some older versions of gcc would complain about uninitialized vars when you passed a pointer to it to another function. That went away a while back, which was good since it often fired on false positives.
What would have been nice here would be for the compiler to notice that wlen was inconsistently initialized before we passed the pointer to the function. Not sure how hard that would be to catch though.
Signature · 1 line
Jeff Layton <[email protected]>
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
Quoted 44 lines · Jeff Layton
Quoted 40 lines · Luís Henriques
Quoted 29 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +2
-0@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)Quoted 9 lines
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 628
Quoted 5 lines
fs/ceph/addr.c +2
-0if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );The above is buggy. We're only setting "wlen" in the encrypted case. You would think that the compiler would catch that, but next usage of wlen just passes a pointer to it to another function and that cloaks the warning.
Quoted 3 lines
Yikes! That's indeed the sort of things we got used to have compilers complaining about. That must have been fun to figure this out. Nice ;-)
Quoted 8 lines
Yeah. I remember that some older versions of gcc would complain about uninitialized vars when you passed a pointer to it to another function. That went away a while back, which was good since it often fired on false positives.
What would have been nice here would be for the compiler to notice that wlen was inconsistently initialized before we passed the pointer to the function. Not sure how hard that would be to catch though.
I didn't compile this branch with the FSCRYPT being disabled yet recently, not sure whether will the compiler complains it then.
Re: [PATCH v14 58/64] ceph: add encryption support to writepage
In reply to Jeff LaytonQuoted 29 lines · Jeff Layton
Quoted 28 lines · Jeff Layton
Allow writepage to issue encrypted writes. Extend out the requested size and offset to cover complete blocks, and then encrypt and write them to the OSDs.
Signed-off-by Jeff Layton <[email protected]>--- fs/ceph/addr.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)fs/ceph/addr.c +2
-0@@ -586,10 +586,12 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)loff_t page_off =page_offset (page);int err;loff_t len =thp_size (page); +loff_t wlen;struct ceph_writeback_ctl ceph_wbc;struct ceph_osd_client *osdc = &fsc->client->osdc;struct ceph_osd_request *req;bool caching =ceph_is_cache_enabled (inode); +struct page *bounce_page =NULL ;dout ("writepage %p idx %lu\n" , page, page->index);@@ -621,6 +623,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
Quoted 9 lines
fs/ceph/addr.c static int writepage_nounlock(struct page *page, struct writeback_control *wbc) · line 628
Quoted 5 lines
fs/ceph/addr.c +2
-0if (ceph_wbc.i_size < page_off + len) len = ceph_wbc.i_size - page_off; +if (IS_ENCRYPTED (inode)) + wlen =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE );The above is buggy. We're only setting "wlen" in the encrypted case. You would think that the compiler would catch that, but next usage of wlen just passes a pointer to it to another function and that cloaks the warning.
Nice catch !
Quoted 61 lines
Fixed in the wip-fscrypt branch, but I'll hold off on re-posting the series in case any other new bugs turn up.
Quoted 58 lines
fs/ceph/addr.c +23
-7dout ("writepage %p page %p index %lu on %llu~%llu snapc %p seq %lld\n" , inode, page, page->index, page_off, len, snapc, snapc->seq);@@ -629,24 +633,39 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc) CONGESTION_ON_THRESH(fsc->mount_options->congestion_kb)) fsc->write_congested = true; - req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), page_off, &len,0 ,1 , -CEPH_OSD_OP_WRITE ,CEPH_OSD_FLAG_WRITE , snapc, - ceph_wbc.truncate_seq, ceph_wbc.truncate_size, - true); + req =ceph_osdc_new_request (osdc, &ci->i_layout,ceph_vino (inode), + page_off, &wlen,0 ,1 ,CEPH_OSD_OP_WRITE , +CEPH_OSD_FLAG_WRITE , snapc, + ceph_wbc.truncate_seq, + ceph_wbc.truncate_size, true);if (IS_ERR (req)) {redirty_page_for_writepage (wbc, page);return PTR_ERR (req); } +if (wlen < len) + len = wlen; +set_page_writeback (page);if (caching)ceph_set_page_fscache (page);ceph_fscache_write_to_cache (inode, page_off, len, caching); +if (IS_ENCRYPTED (inode)) { + bounce_page =fscrypt_encrypt_pagecache_blocks (page,CEPH_FSCRYPT_BLOCK_SIZE , +0 ,GFP_NOFS ); +if (IS_ERR (bounce_page)) { + err =PTR_ERR (bounce_page); + goto out; + } + }/* it may be a short write due to an object boundary */ WARN_ON_ONCE (len >thp_size (page)); -osd_req_op_extent_osd_data_pages (req,0 , &page, len,0 , false, false); -dout ("writepage %llu~%llu (%llu bytes)\n" , page_off, len, len); +osd_req_op_extent_osd_data_pages (req,0 , + bounce_page ? &bounce_page : &page, wlen,0 , + false, false); +dout ("writepage %llu~%llu (%llu bytes, %sencrypted)\n" , + page_off, len, wlen,IS_ENCRYPTED (inode) ?"" :"not " ); req->r_mtime = inode->i_mtime; err =ceph_osdc_start_request (osdc, req, true);@@ -655,7 +674,8 @@ static int writepage_nounlock(struct page *page, struct writeback_control *wbc)ceph_update_write_metrics (&fsc->mdsc->metric, req->r_start_latency, req->r_end_latency, len, err); - +fscrypt_free_bounce_page (bounce_page); +out:ceph_osdc_put_request (req);if (err ==0 ) err = len;
PATCH v14 59/64 ceph: fscrypt support for writepages
In reply to Jeff LaytonAdd the appropriate machinery to write back dirty data with encryption.
--- fs/ceph/addr.c | 63 +++++++++++++++++++++++++++++++++++++++--------- fs/ceph/crypto.h | 18 +++++++++++++- 2 files changed, 68 insertions(+), 13 deletions(-)
fs/ceph/addr.c +51 -12
@@ -547,10 +547,12 @@ static u64 get_writepages_data_length(struct inode *inode, struct page *page, u64 start) {struct ceph_inode_info *ci =ceph_inode (inode); -struct ceph_snap_context *snapc =page_snap_context (page); +struct ceph_snap_context *snapc;struct ceph_cap_snap *capsnap =NULL ;u64 end =i_size_read (inode); +u64 ret; + snapc =page_snap_context (ceph_fscrypt_pagecache_page (page));if (snapc != ci->i_head_snapc) {bool found = false;spin_lock (&ci->i_ceph_lock);
@@ -565,9 +567,12 @@ static u64 get_writepages_data_length(struct inode *inode,spin_unlock (&ci->i_ceph_lock);WARN_ON (!found); } -if (end >page_offset (page) +thp_size (page)) - end =page_offset (page) +thp_size (page); -return end > start ? end - start :0 ; +if (end >ceph_fscrypt_page_offset (page) +thp_size (page)) + end =ceph_fscrypt_page_offset (page) +thp_size (page); + ret = end > start ? end - start :0 ; +if (ret &&fscrypt_is_bounce_page (page)) + ret =round_up (ret,CEPH_FSCRYPT_BLOCK_SIZE ); +return ret; }/*
@@ -791,6 +796,11 @@ static void writepages_finish(struct ceph_osd_request *req) total_pages += num_pages;for (j =0 ; j < num_pages; j++) { page = osd_data->pages[j]; +if (fscrypt_is_bounce_page (page)) { + page =fscrypt_pagecache_page (page); +fscrypt_free_bounce_page (osd_data->pages[j]); + osd_data->pages[j] = page; + }BUG_ON (!page);WARN_ON (!PageUptodate (page));
@@ -1052,9 +1062,28 @@ static int ceph_writepages_start(struct address_space *mapping, fsc->mount_options->congestion_kb)) fsc->write_congested = true; - pages[locked_pages++] = page; - pvec.pages[i] =NULL ; +if (IS_ENCRYPTED (inode)) { + pages[locked_pages] = +fscrypt_encrypt_pagecache_blocks (page, +PAGE_SIZE ,0 , + locked_pages ?GFP_NOWAIT :GFP_NOFS ); +if (IS_ERR (pages[locked_pages])) { +if (PTR_ERR (pages[locked_pages]) == -EINVAL ) +pr_err ("%s: inode->i_blkbits=%hhu\n" , + __func__, inode->i_blkbits); +/* better not fail on first page! */ +BUG_ON (locked_pages ==0 ); + pages[locked_pages] =NULL ; +redirty_page_for_writepage (wbc, page); +unlock_page (page); +break ; + } + ++locked_pages; + }else { + pages[locked_pages++] = page; + } + pvec.pages[i] =NULL ; len +=thp_size (page); }
@@ -1082,7 +1111,7 @@ static int ceph_writepages_start(struct address_space *mapping, } new_request: - offset =page_offset (pages[0 ]); + offset =ceph_fscrypt_page_offset (pages[0 ]); len = wsize; req = ceph_osdc_new_request(&fsc->client->osdc,
@@ -1103,8 +1132,8 @@ static int ceph_writepages_start(struct address_space *mapping, ceph_wbc.truncate_size, true);BUG_ON (IS_ERR (req)); } -BUG_ON (len <page_offset (pages[locked_pages -1 ]) + -thp_size (page) - offset); +BUG_ON (len <ceph_fscrypt_page_offset (pages[locked_pages -1 ]) + +thp_size (pages[locked_pages -1 ]) - offset); req->r_callback = writepages_finish; req->r_inode = inode;
@@ -1114,7 +1143,9 @@ static int ceph_writepages_start(struct address_space *mapping, data_pages = pages; op_idx =0 ;for (i =0 ; i < locked_pages; i++) { -u64 cur_offset =page_offset (pages[i]); +struct page *page =ceph_fscrypt_pagecache_page (pages[i]); + +u64 cur_offset =page_offset (page);/* * Discontinuity in page range? Ceph can handle that by just passing * multiple extents in the write op.
@@ -1143,9 +1174,9 @@ static int ceph_writepages_start(struct address_space *mapping, op_idx++; } -set_page_writeback (pages[i]); +set_page_writeback (page);if (caching) -ceph_set_page_fscache (pages[i]); +ceph_set_page_fscache (page); len +=thp_size (page); }ceph_fscache_write_to_cache (inode, offset, len, caching);
@@ -1161,8 +1192,16 @@ static int ceph_writepages_start(struct address_space *mapping, offset); len =max (len, min_len); } +if (IS_ENCRYPTED (inode)) + len =round_up (len,CEPH_FSCRYPT_BLOCK_SIZE ); +dout ("writepages got pages at %llu~%llu\n" , offset, len); +if (IS_ENCRYPTED (inode) && + ((offset | len) & ~CEPH_FSCRYPT_BLOCK_MASK )) +pr_warn ("%s: bad encrypted write offset=%lld len=%llu\n" , + __func__, offset, len); +osd_req_op_extent_osd_data_pages (req, op_idx, data_pages, len,0 , from_pool, false);osd_req_op_extent_update (req, op_idx, len);
fs/ceph/crypto.h +17 -1
@@ -154,6 +154,12 @@ int ceph_fscrypt_decrypt_extents(struct inode *inode, struct page **page, u64 of struct ceph_sparse_extent *map, u32 ext_cnt);int ceph_fscrypt_encrypt_pages (struct inode *inode,struct page **page,u64 off,int len,gfp_t gfp); + +static inline struct page *ceph_fscrypt_pagecache_page (struct page *page) +{ +return fscrypt_is_bounce_page (page) ?fscrypt_pagecache_page (page) : page; +} +#else /* CONFIG_FS_ENCRYPTION */ static inline void ceph_fscrypt_set_ops (struct super_block *sb)
@@ -243,6 +249,16 @@ static inline int ceph_fscrypt_encrypt_pages(struct inode *inode, struct page ** {return 0 ; } + +static inline struct page *ceph_fscrypt_pagecache_page (struct page *page) +{ +return page; +}#endif /* CONFIG_FS_ENCRYPTION */ -#endif +static inline loff_t ceph_fscrypt_page_offset (struct page *page) +{ +return page_offset (ceph_fscrypt_pagecache_page (page)); +} + +#endif /* _CEPH_CRYPTO_H */
Signature · 1 line
2.35.1
PATCH v14 60/64 ceph: invalidate pages when doing direct/sync writes
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
When doing a direct/sync write, we need to invalidate the page cache in the range being written to. If we don't do this, the cache will include invalid data as we just did a write that avoided the page cache.
In the event that invalidation fails, just ignore the error. That likely just means that we raced with another task doing a buffered write, in which case we want to leave the page intact anyway.
[ jlayton: minor comment update ]
--- fs/ceph/file.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
fs/ceph/file.c +14 -5
@@ -1614,11 +1614,6 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,return ret;ceph_fscache_invalidate (inode, false); - ret =invalidate_inode_pages2_range (inode->i_mapping, - pos >>PAGE_SHIFT , - (pos + count -1 ) >>PAGE_SHIFT ); -if (ret <0 ) -dout ("invalidate_inode_pages2_range returned %d\n" , ret);while ((len =iov_iter_count (from)) >0 ) {size_t left;
@@ -1946,6 +1941,20 @@ ceph_sync_write(struct kiocb *iocb, struct iov_iter *from, loff_t pos,break ; }ceph_clear_error_write (ci); + +/* +* We successfully wrote to a range of the file. Declare +* that region of the pagecache invalid. +*/ + ret =invalidate_inode_pages2_range ( + inode->i_mapping, + pos >>PAGE_SHIFT , + (pos + len -1 ) >>PAGE_SHIFT ); +if (ret <0 ) { +dout ("invalidate_inode_pages2_range returned %d\n" , + ret); + ret =0 ; + } pos += len; written += len;dout ("sync_write written %d\n" , written);
Signature · 1 line
2.35.1
PATCH v14 61/64 ceph: add support for encrypted snapshot names
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
Since filenames in encrypted directories are already encrypted and shown as a base64-encoded string when the directory is locked, snapshot names should show a similar behaviour.
--- fs/ceph/inode.c | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-)
fs/ceph/inode.c +29 -4
@@ -91,9 +91,15 @@ struct inode *ceph_new_inode(struct inode *dir, struct dentry *dentry,if (err <0 ) goto out_err; - err =ceph_fscrypt_prepare_context (dir, inode, as_ctx); -if (err) - goto out_err; +/* +* We'll skip setting fscrypt context for snapshots, leaving that for +* the handle_reply(). +*/ +if (ceph_snap (dir) !=CEPH_SNAPDIR ) { + err =ceph_fscrypt_prepare_context (dir, inode, as_ctx); +if (err) + goto out_err; + }return inode; out_err:
@@ -157,6 +163,7 @@ struct inode *ceph_get_snapdir(struct inode *parent) };struct inode *inode =ceph_get_inode (parent->i_sb, vino,NULL );struct ceph_inode_info *ci =ceph_inode (inode); +int ret = -ENOTDIR ;if (IS_ERR (inode))return inode;
@@ -182,6 +189,24 @@ struct inode *ceph_get_snapdir(struct inode *parent) ci->i_rbytes =0 ; ci->i_btime =ceph_inode (parent)->i_btime; +#ifdef CONFIG_FS_ENCRYPTION +/* if encrypted, just borrow fscrypt_auth from parent */ +if (IS_ENCRYPTED (parent)) { +struct ceph_inode_info *pci =ceph_inode (parent); + + ci->fscrypt_auth =kmemdup (pci->fscrypt_auth, + pci->fscrypt_auth_len, +GFP_KERNEL ); +if (ci->fscrypt_auth) { + inode->i_flags |=S_ENCRYPTED ; + ci->fscrypt_auth_len = pci->fscrypt_auth_len; + }else { +dout ("Failed to alloc snapdir fscrypt_auth\n" ); + ret = -ENOMEM ; + goto err; + } + } +#endif if (inode->i_state &I_NEW ) { inode->i_op = &ceph_snapdir_iops; inode->i_fop = &ceph_snapdir_fops;
@@ -195,7 +220,7 @@ struct inode *ceph_get_snapdir(struct inode *parent)discard_new_inode (inode); elseiput (inode ); -return ERR_PTR (-ENOTDIR ); +return ERR_PTR (ret); }const struct inode_operations ceph_file_iops = {
Signature · 1 line
2.35.1
PATCH v14 62/64 ceph: add support for handling encrypted snapshot names
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
When creating a snapshot, the .snap directories for every subdirectory will show the snapshot name in the "long format":
# mkdir .snap/my-snap # ls my-dir/.snap/ _my-snap_1099511627782
Encrypted snapshots will need to be able to handle these snapshot names by encrypting/decrypting only the snapshot part of the string ('my-snap').
Also, since the MDS prevents snapshot names to be bigger than 240 characters it is necessary to adapt CEPH_NOHASH_NAME_MAX to accommodate this extra limitation.
--- fs/ceph/crypto.c | 190 ++++++++++++++++++++++++++++++++++++++++------- fs/ceph/crypto.h | 4 +- 2 files changed, 165 insertions(+), 29 deletions(-)
fs/ceph/crypto.c +163 -27
@@ -189,16 +189,100 @@ void ceph_fscrypt_as_ctx_to_req(struct ceph_mds_request *req, struct ceph_acl_seswap (req->r_fscrypt_auth, as->fscrypt_auth); } -int ceph_encode_encrypted_dname (const struct inode *parent,struct qstr *d_name,char *buf) +/* +* User-created snapshots can't start with '_'. Snapshots that start with this +* character are special (hint: there aren't real snapshots) and use the +* following format: +* +* _<SNAPSHOT-NAME>_<INODE-NUMBER> +* +* where: +* - <SNAPSHOT-NAME> - the real snapshot name that may need to be decrypted, +* - <INODE-NUMBER> - the inode number (in decimal) for the actual snapshot +* +* This function parses these snapshot names and returns the inode +* <INODE-NUMBER>. 'name_len' will also bet set with the <SNAPSHOT-NAME> +* length. +*/ +static struct inode *parse_longname (const struct inode *parent,const char *name, +int *name_len) +{ +struct inode *dir =NULL ; +struct ceph_vino vino = { .snap =CEPH_NOSNAP }; +char *inode_number; +char *name_end; +int orig_len = *name_len; +int ret = -EIO ; + +/* Skip initial '_' */ + name++; + name_end =strrchr (name,'_' ); +if (!name_end) { +dout ("Failed to parse long snapshot name: %s\n" , name); +return ERR_PTR (-EIO ); + } + *name_len = (name_end - name); +if (*name_len <=0 ) { +pr_err ("Failed to parse long snapshot name\n" ); +return ERR_PTR (-EIO ); + } + +/* Get the inode number */ + inode_number =kmemdup_nul (name_end +1 , + orig_len - *name_len -2 , +GFP_KERNEL ); +if (!inode_number) +return ERR_PTR (-ENOMEM ); + ret =kstrtou64 (inode_number,10 , &vino.ino); +if (ret) { +dout ("Failed to parse inode number: %s\n" , name); + dir =ERR_PTR (ret); + goto out; + } + +/* And finally the inode */ + dir =ceph_find_inode (parent->i_sb, vino); +if (!dir) { +/* This can happen if we're not mounting cephfs on the root */ + dir =ceph_get_inode (parent->i_sb, vino,NULL ); +if (!dir) + dir =ERR_PTR (-ENOENT ); + } +if (IS_ERR (dir)) +dout ("Can't find inode %s (%s)\n" , inode_number, name); + +out: +kfree (inode_number); +return dir; +} + +int ceph_encode_encrypted_dname (struct inode *parent,struct qstr *d_name,char *buf) { +struct inode *dir = parent; +struct qstr iname;u32 len; +int name_len;int elen;int ret; -u8 *cryptbuf; +u8 *cryptbuf =NULL ; + + iname.name = d_name->name; + name_len = d_name->len; + +/* Handle the special case of snapshot names that start with '_' */ +if ((ceph_snap (dir) ==CEPH_SNAPDIR ) && (name_len >0 ) && + (iname.name[0 ] =='_' )) { + dir =parse_longname (parent, iname.name, &name_len); +if (IS_ERR (dir)) +return PTR_ERR (dir); + iname.name++;/* skip initial '_' */ + } + iname.len = name_len; -if (!fscrypt_has_encryption_key (parent)) { +if (!fscrypt_has_encryption_key (dir)) {memcpy (buf, d_name->name, d_name->len); -return d_name->len; + elen = d_name->len; + goto out; }/*
@@ -207,18 +291,22 @@ int ceph_encode_encrypted_dname(const struct inode *parent, struct qstr *d_name,* * See: fscrypt_setup_filename */ -if (!fscrypt_fname_encrypted_size (parent, d_name->len,NAME_MAX , &len)) -return -ENAMETOOLONG ; +if (!fscrypt_fname_encrypted_size (dir, iname.len,NAME_MAX , &len)) { + elen = -ENAMETOOLONG ; + goto out; + }/* Allocate a buffer appropriate to hold the result */ cryptbuf =kmalloc (len >CEPH_NOHASH_NAME_MAX ?NAME_MAX : len,GFP_KERNEL ); -if (!cryptbuf) -return -ENOMEM ; +if (!cryptbuf) { + elen = -ENOMEM ; + goto out; + } - ret =fscrypt_fname_encrypt (parent, d_name, cryptbuf, len); + ret =fscrypt_fname_encrypt (dir, &iname, cryptbuf, len);if (ret) { -kfree (cryptbuf); -return ret; + elen = ret; + goto out; }/* hash the end if the name is long enough */
@@ -234,12 +322,30 @@ int ceph_encode_encrypted_dname(const struct inode *parent, struct qstr *d_name,/* base64 encode the encrypted name */ elen =ceph_base64_encode (cryptbuf, len, buf); -kfree (cryptbuf);dout ("base64-encoded ciphertext name = %.*s\n" , elen, buf); + +/* To understand the 240 limit, see CEPH_NOHASH_NAME_MAX comments */ +WARN_ON (elen >240 ); +if ((elen >0 ) && (dir != parent)) { +char tmp_buf[NAME_MAX ]; + + elen =snprintf (tmp_buf,sizeof (tmp_buf),"_%.*s_%ld" , + elen, buf, dir->i_ino); +memcpy (buf, tmp_buf, elen); + } + +out: +kfree (cryptbuf); +if (dir != parent) { +if ((dir->i_state &I_NEW )) +discard_new_inode (dir); +else +iput (dir); + }return elen; } -int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf) +int ceph_encode_encrypted_fname (struct inode *parent,struct dentry *dentry,char *buf) {WARN_ON_ONCE (!fscrypt_has_encryption_key (parent));
@@ -264,29 +370,42 @@ int ceph_encode_encrypted_fname(const struct inode *parent, struct dentry *dentrint ceph_fname_to_usr (const struct ceph_fname *fname,struct fscrypt_str *tname,struct fscrypt_str *oname,bool *is_nokey) { -int ret; +struct inode *dir = fname->dir;struct fscrypt_str _tname =FSTR_INIT (NULL ,0 );struct fscrypt_str iname; - -if (!IS_ENCRYPTED (fname->dir)) { - oname->name = fname->name; - oname->len = fname->name_len; -return 0 ; - } +char *name = fname->name; +int name_len = fname->name_len; +int ret;/* Sanity check that the resulting name will fit in the buffer */ if (fname->name_len >NAME_MAX || fname->ctext_len >NAME_MAX )return -EIO ; - ret =__fscrypt_prepare_readdir (fname->dir); +/* Handle the special case of snapshot names that start with '_' */ +if ((ceph_snap (dir) ==CEPH_SNAPDIR ) && (name_len >0 ) && + (name[0 ] =='_' )) { + dir =parse_longname (dir, name, &name_len); +if (IS_ERR (dir)) +return PTR_ERR (dir); + name++;/* skip initial '_' */ + } + +if (!IS_ENCRYPTED (dir)) { + oname->name = fname->name; + oname->len = fname->name_len; + ret =0 ; + goto out_inode; + } + + ret =__fscrypt_prepare_readdir (dir);if (ret) -return ret; + goto out_inode;/* * Use the raw dentry name as sent by the MDS instead of * generating a nokey name via fscrypt. */ -if (!fscrypt_has_encryption_key (fname->dir)) { +if (!fscrypt_has_encryption_key (dir)) {if (fname->no_copy) oname->name = fname->name;else
@@ -294,7 +413,8 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, oname->len = fname->name_len;if (is_nokey) *is_nokey = true; -return 0 ; + ret =0 ; + goto out_inode; }if (fname->ctext_len ==0 ) {
@@ -303,11 +423,11 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname,if (!tname) { ret =fscrypt_fname_alloc_buffer (NAME_MAX , &_tname);if (ret) -return ret; + goto out_inode; tname = &_tname; } - declen =ceph_base64_decode (fname->name, fname->name_len, tname->name); + declen =ceph_base64_decode (name, name_len, tname->name);if (declen <=0 ) { ret = -EIO ; goto out;
@@ -319,9 +439,25 @@ int ceph_fname_to_usr(const struct ceph_fname *fname, struct fscrypt_str *tname, iname.len = fname->ctext_len; } - ret =fscrypt_fname_disk_to_usr (fname->dir,0 ,0 , &iname, oname); + ret =fscrypt_fname_disk_to_usr (dir,0 ,0 , &iname, oname); +if (!ret && (dir != fname->dir)) { +char tmp_buf[CEPH_BASE64_CHARS (NAME_MAX )]; + + name_len =snprintf (tmp_buf,sizeof (tmp_buf),"_%.*s_%ld" , + oname->len, oname->name, dir->i_ino); +memcpy (oname->name, tmp_buf, name_len); + oname->len = name_len; + } + out:fscrypt_fname_free_buffer (&_tname); +out_inode: +if ((dir != fname->dir) && !IS_ERR (dir)) { +if ((dir->i_state &I_NEW )) +discard_new_inode (dir); +else +iput (dir); + }return ret; }
fs/ceph/crypto.h +2 -2
@@ -101,8 +101,8 @@ void ceph_fscrypt_free_dummy_policy(struct ceph_fs_client *fsc);int ceph_fscrypt_prepare_context (struct inode *dir,struct inode *inode,struct ceph_acl_sec_ctx *as);void ceph_fscrypt_as_ctx_to_req (struct ceph_mds_request *req,struct ceph_acl_sec_ctx *as); -int ceph_encode_encrypted_dname (const struct inode *parent,struct qstr *d_name,char *buf); -int ceph_encode_encrypted_fname (const struct inode *parent,struct dentry *dentry,char *buf); +int ceph_encode_encrypted_dname (struct inode *parent,struct qstr *d_name,char *buf); +int ceph_encode_encrypted_fname (struct inode *parent,struct dentry *dentry,char *buf);static inline int ceph_fname_alloc_buffer (struct inode *parent,struct fscrypt_str *fname) {
Signature · 1 line
2.35.1
PATCH v14 63/64 ceph: update documentation regarding snapshot naming limitations
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
--- Documentation/filesystems/ceph.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)
Documentation/filesystems/ceph.rst +10 -0
@@ -57,6 +57,16 @@ a snapshot on any subdirectory (and its nested contents) in the system. Snapshot creation and deletion are as simple as 'mkdir .snap/foo' and 'rmdir .snap/foo'. +Snapshot names have two limitations: + +* They can not start with an underscore ('_'), as these names are reserved + for internal usage by the MDS. +* They can not exceed 240 characters in size. This is because the MDS makes + use of long snapshot names internally, which follow the format: + `_<SNAPSHOT-NAME>_<INODE-NUMBER>`. Since filenames in general can't have + more than 255 characters, and `<node-id>` takes 13 characters, the long + snapshot names can take as much as 255 - 1 - 1 - 13 = 240. + Ceph also provides some recursive accounting on directories for nested files and bytes. That is, a 'getfattr -d foo' on any directory in the system will reveal the total number of nested regular files and
Signature · 1 line
2.35.1
PATCH v14 64/64 ceph: prevent snapshots to be created in encrypted locked directories
In reply to Jeff LaytonFrom: Luís Henriques <[email protected]>
With snapshot names encryption we can not allow snapshots to be created in locked directories because the names wouldn't be encrypted. This patch forces the directory to be unlocked to allow a snapshot to be created.
--- fs/ceph/dir.c | 5 +++++ 1 file changed, 5 insertions(+)
fs/ceph/dir.c +5 -0
@@ -1071,6 +1071,11 @@ static int ceph_mkdir(struct user_namespace *mnt_userns, struct inode *dir, err = -EDQUOT ; goto out; } +if ((op ==CEPH_MDS_OP_MKSNAP ) &&IS_ENCRYPTED (dir) && + !fscrypt_has_encryption_key (dir)) { + err = -ENOKEY ; + goto out; + } req =ceph_mdsc_create_request (mdsc, op,USE_AUTH_MDS );
Signature · 1 line
2.35.1
Jeff,
All the patches looks good to me. I saw you have fixed the "[PATCH v14 58/64]" in the wip-fscrypt branch.
Feel free to add:
Thanks.
-- Xiubo
Quoted 134 lines · Jeff Layton
Yet another ceph+fscrypt posting. The main changes since v13:
- rebased onto v5.18-rc4 + ceph testing branch patches, fixed up minor merge conflicts, and squashed a few patches together
- squashed in a number of patches from Xiubo to fix issues with truncation handling
- incorporated Luís' patches to encrypt snapshot names
- dropped a patch to export fscrypt's base64 implementation, since Luís' snapshot implementation added a new variant that we need to use.
At this point, I'm mainly waiting on Al to merge this patch into -next and eventually into v5.19:
fs: change test in
inode_insert5for adding to the sb listOnce that's in, it should clear the way for us to start merging the rest of the pile (though probably not all at once).
There are also some mysterious failures that have cropped up in testing with teuthology, particularly with thrash testing. We'll need to get to the bottom of that before we can merge the bulk of these patches.
I've updated the wip-fscrypt branch in the ceph tree with this pile for now. Help with testing would be welcome!
Jeff Layton (48):
libceph: add spinlock aroundosd->o_requests
libceph: define structceph_sparse_extentand add some helpers
libceph: add sparse read support to msgr2 crc state machine
libceph: add sparse read support to OSD client
libceph: support sparse reads on msgr2 secure codepath
libceph: add sparse read support to msgr1
ceph: add new mount option to enable sparse reads
fs: change test ininode_insert5for adding to the sb list
fscrypt: exportfscrypt_fname_encryptandfscrypt_fname_encrypted_size
fscrypt: addfscrypt_context_for_new_inode
ceph: preallocate inode for ops that may create one
ceph:fscrypt_authhandling for ceph
ceph: ensure that we accept a new context from MDS for new inodes
ceph: add support for fscrypt_auth/fscrypt_file to cap messages
ceph: implement -otest_dummy_encryptionmount option
ceph: decodealternate_namein lease info
ceph: add fscrypt ioctls
ceph: makeceph_msdc_build_pathuse ref-walk
ceph: add encrypted fname handling toceph_mdsc_build_path
ceph: send altname in MClientRequest
ceph: encode encrypted name in dentry release
ceph: properly setDCACHE_NOKEY_NAMEflag in lookup
ceph: setDCACHE_NOKEY_NAMEin atomic open
ceph: maked_revalidatecall fscrypt revalidator for encrypted
dentries
ceph: add helpers for converting names for userland presentation
ceph: add fscrypt support toceph_fill_trace
ceph: create symlinks with encrypted and base64-encoded targets
ceph: makeceph_get_namedecrypt filenames
ceph: add a new ceph.fscrypt.auth vxattr
ceph: add some fscrypt guardrails
libceph: addCEPH_OSD_OP_ASSERT_VERsupport
ceph: size handling for encrypted inodes in cap updates
ceph:fscrypt_filefield handling in MClientRequest messages
ceph: get file size fromfscrypt_filewhen present in inode traces
ceph: handle fscrypt fields in cap messages from MDS
ceph: updateWARN_ONmessage topr_warn
ceph: add infrastructure for file encryption and decryption
libceph: allowceph_osdc_new_requestto accept a multi-op read
ceph: disable fallocate for encrypted inodes
ceph: disable copy offload on encrypted inodes
ceph: don't use special DIO path for encrypted inodes
ceph: align data in pages inceph_sync_write
ceph: add read/modify/write toceph_sync_write
ceph: plumb in decryption during sync reads
ceph: add fscrypt decryption support toceph_netfs_issue_op
ceph: seti_blkbitsto crypto block size for encrypted inodes
ceph: add encryption support to writepage
ceph: fscrypt support for writepagesLuís Henriques (7):
ceph: add base64 endcoding routines for encrypted names
ceph: don't allow changing layout on encrypted files/directories
ceph: invalidate pages when doing direct/sync writes
ceph: add support for encrypted snapshot names
ceph: add support for handling encrypted snapshot names
ceph: update documentation regarding snapshot naming limitations
ceph: prevent snapshots to be created in encrypted locked directoriesXiubo Li (9):
ceph: make the ioctl cmd more readable in debug log
ceph: fix base64 encoded name's length check inceph_fname_to_usr()
ceph: pass the request toparse_reply_info_readdir()
ceph: addceph_encode_encrypted_dname()helper
ceph: add support to readdir for encrypted filenames
ceph: add__ceph_get_capshelper support
ceph: add__ceph_sync_readhelper support
ceph: add object version support for sync read
ceph: add truncate size handling support for fscryptDocumentation/filesystems/ceph.rst | 10 + fs/ceph/Makefile | 1 + fs/ceph/acl.c | 4 +- fs/ceph/addr.c | 136 ++++-- fs/ceph/caps.c | 219 ++++++++-- fs/ceph/crypto.c | 638 +++++++++++++++++++++++++++++ fs/ceph/crypto.h | 264 ++++++++++++ fs/ceph/dir.c | 187 +++++++-- fs/ceph/export.c | 44 +- fs/ceph/file.c | 598 ++++++++++++++++++++++----- fs/ceph/inode.c | 579 +++++++++++++++++++++++--- fs/ceph/ioctl.c | 126 +++++- fs/ceph/mds_client.c | 465 +++++++++++++++++---- fs/ceph/mds_client.h | 24 +- fs/ceph/super.c | 107 ++++- fs/ceph/super.h | 43 +- fs/ceph/xattr.c | 29 ++ fs/crypto/fname.c | 36 +- fs/crypto/fscrypt_private.h | 9 +- fs/crypto/hooks.c | 6 +- fs/crypto/policy.c | 35 +- fs/inode.c | 11 +- include/linux/ceph/ceph_fs.h | 21 +- include/linux/ceph/messenger.h | 32 ++ include/linux/ceph/osd_client.h | 89 +++- include/linux/ceph/rados.h | 4 + include/linux/fscrypt.h | 5 + net/ceph/messenger.c | 1 + net/ceph/messenger_v1.c | 98 ++++- net/ceph/messenger_v2.c | 287 ++++++++++++- net/ceph/osd_client.c | 306 +++++++++++++- 31 files changed, 4009 insertions(+), 405 deletions(-)create mode 100644
fs/ceph/crypto.c
create mode 100644fs/ceph/crypto.h