Skip to contentSkip to conversation

ext4: inherit encryption xattr before other xattrs

PATCH In review Eric Biggers 3 participants 6 messages Feb 17, 2017 – May 2, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ext4: inherit encryption xattr before other xattrs

From: Eric Biggers <[email protected]>

When using both encryption and SELinux (or another feature that requires an xattr per file) on a filesystem with 256-byte inodes, each file's xattrs usually spill into an external xattr block. Currently, the xattrs are inherited in the order ACL, security, then encryption. Therefore, if spillage occurs, the encryption xattr will always end up in the external block. This is not ideal because the encryption xattrs contain a nonce, so they will always be unique and will prevent the external xattr blocks from being deduplicated.

To improve the situation, change the inheritance order to encryption, ACL, then security. This gives the encryption xattr a better chance to be stored in-inode, allowing the other xattr(s) to be deduplicated.

Note that it may be better for userspace to format the filesystem with 512-byte inodes in this case. However, it's not the default.

---
 fs/ext4/ialloc.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
fs/ext4/ialloc.c +11 -6
index b14bae2598bc..0304e28c2014 100644
@@ -1096,6 +1096,17 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir,
 	if (err)
 		goto fail_drop;
 
+	/*
+	 * Since the encryption xattr will always be unique, create it first so
+	 * that it's less likely to end up in an external xattr block and
+	 * prevent its deduplication.
+	 */
+	if (encrypt) {
+		err = fscrypt_inherit_context(dir, inode, handle, true);
+		if (err)
+			goto fail_free_drop;
+	}
+
 	err = ext4_init_acl(handle, inode, dir);
 	if (err)
 		goto fail_free_drop;
@@ -1117,12 +1128,6 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir,
 		ei->i_datasync_tid = handle->h_transaction->t_tid;
 	}
 
-	if (encrypt) {
-		err = fscrypt_inherit_context(dir, inode, handle, true);
-		if (err)
-			goto fail_free_drop;
-	}
-
 	err = ext4_mark_inode_dirty(handle, inode);
 	if (err) {
 		ext4_std_error(sb, err);
Signature · 1 line
2.11.0.483.g087da7b7c-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 12 lines · Eric Biggers

From: Eric Biggers <[email protected]>

When using both encryption and SELinux (or another feature that requires an xattr per file) on a filesystem with 256-byte inodes, each file's xattrs usually spill into an external xattr block. Currently, the xattrs are inherited in the order ACL, security, then encryption. Therefore, if spillage occurs, the encryption xattr will always end up in the external block. This is not ideal because the encryption xattrs contain a nonce, so they will always be unique and will prevent the external xattr blocks from being deduplicated.

Quoted 8 lines

To improve the situation, change the inheritance order to encryption, ACL, then security. This gives the encryption xattr a better chance to be stored in-inode, allowing the other xattr(s) to be deduplicated.

Note that it may be better for userspace to format the filesystem with 512-byte inodes in this case. However, it's not the default.

Note that we've been using 512-byte xattrs for Lustre metadata servers for ages. We may want to consider enabling this by default when the filesystem features are set (e.g. crypto, inline data, etc).

Having a general mechanism to bias xattrs to in-inode or external xattr storage would be nice also, but I don't know any way to do this beyond just having a list of xattr names and then prioritizing the ones that go into the in-inode space.

Cheers, Andreas

Quoted 40 lines

---
fs/ext4/ialloc.c | 17 +++++++++++------

1 file changed, 11 insertions(+), 6 deletions(-)
fs/ext4/ialloc.c +11 -7
index b14bae2598bc..0304e28c2014 100644
@@ -1096,6 +1096,17 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir,
 	if (err)
 		goto fail_drop;
 
+	/*
+	 * Since the encryption xattr will always be unique, create it first so
+	 * that it's less likely to end up in an external xattr block and
+	 * prevent its deduplication.
+	 */
+	if (encrypt) {
+		err = fscrypt_inherit_context(dir, inode, handle, true);
+		if (err)
+			goto fail_free_drop;
+	}
+
 	err = ext4_init_acl(handle, inode, dir);
 	if (err)
 		goto fail_free_drop;
@@ -1117,12 +1128,6 @@ struct inode *__ext4_new_inode(handle_t *handle, struct inode *dir,
 		ei->i_datasync_tid = handle->h_transaction->t_tid;
 	}
 
-	if (encrypt) {
-		err = fscrypt_inherit_context(dir, inode, handle, true);
-		if (err)
-			goto fail_free_drop;
-	}
-
 	err = ext4_mark_inode_dirty(handle, inode);
 	if (err) {
 		ext4_std_error(sb, err);
--
Quoted 2 lines

2.11.0.483.g087da7b7c-goog

Cheers, Andreas

#2 lore raw
Quoted 24 lines · Andreas Dilger
Quoted 12 lines · Eric Biggers

From: Eric Biggers <[email protected]>

When using both encryption and SELinux (or another feature that requires an xattr per file) on a filesystem with 256-byte inodes, each file's xattrs usually spill into an external xattr block. Currently, the xattrs are inherited in the order ACL, security, then encryption. Therefore, if spillage occurs, the encryption xattr will always end up in the external block. This is not ideal because the encryption xattrs contain a nonce, so they will always be unique and will prevent the external xattr blocks from being deduplicated.

Quoted 8 lines

To improve the situation, change the inheritance order to encryption, ACL, then security. This gives the encryption xattr a better chance to be stored in-inode, allowing the other xattr(s) to be deduplicated.

Note that it may be better for userspace to format the filesystem with 512-byte inodes in this case. However, it's not the default.

Quoted 9 lines

Note that we've been using 512-byte xattrs for Lustre metadata servers for ages. We may want to consider enabling this by default when the filesystem features are set (e.g. crypto, inline data, etc).

Having a general mechanism to bias xattrs to in-inode or external xattr storage would be nice also, but I don't know any way to do this beyond just having a list of xattr names and then prioritizing the ones that go into the in-inode space.

I think it's a good idea to have mke2fs default to 512-byte inodes if '-O inline_data' is specified. But with '-O encrypt' it may be more debatable because there is no guarantee as to how many files userspace will actually choose to encrypt. It could be almost the whole filesystem, or just a few files, or even nothing at all.

Regardless, I think adjusting the xattr inheritance order (as this patch does) has advantages but no real disadvantages, so it might as well be done too.

Eric

#3 lore raw
Quoted 36 lines · Eric Biggers
Quoted 24 lines · Andreas Dilger
Quoted 12 lines · Eric Biggers

From: Eric Biggers <[email protected]>

When using both encryption and SELinux (or another feature that requires an xattr per file) on a filesystem with 256-byte inodes, each file's xattrs usually spill into an external xattr block. Currently, the xattrs are inherited in the order ACL, security, then encryption. Therefore, if spillage occurs, the encryption xattr will always end up in the external block. This is not ideal because the encryption xattrs contain a nonce, so they will always be unique and will prevent the external xattr blocks from being deduplicated.

Quoted 8 lines

To improve the situation, change the inheritance order to encryption, ACL, then security. This gives the encryption xattr a better chance to be stored in-inode, allowing the other xattr(s) to be deduplicated.

Note that it may be better for userspace to format the filesystem with 512-byte inodes in this case. However, it's not the default.

Quoted 9 lines

Note that we've been using 512-byte xattrs for Lustre metadata servers for ages. We may want to consider enabling this by default when the filesystem features are set (e.g. crypto, inline data, etc).

Having a general mechanism to bias xattrs to in-inode or external xattr storage would be nice also, but I don't know any way to do this beyond just having a list of xattr names and then prioritizing the ones that go into the in-inode space.

Quoted 8 lines

I think it's a good idea to have mke2fs default to 512-byte inodes if '-O inline_data' is specified. But with '-O encrypt' it may be more debatable because there is no guarantee as to how many files userspace will actually choose to encrypt. It could be almost the whole filesystem, or just a few files, or even nothing at all.

Regardless, I think adjusting the xattr inheritance order (as this patch does) has advantages but no real disadvantages, so it might as well be done too.

Sorry if I wasn't clear. I have no objections to this patch at all, and would be happy to see it land. My comments were just for related improvements that could be done in different patches.

Cheers, Andreas

#4 lore raw
In reply to Eric Biggers
Quoted 41 lines · Eric Biggers
Quoted 24 lines · Andreas Dilger
Quoted 12 lines · Eric Biggers

From: Eric Biggers <[email protected]>

When using both encryption and SELinux (or another feature that requires an xattr per file) on a filesystem with 256-byte inodes, each file's xattrs usually spill into an external xattr block. Currently, the xattrs are inherited in the order ACL, security, then encryption. Therefore, if spillage occurs, the encryption xattr will always end up in the external block. This is not ideal because the encryption xattrs contain a nonce, so they will always be unique and will prevent the external xattr blocks from being deduplicated.

Quoted 8 lines

To improve the situation, change the inheritance order to encryption, ACL, then security. This gives the encryption xattr a better chance to be stored in-inode, allowing the other xattr(s) to be deduplicated.

Note that it may be better for userspace to format the filesystem with 512-byte inodes in this case. However, it's not the default.

Quoted 9 lines

Note that we've been using 512-byte xattrs for Lustre metadata servers for ages. We may want to consider enabling this by default when the filesystem features are set (e.g. crypto, inline data, etc).

Having a general mechanism to bias xattrs to in-inode or external xattr storage would be nice also, but I don't know any way to do this beyond just having a list of xattr names and then prioritizing the ones that go into the in-inode space.

I think it's a good idea to have mke2fs default to 512-byte inodes if '-O inline_data' is specified. But with '-O encrypt' it may be more debatable because there is no guarantee as to how many files userspace will actually choose to encrypt. It could be almost the whole filesystem, or just a few files, or even nothing at all.

Quoted 4 lines

Regardless, I think adjusting the xattr inheritance order (as this patch does) has advantages but no real disadvantages, so it might as well be done too.

Eric

Ted, this patch seems to have gotten missed; are you planning to apply it? Next cycle is fine too, but I'd like to get it in sometime.

- Eric

maintainer
#5 lore raw
Quoted 3 lines · Eric Biggers

Ted, this patch seems to have gotten missed; are you planning to apply it? Next cycle is fine too, but I'd like to get it in sometime.

Yes, I had missed it. Since it's simple and low-risk, I'll include it this cycle; it's in the dev branch now. Thanks for reminding me!

- Ted