Skip to contentSkip to conversation

fscrypt, fs-verity: one-time init fixes

PATCH In review Eric Biggers 1 participant 7 messages Jul 21, 2020 – Jul 27, 2020

Subsystems (3)
lore
Share & tools
lore

Eric Biggers

author maintainer
#0 lore raw

cover fscrypt, fs-verity: one-time init fixes

This series fixes up some cases in fs/crypto/ and fs/verity/ where "one-time init" is implemented using READ_ONCE() instead of smp_load_acquire() but it's not obviously correct.

One case is fixed by using a better approach that removes the need to initialize anything. The others are fixed by upgrading READ_ONCE() to smp_load_acquire(). I've also improved the comments.

This is motivated by the discussions at
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u
and
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u

These fixes are improvements over the status quo, so I'd prefer to apply them now, without waiting for any potential new generic one-time-init macros (which based on the latest discussion, won't be flexible enough to handle most of these cases anyway).

Eric Biggers (5):
fscrypt: switch fscrypt_do_sha256() to use the SHA-256 library
fscrypt: use smp_load_acquire() for fscrypt_prepared_key
fscrypt: use smp_load_acquire() for ->s_master_keys
fscrypt: use smp_load_acquire() for ->i_crypt_info
fs-verity: use smp_load_acquire() for ->i_verity_info

 fs/crypto/Kconfig           |  2 +-
 fs/crypto/fname.c           | 41 +++++++++----------------------------
 fs/crypto/fscrypt_private.h | 15 ++++++++------
 fs/crypto/inline_crypt.c    |  6 ++++--
 fs/crypto/keyring.c         | 15 +++++++++++---
 fs/crypto/keysetup.c        | 18 +++++++++++++---
 fs/crypto/policy.c          |  4 ++--
 fs/verity/open.c            | 15 +++++++++++---
 include/linux/fscrypt.h     | 29 +++++++++++++++++++++-----
 include/linux/fsverity.h    |  9 ++++++--
 10 files changed, 96 insertions(+), 58 deletions(-)
Signature · 1 line
2.27.0
author maintainer
#1 lore raw

PATCH 1/5 fscrypt: switch fscrypt_do_sha256() to use the SHA-256 library

From: Eric Biggers <[email protected]>

fscrypt_do_sha256() is only used for hashing encrypted filenames to create no-key tokens, which isn't performance-critical. Therefore a C implementation of SHA-256 is sufficient.

Also, the logic to create no-key tokens is always potentially needed. This differs from fscrypt's other dependencies on crypto API algorithms, which are conditionally needed depending on what encryption policies userspace is using. Therefore, for fscrypt there isn't much benefit to allowing SHA-256 to be a loadable module.

So, make fscrypt_do_sha256() use the SHA-256 library instead of the crypto_shash API. This is much simpler, since it avoids having to implement one-time-init (which is hard to do correctly, and in fact was implemented incorrectly) and handle failures to allocate the crypto_shash object.

Fixes edc440e3d27f ("fscrypt: improve format of no-key names")
Cc Daniel Rosenberg <[email protected]>
---
 fs/crypto/Kconfig |  2 +-
 fs/crypto/fname.c | 41 ++++++++++-------------------------------
 2 files changed, 11 insertions(+), 32 deletions(-)
fs/crypto/Kconfig +1 -1
index f1f11a6228eb..a5f5c30368a2 100644
@@ -4,6 +4,7 @@ config FS_ENCRYPTION
 	select CRYPTO
 	select CRYPTO_HASH
 	select CRYPTO_SKCIPHER
+	select CRYPTO_LIB_SHA256
 	select KEYS
 	help
 	  Enable encryption of files and directories.  This
@@ -21,7 +22,6 @@ config FS_ENCRYPTION_ALGS
 	select CRYPTO_CTS
 	select CRYPTO_ECB
 	select CRYPTO_HMAC
-	select CRYPTO_SHA256
 	select CRYPTO_SHA512
 	select CRYPTO_XTS
 
fs/crypto/fname.c +10 -31

67 lines not on this page · raw message

Signature · 1 line
2.27.0
author maintainer
#4 lore raw

PATCH 4/5 fscrypt: use smp_load_acquire() for ->i_crypt_info

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.

fscrypt_info includes various sub-objects which are internal to and are allocated by other kernel subsystems such as keyrings and crypto. So by using READ_ONCE() for ->i_crypt_info, we're relying on internal implementation details of these other kernel subsystems.

Remove this fragile assumption by using smp_load_acquire() instead.

(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)

Fixes e37a784d8b6a ("fscrypt: use READ_ONCE() to access ->i_crypt_info")
---
 fs/crypto/keysetup.c    | 12 +++++++++++-
 fs/crypto/policy.c      |  4 ++--
 include/linux/fscrypt.h | 29 ++++++++++++++++++++++++-----
 3 files changed, 37 insertions(+), 8 deletions(-)
fs/crypto/keysetup.c +11 -1
index 7f85fc645602..fea6226afc2b 100644
@@ -518,7 +518,17 @@ int fscrypt_get_encryption_info(struct inode *inode)
 	if (res)
 		goto out;
 
+	/*
+	 * Multiple tasks may race to set ->i_crypt_info, so use
+	 * cmpxchg_release().  This pairs with the smp_load_acquire() in
+	 * fscrypt_get_info().  I.e., here we publish ->i_crypt_info with a
+	 * RELEASE barrier so that other tasks can ACQUIRE it.
+	 */
 	if (cmpxchg_release(&inode->i_crypt_info, NULL, crypt_info) == NULL) {
+		/*
+		 * We won the race and set ->i_crypt_info to our crypt_info.
+		 * Now link it into the master key's inode list.
+		 */
 		if (master_key) {
 			struct fscrypt_master_key *mk =
 				master_key->payload.data[0];
@@ -589,7 +599,7 @@ EXPORT_SYMBOL(fscrypt_free_inode);
  */
 int fscrypt_drop_inode(struct inode *inode)
 {
-	const struct fscrypt_info *ci = READ_ONCE(inode->i_crypt_info);
+	const struct fscrypt_info *ci = fscrypt_get_info(inode);
 	const struct fscrypt_master_key *mk;
 
 	/*
fs/crypto/policy.c +2 -2
index 8a8ad0e44bb8..2a2d0c06147b 100644
@@ -338,7 +338,7 @@ static int fscrypt_get_policy(struct inode *inode, union fscrypt_policy *policy)
 	union fscrypt_context ctx;
 	int ret;
 
-	ci = READ_ONCE(inode->i_crypt_info);
+	ci = fscrypt_get_info(inode);
 	if (ci) {
 		/* key available, use the cached policy */
 		*policy = ci->ci_policy;
@@ -627,7 +627,7 @@ int fscrypt_inherit_context(struct inode *parent, struct inode *child,
 	if (res < 0)
 		return res;
 
-	ci = READ_ONCE(parent->i_crypt_info);
+	ci = fscrypt_get_info(parent);
 	if (ci == NULL)
 		return -ENOKEY;
 
include/linux/fscrypt.h +24 -5
index bb257411365f..991ff8575d0e 100644
@@ -74,10 +74,15 @@ struct fscrypt_operations {
 			    struct request_queue **devs);
 };
 
-static inline bool fscrypt_has_encryption_key(const struct inode *inode)
+static inline struct fscrypt_info *fscrypt_get_info(const struct inode *inode)
 {
-	/* pairs with cmpxchg_release() in fscrypt_get_encryption_info() */
-	return READ_ONCE(inode->i_crypt_info) != NULL;
+	/*
+	 * Pairs with the cmpxchg_release() in fscrypt_get_encryption_info().
+	 * I.e., another task may publish ->i_crypt_info concurrently, executing
+	 * a RELEASE barrier.  We need to use smp_load_acquire() here to safely
+	 * ACQUIRE the memory the other task published.
+	 */
+	return smp_load_acquire(&inode->i_crypt_info);
 }
 
 /**
@@ -234,9 +239,9 @@ static inline void fscrypt_set_ops(struct super_block *sb,
 }
 #else  /* !CONFIG_FS_ENCRYPTION */
 
-static inline bool fscrypt_has_encryption_key(const struct inode *inode)
+static inline struct fscrypt_info *fscrypt_get_info(const struct inode *inode)
 {
-	return false;
+	return NULL;
 }
 
 static inline bool fscrypt_needs_contents_encryption(const struct inode *inode)
@@ -619,6 +624,20 @@ static inline bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode)
 	       !__fscrypt_inode_uses_inline_crypto(inode);
 }
 
+/**
+ * fscrypt_has_encryption_key() - check whether an inode has had its key set up
+ * @inode: the inode to check
+ *
+ * Return: %true if the inode has had its encryption key set up, else %false.
+ *
+ * Usually this should be preceded by fscrypt_get_encryption_info() to try to
+ * set up the key first.
+ */
+static inline bool fscrypt_has_encryption_key(const struct inode *inode)
+{
+	return fscrypt_get_info(inode) != NULL;
+}
+
 /**
  * fscrypt_require_key() - require an inode's encryption key
  * @inode: the inode we need the key for
Signature · 1 line
2.27.0
author maintainer
#2 lore raw

PATCH 2/5 fscrypt: use smp_load_acquire() for fscrypt_prepared_key

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.

fscrypt_prepared_key includes a pointer to a crypto_skcipher object, which is internal to and is allocated by the crypto subsystem. By using READ_ONCE() for it, we're relying on internal implementation details of the crypto subsystem.

Remove this fragile assumption by using smp_load_acquire() instead.

(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)

Fixes 5fee36095cda ("fscrypt: add inline encryption support")
Cc Satya Tangirala <[email protected]>
---
 fs/crypto/fscrypt_private.h | 15 +++++++++------
 fs/crypto/inline_crypt.c    |  6 ++++--
 fs/crypto/keysetup.c        |  6 ++++--
 3 files changed, 17 insertions(+), 10 deletions(-)
fs/crypto/fscrypt_private.h +9 -6
index bc1a3fcd45ed..8117a61b6f55 100644
@@ -351,13 +351,16 @@ fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
 			const struct fscrypt_info *ci)
 {
 	/*
-	 * The READ_ONCE() here pairs with the smp_store_release() in
-	 * fscrypt_prepare_key().  (This only matters for the per-mode keys,
-	 * which are shared by multiple inodes.)
+	 * The two smp_load_acquire()'s here pair with the smp_store_release()'s
+	 * in fscrypt_prepare_inline_crypt_key() and fscrypt_prepare_key().
+	 * I.e., in some cases (namely, if this prep_key is a per-mode
+	 * encryption key) another task can publish blk_key or tfm concurrently,
+	 * executing a RELEASE barrier.  We need to use smp_load_acquire() here
+	 * to safely ACQUIRE the memory the other task published.
 	 */
 	if (fscrypt_using_inline_encryption(ci))
-		return READ_ONCE(prep_key->blk_key) != NULL;
-	return READ_ONCE(prep_key->tfm) != NULL;
+		return smp_load_acquire(&prep_key->blk_key) != NULL;
+	return smp_load_acquire(&prep_key->tfm) != NULL;
 }
 
 #else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
@@ -391,7 +394,7 @@ static inline bool
 fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
 			const struct fscrypt_info *ci)
 {
-	return READ_ONCE(prep_key->tfm) != NULL;
+	return smp_load_acquire(&prep_key->tfm) != NULL;
 }
 #endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
 
fs/crypto/inline_crypt.c +4 -2
index d7aecadf33c1..dfb06375099a 100644
@@ -176,8 +176,10 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
 		}
 	}
 	/*
-	 * Pairs with READ_ONCE() in fscrypt_is_key_prepared().  (Only matters
-	 * for the per-mode keys, which are shared by multiple inodes.)
+	 * Pairs with the smp_load_acquire() in fscrypt_is_key_prepared().
+	 * I.e., here we publish ->blk_key with a RELEASE barrier so that
+	 * concurrent tasks can ACQUIRE it.  Note that this concurrency is only
+	 * possible for per-mode keys, not for per-file keys.
 	 */
 	smp_store_release(&prep_key->blk_key, blk_key);
 	return 0;
fs/crypto/keysetup.c +4 -2
index 22a94b18fe70..7f85fc645602 100644
@@ -129,8 +129,10 @@ int fscrypt_prepare_key(struct fscrypt_prepared_key *prep_key,
 	if (IS_ERR(tfm))
 		return PTR_ERR(tfm);
 	/*
-	 * Pairs with READ_ONCE() in fscrypt_is_key_prepared().  (Only matters
-	 * for the per-mode keys, which are shared by multiple inodes.)
+	 * Pairs with the smp_load_acquire() in fscrypt_is_key_prepared().
+	 * I.e., here we publish ->tfm with a RELEASE barrier so that
+	 * concurrent tasks can ACQUIRE it.  Note that this concurrency is only
+	 * possible for per-mode keys, not for per-file keys.
 	 */
 	smp_store_release(&prep_key->tfm, tfm);
 	return 0;
Signature · 1 line
2.27.0
author maintainer
#5 lore raw

PATCH 5/5 fs-verity: use smp_load_acquire() for ->i_verity_info

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.

fsverity_info::tree_params.hash_alg->tfm is a crypto_ahash object that's internal to and is allocated by the crypto subsystem. So by using READ_ONCE() for ->i_verity_info, we're relying on internal implementation details of the crypto subsystem.

Remove this fragile assumption by using smp_load_acquire() instead.

Also fix the cmpxchg logic to correctly execute an ACQUIRE barrier when losing the cmpxchg race, since cmpxchg doesn't guarantee a memory barrier on failure.

(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)

Fixes fd2d1acfcadf ("fs-verity: add the hook for file ->open()")
---
 fs/verity/open.c         | 15 ++++++++++++---
 include/linux/fsverity.h |  9 +++++++--
 2 files changed, 19 insertions(+), 5 deletions(-)
fs/verity/open.c +12 -3
index d007db0c9304..bfe0280c14e4 100644
@@ -221,11 +221,20 @@ struct fsverity_info *fsverity_create_info(const struct inode *inode,
 void fsverity_set_info(struct inode *inode, struct fsverity_info *vi)
 {
 	/*
-	 * Multiple processes may race to set ->i_verity_info, so use cmpxchg.
-	 * This pairs with the READ_ONCE() in fsverity_get_info().
+	 * Multiple tasks may race to set ->i_verity_info, so use
+	 * cmpxchg_release().  This pairs with the smp_load_acquire() in
+	 * fsverity_get_info().  I.e., here we publish ->i_verity_info with a
+	 * RELEASE barrier so that other tasks can ACQUIRE it.
 	 */
-	if (cmpxchg(&inode->i_verity_info, NULL, vi) != NULL)
+	if (cmpxchg_release(&inode->i_verity_info, NULL, vi) != NULL) {
+		/* Lost the race, so free the fsverity_info we allocated. */
 		fsverity_free_info(vi);
+		/*
+		 * Afterwards, the caller may access ->i_verity_info directly,
+		 * so make sure to ACQUIRE the winning fsverity_info.
+		 */
+		(void)fsverity_get_info(inode);
+	}
 }
 
 void fsverity_free_info(struct fsverity_info *vi)
include/linux/fsverity.h +7 -2
index 78201a6d35f6..c1144a450392 100644
@@ -115,8 +115,13 @@ struct fsverity_operations {
 
 static inline struct fsverity_info *fsverity_get_info(const struct inode *inode)
 {
-	/* pairs with the cmpxchg() in fsverity_set_info() */
-	return READ_ONCE(inode->i_verity_info);
+	/*
+	 * Pairs with the cmpxchg_release() in fsverity_set_info().
+	 * I.e., another task may publish ->i_verity_info concurrently,
+	 * executing a RELEASE barrier.  We need to use smp_load_acquire() here
+	 * to safely ACQUIRE the memory the other task published.
+	 */
+	return smp_load_acquire(&inode->i_verity_info);
 }
 
 /* enable.c */
Signature · 1 line
2.27.0
author maintainer
#3 lore raw

PATCH 3/5 fscrypt: use smp_load_acquire() for ->s_master_keys

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.

super_block::s_master_keys is a keyring, which is internal to and is allocated by the keyrings subsystem. By using READ_ONCE() for it, we're relying on internal implementation details of the keyrings subsystem.

Remove this fragile assumption by using smp_load_acquire() instead.

(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)

Fixes 22d94f493bfb ("fscrypt: add FS_IOC_ADD_ENCRYPTION_KEY ioctl")
---
 fs/crypto/keyring.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
fs/crypto/keyring.c +12 -3
index 7f8ac61a20d6..71d56f8e2870 100644
@@ -213,7 +213,11 @@ static int allocate_filesystem_keyring(struct super_block *sb)
 	if (IS_ERR(keyring))
 		return PTR_ERR(keyring);
 
-	/* Pairs with READ_ONCE() in fscrypt_find_master_key() */
+	/*
+	 * Pairs with the smp_load_acquire() in fscrypt_find_master_key().
+	 * I.e., here we publish ->s_master_keys with a RELEASE barrier so that
+	 * concurrent tasks can ACQUIRE it.
+	 */
 	smp_store_release(&sb->s_master_keys, keyring);
 	return 0;
 }
@@ -234,8 +238,13 @@ struct key *fscrypt_find_master_key(struct super_block *sb,
 	struct key *keyring;
 	char description[FSCRYPT_MK_DESCRIPTION_SIZE];
 
-	/* pairs with smp_store_release() in allocate_filesystem_keyring() */
-	keyring = READ_ONCE(sb->s_master_keys);
+	/*
+	 * Pairs with the smp_store_release() in allocate_filesystem_keyring().
+	 * I.e., another task can publish ->s_master_keys concurrently,
+	 * executing a RELEASE barrier.  We need to use smp_load_acquire() here
+	 * to safely ACQUIRE the memory the other task published.
+	 */
+	keyring = smp_load_acquire(&sb->s_master_keys);
 	if (keyring == NULL)
 		return ERR_PTR(-ENOKEY); /* No keyring yet, so no keys yet. */
 
Signature · 1 line
2.27.0
author maintainer
#6 lore raw
In reply to Eric Biggers
Quoted 25 lines · Eric Biggers

This series fixes up some cases in fs/crypto/ and fs/verity/ where "one-time init" is implemented using READ_ONCE() instead of smp_load_acquire() but it's not obviously correct.

One case is fixed by using a better approach that removes the need to initialize anything. The others are fixed by upgrading READ_ONCE() to smp_load_acquire(). I've also improved the comments.

This is motivated by the discussions at
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u
and
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u

These fixes are improvements over the status quo, so I'd prefer to apply them now, without waiting for any potential new generic one-time-init macros (which based on the latest discussion, won't be flexible enough to handle most of these cases anyway).

Eric Biggers (5):
fscrypt: switch fscrypt_do_sha256() to use the SHA-256 library
fscrypt: use smp_load_acquire() for fscrypt_prepared_key
fscrypt: use smp_load_acquire() for ->s_master_keys
fscrypt: use smp_load_acquire() for ->i_crypt_info
fs-verity: use smp_load_acquire() for ->i_verity_info

Quoted 11 lines
 fs/crypto/Kconfig           |  2 +-
 fs/crypto/fname.c           | 41 +++++++++----------------------------
 fs/crypto/fscrypt_private.h | 15 ++++++++------
 fs/crypto/inline_crypt.c    |  6 ++++--
 fs/crypto/keyring.c         | 15 +++++++++++---
 fs/crypto/keysetup.c        | 18 +++++++++++++---
 fs/crypto/policy.c          |  4 ++--
 fs/verity/open.c            | 15 +++++++++++---
 include/linux/fscrypt.h     | 29 +++++++++++++++++++++-----
 include/linux/fsverity.h    |  9 ++++++--
 10 files changed, 96 insertions(+), 58 deletions(-)

Patches 1-4 applied to fscrypt.git#master for 5.9. Patch 5 applied to fscrypt.git#fsverity for 5.9.

- Eric