fscrypt, fs-verity: one-time init fixes
Subsystems (3)
Eric Biggers
maintainercover fscrypt, fs-verity: one-time init fixes
This series fixes up some cases in fs/crypto/ and fs/verity/ where "one-time init" is implemented using READ_ONCE() instead of smp_load_acquire() but it's not obviously correct.
One case is fixed by using a better approach that removes the need to initialize anything. The others are fixed by upgrading READ_ONCE() to smp_load_acquire(). I've also improved the comments.
This is motivated by the discussions at
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u
and
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u
These fixes are improvements over the status quo, so I'd prefer to apply them now, without waiting for any potential new generic one-time-init macros (which based on the latest discussion, won't be flexible enough to handle most of these cases anyway).
Eric Biggers (5):
fscrypt: switch fscrypt_do_sha256() to use the SHA-256 library
fscrypt: use smp_load_acquire() for fscrypt_prepared_key
fscrypt: use smp_load_acquire() for ->s_master_keys
fscrypt: use smp_load_acquire() for ->i_crypt_info
fs-verity: use smp_load_acquire() for ->i_verity_info
fs/crypto/Kconfig | 2 +- fs/crypto/fname.c | 41 +++++++++---------------------------- fs/crypto/fscrypt_private.h | 15 ++++++++------ fs/crypto/inline_crypt.c | 6 ++++-- fs/crypto/keyring.c | 15 +++++++++++--- fs/crypto/keysetup.c | 18 +++++++++++++--- fs/crypto/policy.c | 4 ++-- fs/verity/open.c | 15 +++++++++++--- include/linux/fscrypt.h | 29 +++++++++++++++++++++----- include/linux/fsverity.h | 9 ++++++-- 10 files changed, 96 insertions(+), 58 deletions(-)
Signature · 1 line
2.27.0
Eric Biggers
maintainerPATCH 1/5 fscrypt: switch fscrypt_do_sha256() to use the SHA-256 library
From: Eric Biggers <[email protected]>
fscrypt_do_sha256() is only used for hashing encrypted filenames to create no-key tokens, which isn't performance-critical. Therefore a C implementation of SHA-256 is sufficient.
Also, the logic to create no-key tokens is always potentially needed. This differs from fscrypt's other dependencies on crypto API algorithms, which are conditionally needed depending on what encryption policies userspace is using. Therefore, for fscrypt there isn't much benefit to allowing SHA-256 to be a loadable module.
So, make fscrypt_do_sha256() use the SHA-256 library instead of the crypto_shash API. This is much simpler, since it avoids having to implement one-time-init (which is hard to do correctly, and in fact was implemented incorrectly) and handle failures to allocate the crypto_shash object.
--- fs/crypto/Kconfig | 2 +- fs/crypto/fname.c | 41 ++++++++++------------------------------- 2 files changed, 11 insertions(+), 32 deletions(-)
fs/crypto/Kconfig +1 -1
@@ -4,6 +4,7 @@ config FS_ENCRYPTION select CRYPTO select CRYPTO_HASH select CRYPTO_SKCIPHER + select CRYPTO_LIB_SHA256 select KEYS help Enable encryption of files and directories. This
@@ -21,7 +22,6 @@ config FS_ENCRYPTION_ALGS select CRYPTO_CTS select CRYPTO_ECB select CRYPTO_HMAC - select CRYPTO_SHA256 select CRYPTO_SHA512 select CRYPTO_XTS
fs/crypto/fname.c +10 -31
67 lines not on this page · raw message
Signature · 1 line
2.27.0
Eric Biggers
maintainerPATCH 2/5 fscrypt: use smp_load_acquire() for fscrypt_prepared_key
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.
fscrypt_prepared_key includes a pointer to a crypto_skcipher object, which is internal to and is allocated by the crypto subsystem. By using READ_ONCE() for it, we're relying on internal implementation details of the crypto subsystem.
Remove this fragile assumption by using smp_load_acquire() instead.
(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)
--- fs/crypto/fscrypt_private.h | 15 +++++++++------ fs/crypto/inline_crypt.c | 6 ++++-- fs/crypto/keysetup.c | 6 ++++-- 3 files changed, 17 insertions(+), 10 deletions(-)
fs/crypto/fscrypt_private.h +9 -6
@@ -351,13 +351,16 @@ fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key, const struct fscrypt_info *ci) {/* -* The READ_ONCE() here pairs with the smp_store_release() in -* fscrypt_prepare_key(). (This only matters for the per-mode keys, -* which are shared by multiple inodes.) +* The two smp_load_acquire()'s here pair with the smp_store_release()'s +* in fscrypt_prepare_inline_crypt_key() and fscrypt_prepare_key(). +* I.e., in some cases (namely, if this prep_key is a per-mode +* encryption key) another task can publish blk_key or tfm concurrently, +* executing a RELEASE barrier. We need to use smp_load_acquire() here +* to safely ACQUIRE the memory the other task published. */ if (fscrypt_using_inline_encryption (ci)) -return READ_ONCE (prep_key->blk_key) !=NULL ; -return READ_ONCE (prep_key->tfm) !=NULL ; +return smp_load_acquire (&prep_key->blk_key) !=NULL ; +return smp_load_acquire (&prep_key->tfm) !=NULL ; }#else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
@@ -391,7 +394,7 @@ static inline bool fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key, const struct fscrypt_info *ci) { - return READ_ONCE(prep_key->tfm) != NULL; + return smp_load_acquire(&prep_key->tfm) != NULL; }#endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
fs/crypto/inline_crypt.c +4 -2
@@ -176,8 +176,10 @@ int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key, } }/* -* Pairs with READ_ONCE() in fscrypt_is_key_prepared(). (Only matters -* for the per-mode keys, which are shared by multiple inodes.) +* Pairs with the smp_load_acquire() in fscrypt_is_key_prepared(). +* I.e., here we publish ->blk_key with a RELEASE barrier so that +* concurrent tasks can ACQUIRE it. Note that this concurrency is only +* possible for per-mode keys, not for per-file keys. */ smp_store_release (&prep_key->blk_key, blk_key);return 0 ;
fs/crypto/keysetup.c +4 -2
@@ -129,8 +129,10 @@ int fscrypt_prepare_key(struct fscrypt_prepared_key *prep_key,if (IS_ERR (tfm))return PTR_ERR (tfm);/* -* Pairs with READ_ONCE() in fscrypt_is_key_prepared(). (Only matters -* for the per-mode keys, which are shared by multiple inodes.) +* Pairs with the smp_load_acquire() in fscrypt_is_key_prepared(). +* I.e., here we publish ->tfm with a RELEASE barrier so that +* concurrent tasks can ACQUIRE it. Note that this concurrency is only +* possible for per-mode keys, not for per-file keys. */ smp_store_release (&prep_key->tfm, tfm);return 0 ;
Signature · 1 line
2.27.0
Eric Biggers
maintainerPATCH 3/5 fscrypt: use smp_load_acquire() for ->s_master_keys
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.
super_block::s_master_keys is a keyring, which is internal to and is allocated by the keyrings subsystem. By using READ_ONCE() for it, we're relying on internal implementation details of the keyrings subsystem.
Remove this fragile assumption by using smp_load_acquire() instead.
(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)
--- fs/crypto/keyring.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
fs/crypto/keyring.c +12 -3
@@ -213,7 +213,11 @@ static int allocate_filesystem_keyring(struct super_block *sb)if (IS_ERR (keyring))return PTR_ERR (keyring); -/* Pairs with READ_ONCE() in fscrypt_find_master_key() */ +/* +* Pairs with the smp_load_acquire() in fscrypt_find_master_key(). +* I.e., here we publish ->s_master_keys with a RELEASE barrier so that +* concurrent tasks can ACQUIRE it. +*/ smp_store_release (&sb->s_master_keys, keyring);return 0 ; }
@@ -234,8 +238,13 @@ struct key *fscrypt_find_master_key(struct super_block *sb,struct key *keyring;char description[FSCRYPT_MK_DESCRIPTION_SIZE ]; -/* pairs with smp_store_release() in allocate_filesystem_keyring() */ - keyring =READ_ONCE (sb->s_master_keys); +/* +* Pairs with the smp_store_release() in allocate_filesystem_keyring(). +* I.e., another task can publish ->s_master_keys concurrently, +* executing a RELEASE barrier. We need to use smp_load_acquire() here +* to safely ACQUIRE the memory the other task published. +*/ + keyring =smp_load_acquire (&sb->s_master_keys);if (keyring ==NULL )return ERR_PTR (-ENOKEY );/* No keyring yet, so no keys yet. */
Signature · 1 line
2.27.0
Eric Biggers
maintainerPATCH 4/5 fscrypt: use smp_load_acquire() for ->i_crypt_info
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.
fscrypt_info includes various sub-objects which are internal to and are allocated by other kernel subsystems such as keyrings and crypto. So by using READ_ONCE() for ->i_crypt_info, we're relying on internal implementation details of these other kernel subsystems.
Remove this fragile assumption by using smp_load_acquire() instead.
(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)
--- fs/crypto/keysetup.c | 12 +++++++++++- fs/crypto/policy.c | 4 ++-- include/linux/fscrypt.h | 29 ++++++++++++++++++++++++----- 3 files changed, 37 insertions(+), 8 deletions(-)
fs/crypto/keysetup.c +11 -1
@@ -518,7 +518,17 @@ int fscrypt_get_encryption_info(struct inode *inode)if (res) goto out; +/* +* Multiple tasks may race to set ->i_crypt_info, so use +* cmpxchg_release(). This pairs with the smp_load_acquire() in +* fscrypt_get_info(). I.e., here we publish ->i_crypt_info with a +* RELEASE barrier so that other tasks can ACQUIRE it. +*/ if (cmpxchg_release (&inode->i_crypt_info,NULL , crypt_info) ==NULL ) { +/* +* We won the race and set ->i_crypt_info to our crypt_info. +* Now link it into the master key's inode list. +*/ if (master_key) {struct fscrypt_master_key *mk = master_key->payload.data[0 ];
@@ -589,7 +599,7 @@ EXPORT_SYMBOL(fscrypt_free_inode);*/ int fscrypt_drop_inode (struct inode *inode) { -const struct fscrypt_info *ci =READ_ONCE (inode->i_crypt_info); +const struct fscrypt_info *ci =fscrypt_get_info (inode);const struct fscrypt_master_key *mk;/*
fs/crypto/policy.c +2 -2
@@ -338,7 +338,7 @@ static int fscrypt_get_policy(struct inode *inode, union fscrypt_policy *policy)union fscrypt_context ctx;int ret; - ci =READ_ONCE (inode->i_crypt_info); + ci =fscrypt_get_info (inode);if (ci) {/* key available, use the cached policy */ *policy = ci->ci_policy;
@@ -627,7 +627,7 @@ int fscrypt_inherit_context(struct inode *parent, struct inode *child,if (res <0 )return res; - ci =READ_ONCE (parent->i_crypt_info); + ci =fscrypt_get_info (parent);if (ci ==NULL )return -ENOKEY ;
include/linux/fscrypt.h +24 -5
@@ -74,10 +74,15 @@ struct fscrypt_operations { struct request_queue **devs); }; -static inline bool fscrypt_has_encryption_key (const struct inode *inode) +static inline struct fscrypt_info *fscrypt_get_info (const struct inode *inode) { -/* pairs with cmpxchg_release() in fscrypt_get_encryption_info() */ -return READ_ONCE (inode->i_crypt_info) !=NULL ; +/* +* Pairs with the cmpxchg_release() in fscrypt_get_encryption_info(). +* I.e., another task may publish ->i_crypt_info concurrently, executing +* a RELEASE barrier. We need to use smp_load_acquire() here to safely +* ACQUIRE the memory the other task published. +*/ +return smp_load_acquire (&inode->i_crypt_info); }/**
@@ -234,9 +239,9 @@ static inline void fscrypt_set_ops(struct super_block *sb, }#else /* !CONFIG_FS_ENCRYPTION */ -static inline bool fscrypt_has_encryption_key (const struct inode *inode) +static inline struct fscrypt_info *fscrypt_get_info (const struct inode *inode) { -return false; +return NULL ; }static inline bool fscrypt_needs_contents_encryption (const struct inode *inode)
@@ -619,6 +624,20 @@ static inline bool fscrypt_inode_uses_fs_layer_crypto(const struct inode *inode) !__fscrypt_inode_uses_inline_crypto (inode); } +/** +* fscrypt_has_encryption_key() - check whether an inode has had its key set up +* @inode: the inode to check +* +* Return: %true if the inode has had its encryption key set up, else %false. +* +* Usually this should be preceded by fscrypt_get_encryption_info() to try to +* set up the key first. +*/ +static inline bool fscrypt_has_encryption_key (const struct inode *inode) +{ +return fscrypt_get_info (inode) !=NULL ; +} +/** * fscrypt_require_key() - require an inode's encryption key * @inode: the inode we need the key for
Signature · 1 line
2.27.0
Eric Biggers
maintainerPATCH 5/5 fs-verity: use smp_load_acquire() for ->i_verity_info
In reply to Eric BiggersFrom: Eric Biggers <[email protected]>
Normally smp_store_release() or cmpxchg_release() is paired with smp_load_acquire(). Sometimes smp_load_acquire() can be replaced with the more lightweight READ_ONCE(). However, for this to be safe, all the published memory must only be accessed in a way that involves the pointer itself. This may not be the case if allocating the object also involves initializing a static or global variable, for example.
fsverity_info::tree_params.hash_alg->tfm is a crypto_ahash object that's internal to and is allocated by the crypto subsystem. So by using READ_ONCE() for ->i_verity_info, we're relying on internal implementation details of the crypto subsystem.
Remove this fragile assumption by using smp_load_acquire() instead.
Also fix the cmpxchg logic to correctly execute an ACQUIRE barrier when losing the cmpxchg race, since cmpxchg doesn't guarantee a memory barrier on failure.
(Note: I haven't seen any real-world problems here. This change is just fixing the code to be guaranteed correct and less fragile.)
--- fs/verity/open.c | 15 ++++++++++++--- include/linux/fsverity.h | 9 +++++++-- 2 files changed, 19 insertions(+), 5 deletions(-)
fs/verity/open.c +12 -3
@@ -221,11 +221,20 @@ struct fsverity_info *fsverity_create_info(const struct inode *inode,void fsverity_set_info (struct inode *inode,struct fsverity_info *vi) {/* -* Multiple processes may race to set ->i_verity_info, so use cmpxchg. -* This pairs with the READ_ONCE() in fsverity_get_info(). +* Multiple tasks may race to set ->i_verity_info, so use +* cmpxchg_release(). This pairs with the smp_load_acquire() in +* fsverity_get_info(). I.e., here we publish ->i_verity_info with a +* RELEASE barrier so that other tasks can ACQUIRE it. */ -if (cmpxchg (&inode->i_verity_info,NULL , vi) !=NULL ) +if (cmpxchg_release (&inode->i_verity_info,NULL , vi) !=NULL ) { +/* Lost the race, so free the fsverity_info we allocated. */ fsverity_free_info (vi); +/* +* Afterwards, the caller may access ->i_verity_info directly, +* so make sure to ACQUIRE the winning fsverity_info. +*/ + (void )fsverity_get_info (inode); + } }void fsverity_free_info (struct fsverity_info *vi)
include/linux/fsverity.h +7 -2
@@ -115,8 +115,13 @@ struct fsverity_operations {static inline struct fsverity_info *fsverity_get_info (const struct inode *inode) { -/* pairs with the cmpxchg() in fsverity_set_info() */ -return READ_ONCE (inode->i_verity_info); +/* +* Pairs with the cmpxchg_release() in fsverity_set_info(). +* I.e., another task may publish ->i_verity_info concurrently, +* executing a RELEASE barrier. We need to use smp_load_acquire() here +* to safely ACQUIRE the memory the other task published. +*/ +return smp_load_acquire (&inode->i_verity_info); }/* enable.c */
Signature · 1 line
2.27.0
Eric Biggers
maintainerQuoted 25 lines · Eric Biggers
This series fixes up some cases in
fs/crypto/andfs/verity/where "one-time init" is implemented usingREAD_ONCE()instead ofsmp_load_acquire()but it's not obviously correct.One case is fixed by using a better approach that removes the need to initialize anything. The others are fixed by upgrading
READ_ONCE()tosmp_load_acquire(). I've also improved the comments.This is motivated by the discussions at
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#u
and
https://lkml.kernel.org/linux-fsdevel/[email protected]/T/#uThese fixes are improvements over the status quo, so I'd prefer to apply them now, without waiting for any potential new generic one-time-init macros (which based on the latest discussion, won't be flexible enough to handle most of these cases anyway).
Eric Biggers (5):
fscrypt: switchfscrypt_do_sha256()to use the SHA-256 library
fscrypt: usesmp_load_acquire()forfscrypt_prepared_key
fscrypt: usesmp_load_acquire()for ->s_master_keys
fscrypt: usesmp_load_acquire()for ->i_crypt_info
fs-verity: usesmp_load_acquire()for ->i_verity_info
Quoted 11 lines
fs/crypto/Kconfig | 2 +- fs/crypto/fname.c | 41 +++++++++---------------------------- fs/crypto/fscrypt_private.h | 15 ++++++++------ fs/crypto/inline_crypt.c | 6 ++++-- fs/crypto/keyring.c | 15 +++++++++++--- fs/crypto/keysetup.c | 18 +++++++++++++--- fs/crypto/policy.c | 4 ++-- fs/verity/open.c | 15 +++++++++++--- include/linux/fscrypt.h | 29 +++++++++++++++++++++----- include/linux/fsverity.h | 9 ++++++-- 10 files changed, 96 insertions(+), 58 deletions(-)
Patches 1-4 applied to fscrypt.git#master for 5.9. Patch 5 applied to fscrypt.git#fsverity for 5.9.
- Eric