running kernel mode SIMD with softirqs disabled
Subsystems (2)
v3 cover running kernel mode SIMD with softirqs disabled
This is a follow-up to [0], but given that the arm64 architectural pieces have been merged for arm64, the only remaining changes are crypto specific. Therefore, the audience has been reduced to those people who are likely to care about these specifics.
Patch #1 addresses an issue in the skcipher walker which doesn't handle zero sized AEAD inputs entirely consistently, which is uncovered by the change in patch #7.
Patches #2 and #3 add some sanity checks to the public AEAD and skcipher APIs to limit their availibility to either task or softirq context (which is the only way in which they are currently being used). Adding this restriction permits the arm64 crypto code to get rid of all scalar fallbacks, given that on this architecture, softirqs are no longer served while the SIMD unit is being used in kernel mode, which means that the scalar fallbacks are never needed. These are removed in the remaining 4 patches.
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Ard Biesheuvel (7):
crypto: handle zero sized AEAD inputs correctly
crypto: aead - disallow en/decrypt for non-task or non-softirq context
crypto: skcipher - disallow en/decrypt for non-task or non-softirq
context
crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path
crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
crypto: arm64/aes-ce - stop using SIMD helper for skciphers
crypto: arm64/aes-ccm - remove non-SIMD fallback path
arch/arm64/crypto/Kconfig | 6 - arch/arm64/crypto/aes-ce-ccm-core.S | 1 + arch/arm64/crypto/aes-ce-ccm-glue.c | 183 +++++------------ arch/arm64/crypto/aes-glue.c | 102 ++-------- arch/arm64/crypto/aes-neonbs-glue.c | 122 +----------- arch/arm64/crypto/ghash-ce-glue.c | 209 +++++--------------- crypto/aead.c | 10 + crypto/skcipher.c | 12 ++ 8 files changed, 148 insertions(+), 497 deletions(-)
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 1/7 crypto: handle zero sized AEAD inputs correctly
There are corner cases where skcipher_walk_aead_[en|de]crypt() may be invoked with a zero sized input, which is not rejected by the walker code, but results in the skcipher_walk structure to not be fully initialized. This will leave stale values in its page and buffer members, which will be subsequently passed to kfree() or free_page() by skcipher_walk_done(), resulting in a crash if those routines fail to identify them as in valid inputs.
Fix this by setting page and buffer to NULL even if the size of the input is zero.
--- crypto/skcipher.c | 2 ++ 1 file changed, 2 insertions(+)
crypto/skcipher.c +2 -0
@@ -511,6 +511,8 @@ static int skcipher_walk_aead_common(struct skcipher_walk *walk, walk->nbytes =0 ; walk->iv = req->iv; walk->oiv = req->iv; + walk->buffer =NULL ; + walk->page =NULL ;if (unlikely (!walk->total))return 0 ;
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 2/7 crypto: aead - disallow en/decrypt for non-task or non-softirq context
In reply to Ard BiesheuvelIn order to ensure that kernel mode SIMD routines will not need a scalar fallback if they run with softirqs disabled, disallow any use of the AEAD encrypt and decrypt routines from outside of task or softirq context.
--- crypto/aead.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
crypto/aead.c +10 -0
@@ -87,6 +87,11 @@ int crypto_aead_encrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;
@@ -104,6 +109,11 @@ int crypto_aead_decrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 4/7 crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path
In reply to Ard BiesheuvelNow that kernel mode SIMD is guaranteed to be available when executing in task or softirq context, we no longer need scalar fallbacks to use when the NEON is unavailable. So get rid of them.
--- arch/arm64/crypto/ghash-ce-glue.c | 209 +++++--------------- 1 file changed, 51 insertions(+), 158 deletions(-)
arch/arm64/crypto/ghash-ce-glue.c +51 -158
@@ -362,84 +362,36 @@ static int gcm_encrypt(struct aead_request *req) err =skcipher_walk_aead_encrypt (&walk, req, false); -if (likely (crypto_simd_usable ())) { -do { -const u8 *src = walk.src.virt.addr; -u8 *dst = walk.dst.virt.addr; -int nbytes = walk.nbytes; - - tag = (u8 *)&lengths; - -if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) { - src = dst =memcpy (buf +sizeof (buf) - nbytes, - src, nbytes); - }else if (nbytes < walk.total) { - nbytes &= ~(AES_BLOCK_SIZE -1 ); - tag =NULL ; - } - -kernel_neon_begin (); -pmull_gcm_encrypt (nbytes, dst, src, ctx->ghash_key.h, - dg, iv, ctx->aes_key.key_enc, nrounds, - tag); -kernel_neon_end (); - -if (unlikely (!nbytes)) -break ; - -if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) -memcpy (walk.dst.virt.addr, - buf +sizeof (buf) - nbytes, nbytes); - - err =skcipher_walk_done (&walk, walk.nbytes - nbytes); - }while (walk.nbytes); - }else { -while (walk.nbytes >=AES_BLOCK_SIZE ) { -int blocks = walk.nbytes /AES_BLOCK_SIZE ; -const u8 *src = walk.src.virt.addr; -u8 *dst = walk.dst.virt.addr; -int remaining = blocks; - -do { -aes_encrypt (&ctx->aes_key, buf, iv); -crypto_xor_cpy (dst, src, buf,AES_BLOCK_SIZE ); -crypto_inc (iv,AES_BLOCK_SIZE ); - - dst +=AES_BLOCK_SIZE ; - src +=AES_BLOCK_SIZE ; - }while (--remaining >0 ); - -ghash_do_update (blocks, dg, walk.dst.virt.addr, - &ctx->ghash_key,NULL ); - - err =skcipher_walk_done (&walk, - walk.nbytes %AES_BLOCK_SIZE ); - } - -/* handle the tail */ -if (walk.nbytes) { -aes_encrypt (&ctx->aes_key, buf, iv); +do { +const u8 *src = walk.src.virt.addr; +u8 *dst = walk.dst.virt.addr; +int nbytes = walk.nbytes; -crypto_xor_cpy (walk.dst.virt.addr, walk.src.virt.addr, - buf, walk.nbytes); + tag = (u8 *)&lengths; -memcpy (buf, walk.dst.virt.addr, walk.nbytes); -memset (buf + walk.nbytes,0 ,sizeof (buf) - walk.nbytes); +if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) { + src = dst =memcpy (buf +sizeof (buf) - nbytes, + src, nbytes); + }else if (nbytes < walk.total) { + nbytes &= ~(AES_BLOCK_SIZE -1 ); + tag =NULL ; } - tag = (u8 *)&lengths; -ghash_do_update (1 , dg, tag, &ctx->ghash_key, - walk.nbytes ? buf :NULL ); +kernel_neon_begin (); +pmull_gcm_encrypt (nbytes, dst, src, ctx->ghash_key.h, + dg, iv, ctx->aes_key.key_enc, nrounds, + tag); +kernel_neon_end (); -if (walk.nbytes) - err =skcipher_walk_done (&walk,0 ); +if (unlikely (!nbytes)) +break ; -put_unaligned_be64 (dg[1 ], tag); -put_unaligned_be64 (dg[0 ], tag +8 ); -put_unaligned_be32 (1 , iv +GCM_IV_SIZE ); -aes_encrypt (&ctx->aes_key, iv, iv); -crypto_xor (tag, iv,AES_BLOCK_SIZE ); - } +if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) +memcpy (walk.dst.virt.addr, + buf +sizeof (buf) - nbytes, nbytes); + + err =skcipher_walk_done (&walk, walk.nbytes - nbytes); + }while (walk.nbytes);if (err)return err;
@@ -464,6 +416,7 @@ static int gcm_decrypt(struct aead_request *req)u64 dg[2 ] = {};be128 lengths;u8 *tag; +int ret;int err; lengths.a =cpu_to_be64 (req->assoclen *8 );
@@ -481,101 +434,41 @@ static int gcm_decrypt(struct aead_request *req) err =skcipher_walk_aead_decrypt (&walk, req, false); -if (likely (crypto_simd_usable ())) { -int ret; - -do { -const u8 *src = walk.src.virt.addr; -u8 *dst = walk.dst.virt.addr; -int nbytes = walk.nbytes; - - tag = (u8 *)&lengths; - -if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) { - src = dst =memcpy (buf +sizeof (buf) - nbytes, - src, nbytes); - }else if (nbytes < walk.total) { - nbytes &= ~(AES_BLOCK_SIZE -1 ); - tag =NULL ; - } - -kernel_neon_begin (); - ret =pmull_gcm_decrypt (nbytes, dst, src, - ctx->ghash_key.h, - dg, iv, ctx->aes_key.key_enc, - nrounds, tag, otag, authsize); -kernel_neon_end (); - -if (unlikely (!nbytes)) -break ; - -if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) -memcpy (walk.dst.virt.addr, - buf +sizeof (buf) - nbytes, nbytes); - - err =skcipher_walk_done (&walk, walk.nbytes - nbytes); - }while (walk.nbytes); - -if (err) -return err; -if (ret) -return -EBADMSG ; - }else { -while (walk.nbytes >=AES_BLOCK_SIZE ) { -int blocks = walk.nbytes /AES_BLOCK_SIZE ; -const u8 *src = walk.src.virt.addr; -u8 *dst = walk.dst.virt.addr; - -ghash_do_update (blocks, dg, walk.src.virt.addr, - &ctx->ghash_key,NULL ); - -do { -aes_encrypt (&ctx->aes_key, buf, iv); -crypto_xor_cpy (dst, src, buf,AES_BLOCK_SIZE ); -crypto_inc (iv,AES_BLOCK_SIZE ); - - dst +=AES_BLOCK_SIZE ; - src +=AES_BLOCK_SIZE ; - }while (--blocks >0 ); +do { +const u8 *src = walk.src.virt.addr; +u8 *dst = walk.dst.virt.addr; +int nbytes = walk.nbytes; - err =skcipher_walk_done (&walk, - walk.nbytes %AES_BLOCK_SIZE ); - } + tag = (u8 *)&lengths; -/* handle the tail */ -if (walk.nbytes) { -memcpy (buf, walk.src.virt.addr, walk.nbytes); -memset (buf + walk.nbytes,0 ,sizeof (buf) - walk.nbytes); +if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) { + src = dst =memcpy (buf +sizeof (buf) - nbytes, + src, nbytes); + }else if (nbytes < walk.total) { + nbytes &= ~(AES_BLOCK_SIZE -1 ); + tag =NULL ; } - tag = (u8 *)&lengths; -ghash_do_update (1 , dg, tag, &ctx->ghash_key, - walk.nbytes ? buf :NULL ); - -if (walk.nbytes) { -aes_encrypt (&ctx->aes_key, buf, iv); +kernel_neon_begin (); + ret =pmull_gcm_decrypt (nbytes, dst, src, ctx->ghash_key.h, + dg, iv, ctx->aes_key.key_enc, + nrounds, tag, otag, authsize); +kernel_neon_end (); -crypto_xor_cpy (walk.dst.virt.addr, walk.src.virt.addr, - buf, walk.nbytes); +if (unlikely (!nbytes)) +break ; - err =skcipher_walk_done (&walk,0 ); - } +if (unlikely (nbytes >0 && nbytes <AES_BLOCK_SIZE )) +memcpy (walk.dst.virt.addr, + buf +sizeof (buf) - nbytes, nbytes); -if (err) -return err; + err =skcipher_walk_done (&walk, walk.nbytes - nbytes); + }while (walk.nbytes); -put_unaligned_be64 (dg[1 ], tag); -put_unaligned_be64 (dg[0 ], tag +8 ); -put_unaligned_be32 (1 , iv +GCM_IV_SIZE ); -aes_encrypt (&ctx->aes_key, iv, iv); -crypto_xor (tag, iv,AES_BLOCK_SIZE ); +if (err) +return err; -if (crypto_memneq (tag, otag, authsize)) { -memzero_explicit (tag,AES_BLOCK_SIZE ); -return -EBADMSG ; - } - } -return 0 ; +return ret ? -EBADMSG :0 ; }static struct aead_alg gcm_aes_alg = {
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 3/7 crypto: skcipher - disallow en/decrypt for non-task or non-softirq context
In reply to Ard BiesheuvelIn order to ensure that kernel mode SIMD routines will not need a scalar fallback if they run with softirqs disabled, disallow any use of the skcipher encrypt and decrypt routines from outside of task or softirq context.
--- crypto/skcipher.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
crypto/skcipher.c +10 -0
@@ -625,6 +625,11 @@ int crypto_skcipher_encrypt(struct skcipher_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_skcipher_get_flags (tfm) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;
@@ -642,6 +647,11 @@ int crypto_skcipher_decrypt(struct skcipher_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_skcipher_get_flags (tfm) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 6/7 crypto: arm64/aes-ce - stop using SIMD helper for skciphers
In reply to Ard BiesheuvelCalls into the skcipher API can only occur from contexts where the SIMD unit is available, so there is no need for the SIMD helper.
--- arch/arm64/crypto/Kconfig | 4 - arch/arm64/crypto/aes-glue.c | 102 +++----------------- 2 files changed, 13 insertions(+), 93 deletions(-)
arch/arm64/crypto/Kconfig +0 -4
@@ -88,16 +88,12 @@ config CRYPTO_AES_ARM64_CE_BLK depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER select CRYPTO_AES_ARM64_CE - select CRYPTO_AES_ARM64 - select CRYPTO_SIMD config CRYPTO_AES_ARM64_NEON_BLK tristate "AES in ECB/CBC/CTR/XTS modes using NEON instructions" depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER - select CRYPTO_AES_ARM64 select CRYPTO_LIB_AES - select CRYPTO_SIMD config CRYPTO_CHACHA20_NEON tristate "ChaCha20, XChaCha20, and XChaCha12 stream ciphers using NEON instructions"
arch/arm64/crypto/aes-glue.c +13 -89
179 lines not on this page · raw message
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 5/7 crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
In reply to Ard BiesheuvelCalls into the skcipher API can only occur from contexts where the SIMD unit is available, so there is no need for the SIMD helper.
--- arch/arm64/crypto/Kconfig | 2 - arch/arm64/crypto/aes-neonbs-glue.c | 122 ++------------------ 2 files changed, 9 insertions(+), 115 deletions(-)
arch/arm64/crypto/Kconfig +0 -2
@@ -122,8 +122,6 @@ config CRYPTO_AES_ARM64_BS depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER select CRYPTO_AES_ARM64_NEON_BLK - select CRYPTO_AES_ARM64 select CRYPTO_LIB_AES - select CRYPTO_SIMD endif
arch/arm64/crypto/aes-neonbs-glue.c +9 -113
196 lines not on this page · raw message
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v3 7/7 crypto: arm64/aes-ccm - remove non-SIMD fallback path
In reply to Ard BiesheuvelAES/CCM on arm64 is implemented as a synchronous AEAD, and so it is guaranteed by the API that it is only invoked in task or softirq context. Since softirqs are now only handled when the SIMD is not being used in the task context that was interrupted to service the softirq, we no longer need a fallback path. Let's remove it.
--- arch/arm64/crypto/aes-ce-ccm-core.S | 1 + arch/arm64/crypto/aes-ce-ccm-glue.c | 183 ++++++-------------- 2 files changed, 53 insertions(+), 131 deletions(-)
arch/arm64/crypto/aes-ce-ccm-core.S +1 -0
@@ -124,6 +124,7 @@ SYM_FUNC_START(ce_aes_ccm_final) SYM_FUNC_END(ce_aes_ccm_final) .macro aes_ccm_do_crypt,enc + cbz x2, 5f ldr x8, [x6, #8] /* load lower ctr */ ld1 {v0.16b}, [x5] /* load mac */ CPU_LE( rev x8, x8 ) /* keep swabbed ctr in reg */
arch/arm64/crypto/aes-ce-ccm-glue.c +52 -131
225 lines not on this page · raw message
Signature · 7 lines
2.20.1 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 1/7] crypto: handle zero sized AEAD inputs correctly
In reply to Ard BiesheuvelQuoted 8 lines · Ard Biesheuvel
There are corner cases where skcipher_walk_aead_[en|de]crypt() may be invoked with a zero sized input, which is not rejected by the walker code, but results in the
skcipher_walkstructure to not be fully initialized. This will leave stale values in its page and buffer members, which will be subsequently passed tokfree()orfree_page()byskcipher_walk_done(), resulting in a crash if those routines fail to identify them as in valid inputs.
Quoted 4 lines
Fix this by setting page and buffer to NULL even if the size of the input is zero.
Signed-off-by Ard Biesheuvel <[email protected]>
Is this fixing an existing bug, or only a bug that got exposed by this patchset? It would be helpful to make that clear (and if it fixes an existing bug, include a Fixes tag).
Also, skcipher_walk_virt() doesn't set page and buffer to NULL, as it is currently expected that skcipher_walk_done() is only called when walk.nbytes != 0. Is something different for skcipher_walk_aead_[en|de]crypt()?
- Eric
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 2/7] crypto: aead - disallow en/decrypt for non-task or non-softirq context
In reply to Ard BiesheuvelQuoted 27 lines · Ard Biesheuvel
In order to ensure that kernel mode SIMD routines will not need a scalar fallback if they run with softirqs disabled, disallow any use of the AEAD encrypt and decrypt routines from outside of task or softirq context.
Signed-off-by Ard Biesheuvel <[email protected]>--- crypto/aead.c | 10 ++++++++++ 1 file changed, 10 insertions(+)crypto/aead.c +5
-0@@ -87,6 +87,11 @@ int crypto_aead_encrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;@@ -104,6 +109,11 @@ int crypto_aead_decrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;
crypto/aead.c int crypto_aead_decrypt(struct aead_request *req) · line 119
Quoted 10 lines
crypto/aead.c +5
-0int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;
This probably should go after crypto_stats_get() so that the error gets counted in the stats (if stats are enabled) -- analogous to how the ENOKEY error is counted.
Likewise for the skcipher patch.
- Eric
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 5/7] crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
In reply to Ard BiesheuvelQuoted 4 lines · Ard Biesheuvel
Calls into the skcipher API can only occur from contexts where the SIMD unit is available, so there is no need for the SIMD helper.
Signed-off-by Ard Biesheuvel <[email protected]>
It would be helpful if the commit message made it clear that "Calls into the skcipher API can only occur from contexts where the SIMD unit is available" is something that is now the case but wasn't the case previously. Otherwise I could see people backporting this patch without its prerequisites.
Likewise for some of the other patches in this patchset.
- Eric
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 18 lines · Ard Biesheuvel
This is a follow-up to [0], but given that the arm64 architectural pieces have been merged for arm64, the only remaining changes are crypto specific. Therefore, the audience has been reduced to those people who are likely to care about these specifics.
Patch #1 addresses an issue in the skcipher walker which doesn't handle zero sized AEAD inputs entirely consistently, which is uncovered by the change in patch #7.
Patches #2 and #3 add some sanity checks to the public AEAD and skcipher APIs to limit their availibility to either task or softirq context (which is the only way in which they are currently being used). Adding this restriction permits the arm64 crypto code to get rid of all scalar fallbacks, given that on this architecture, softirqs are no longer served while the SIMD unit is being used in kernel mode, which means that the scalar fallbacks are never needed. These are removed in the remaining 4 patches.
Quoted 1 line
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Did you check whether any updates to the self-tests in testmgr.c are warranted? Specifically, is disabling the use of SIMD for testing still something that makes sense?
- Eric
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 1/7] crypto: handle zero sized AEAD inputs correctly
In reply to Eric BiggersQuoted 10 lines · Eric Biggers
Quoted 8 lines · Ard Biesheuvel
There are corner cases where skcipher_walk_aead_[en|de]crypt() may be invoked with a zero sized input, which is not rejected by the walker code, but results in the
skcipher_walkstructure to not be fully initialized. This will leave stale values in its page and buffer members, which will be subsequently passed tokfree()orfree_page()byskcipher_walk_done(), resulting in a crash if those routines fail to identify them as in valid inputs.
Quoted 9 lines
Quoted 4 lines
Fix this by setting page and buffer to NULL even if the size of the input is zero.
Signed-off-by Ard Biesheuvel <[email protected]>Is this fixing an existing bug, or only a bug that got exposed by this patchset? It would be helpful to make that clear (and if it fixes an existing bug, include a Fixes tag).
The CCM change in the last patch uncovers this issue, and I don't think it is likely we would ever hit it anywhere else.
Quoted 4 lines
Also,
skcipher_walk_virt()doesn't set page and buffer to NULL, as it is currently expected thatskcipher_walk_done()is only called when walk.nbytes != 0. Is something different for skcipher_walk_aead_[en|de]crypt()?
The difference is that zero sized inputs never make sense for skciphers, but for AEADs, they could occur, even if they are uncommon (the AEAD could have associated data only, and no plain/ciphertext)
But in the general case, I would assume that skcipher_walk_done() can be called on a walk that was successfully started with skcipher_walk_virt() without crashing, even if the scatterlist has size zero, so perhaps we should fix that one as well.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 2/7] crypto: aead - disallow en/decrypt for non-task or non-softirq context
In reply to Eric BiggersQuoted 44 lines · Eric Biggers
Quoted 27 lines · Ard Biesheuvel
In order to ensure that kernel mode SIMD routines will not need a scalar fallback if they run with softirqs disabled, disallow any use of the AEAD encrypt and decrypt routines from outside of task or softirq context.
Signed-off-by Ard Biesheuvel <[email protected]>--- crypto/aead.c | 10 ++++++++++ 1 file changed, 10 insertions(+)crypto/aead.c +5
-0@@ -87,6 +87,11 @@ int crypto_aead_encrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;@@ -104,6 +109,11 @@ int crypto_aead_decrypt(struct aead_request *req)unsigned int cryptlen = req->cryptlen;crypto/aead.c int crypto_aead_decrypt(struct aead_request *req) · line 119
Quoted 10 lines
crypto/aead.c +5
-0int ret; +if (!(alg->cra_flags &CRYPTO_ALG_ASYNC ) && +WARN_ONCE (!in_task () && !in_serving_softirq (), +"synchronous call from invalid context\n" )) +return -EBUSY ; +crypto_stats_get (alg);if (crypto_aead_get_flags (aead) &CRYPTO_TFM_NEED_KEY ) ret = -ENOKEY ;This probably should go after
crypto_stats_get()so that the error gets counted in the stats (if stats are enabled) -- analogous to how the ENOKEY error is counted.
Quoted 2 lines
Likewise for the skcipher patch.
Good point, I'll fix that
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH v3 5/7] crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
In reply to Eric BiggersQuoted 5 lines · Eric Biggers
Quoted 3 lines · Ard Biesheuvel
Calls into the skcipher API can only occur from contexts where the SIMD unit is available, so there is no need for the SIMD helper.
Quoted 9 lines
Quoted 1 line
Signed-off-by Ard Biesheuvel <[email protected]>It would be helpful if the commit message made it clear that "Calls into the skcipher API can only occur from contexts where the SIMD unit is available" is something that is now the case but wasn't the case previously. Otherwise I could see people backporting this patch without its prerequisites.
Likewise for some of the other patches in this patchset.
OK
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 20 lines · Eric Biggers
Quoted 18 lines · Ard Biesheuvel
This is a follow-up to [0], but given that the arm64 architectural pieces have been merged for arm64, the only remaining changes are crypto specific. Therefore, the audience has been reduced to those people who are likely to care about these specifics.
Patch #1 addresses an issue in the skcipher walker which doesn't handle zero sized AEAD inputs entirely consistently, which is uncovered by the change in patch #7.
Patches #2 and #3 add some sanity checks to the public AEAD and skcipher APIs to limit their availibility to either task or softirq context (which is the only way in which they are currently being used). Adding this restriction permits the arm64 crypto code to get rid of all scalar fallbacks, given that on this architecture, softirqs are no longer served while the SIMD unit is being used in kernel mode, which means that the scalar fallbacks are never needed. These are removed in the remaining 4 patches.
Quoted 6 lines
Quoted 1 line
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Did you check whether any updates to the self-tests in
testmgr.care warranted? Specifically, is disabling the use of SIMD for testing still something that makes sense?
The situation is not ideal, but I am not sure what we can do about
this: the scalar fallbacks are gone, which means that the SIMD unit will be used in the test even if testmgr attempts to disable it. But keeping the scalar fallbacks just for the test suite makes no sense either. So I don't think we should change anything, other than perhaps document this somewhere (any suggestions on a place to put that)
Note that the library routines, as well as shashes (which are sometimes exposed via library routines, e.g., CRC-T10DIF and CRC-32, and maybe others) are different, which is why their scalar fallbacks are retained. There, we need the testmgr to override SIMD availability to ensure that combinations of the SIMD and scalar code are tested.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Herbert Xu
maintainerRe: [PATCH v3 1/7] crypto: handle zero sized AEAD inputs correctly
In reply to Ard BiesheuvelQuoted 4 lines · Ard Biesheuvel
The difference is that zero sized inputs never make sense for skciphers, but for AEADs, they could occur, even if they are uncommon (the AEAD could have associated data only, and no plain/ciphertext)
I don't see what a zero-sized input has to do with this though. When the walk->nbytes is zero, that means that you must never call the done function, because the walk state could be in error in which case everything would have been freed already and calling the done function may potentially cause a double-free.
I don't understand why in the case of AEAD you cannot structure your code such that the done function is not called when nbytes is zero.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Re: [PATCH v3 1/7] crypto: handle zero sized AEAD inputs correctly
Quoted 13 lines · Herbert Xu
Quoted 4 lines · Ard Biesheuvel
The difference is that zero sized inputs never make sense for skciphers, but for AEADs, they could occur, even if they are uncommon (the AEAD could have associated data only, and no plain/ciphertext)
I don't see what a zero-sized input has to do with this though. When the
walk->nbytesis zero, that means that you must never call the done function, because the walk state could be in error in which case everything would have been freed already and calling the done function may potentially cause a double-free.
Quoted 4 lines
I don't understand why in the case of AEAD you cannot structure your code such that the done function is not called when nbytes is zero.
OK.
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel