crypto: avoid DMA padding for request structures
Subsystems (22)
cover crypto: avoid DMA padding for request structures
This series is related to Catalin's work [0] on reducing the memory overhead of supporting non-coherent DMA, which requires memory buffers to be rounded up to avoid corruption by the cache invalidation that is needed for inbound DMA.
In the crypto subsystem, every skcipher, aead or ahash request requires a request struct to be allocated, and these are usually backed by kmalloc(). Such request buffers are sized dynamically, based on the requirements of the implementation of the algorithm, and the surplus is made available to the driver via an opaque context pointer.
Since some drivers may perform inbound non-coherent DMA into that buffer, it must not share any cachelines with adjacent allocations, and this is why the context pointer is rounded up to ARCH_KMALLOC_MINALIGN, which takes the minimum DMA alignment into account on architectures where this is needed.
This means that, even when using crypto drivers that don't do DMA to begin with (which includes synchronous skciphers, aeads and ahashes based on CPU instructions), or that do only coherent DMA are forced to perform this padding and alignment, which may affect the memory footprint substantially: on arm64, the compile time minimum DMA alignment is 128 bytes.
So instead, require drivers to set a new flag CRYPTO_ALG_NEED_DMA_ALIGNMENT if it performs DMA into the context buffers, and only take DMA alignment into account if the flag is set.
Initially, we set this flag for all asynchronous accelerator drivers in drivers/crypto, which simply preserves the status quo for these systems once the subsequent patches get rid of this overhead. Future patches could be applied to drivers that don't actually need it, or only need it when running in non-coherent mode.
Note that the new approach proposed here still uses the compile time value for DMA alignment, but this can be updated to use the runtime values (which is usually lower and therefore less wasteful) after Catalin's changes land.
[0] https://lore.kernel.org/linux-arm-kernel/[email protected]/
Ard Biesheuvel (8):
crypto: add flag for algos that need DMA aligned context buffers
crypto: safexcel - take request size after setting TFM
crypto: drivers - set CRYPTO_ALG_NEED_DMA_ALIGNMENT where needed
crypto: drivers - avoid setting skcipher TFM reqsize directly
crypto: skcipher - avoid rounding up request size to DMA alignment
crypto: aead - avoid DMA alignment for request structures unless
needed
crypto: ahash - avoid DMA alignment for request structures unless
needed
crypto: safexcel - reduce alignment of stack buffer
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c | 6 +- drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c | 20 +++---- drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c | 6 +- drivers/crypto/amcc/crypto4xx_core.c | 8 +++ drivers/crypto/amlogic/amlogic-gxl-cipher.c | 5 +- drivers/crypto/amlogic/amlogic-gxl-core.c | 2 + drivers/crypto/atmel-aes.c | 2 +- drivers/crypto/atmel-sha.c | 2 +- drivers/crypto/atmel-tdes.c | 2 +- drivers/crypto/axis/artpec6_crypto.c | 8 +++ drivers/crypto/bcm/cipher.c | 23 +++++++- drivers/crypto/caam/caamalg.c | 4 +- drivers/crypto/caam/caamalg_qi.c | 2 + drivers/crypto/caam/caamalg_qi2.c | 4 +- drivers/crypto/caam/caamhash.c | 3 +- drivers/crypto/cavium/cpt/cptvf_algs.c | 6 ++ drivers/crypto/cavium/nitrox/nitrox_aead.c | 6 +- drivers/crypto/cavium/nitrox/nitrox_skcipher.c | 24 +++++--- drivers/crypto/ccree/cc_aead.c | 3 + drivers/crypto/ccree/cc_cipher.c | 3 + drivers/crypto/ccree/cc_hash.c | 6 ++ drivers/crypto/chelsio/chcr_algo.c | 5 +- drivers/crypto/gemini/sl3516-ce-cipher.c | 5 +- drivers/crypto/gemini/sl3516-ce-core.c | 1 + drivers/crypto/hifn_795x.c | 4 +- drivers/crypto/hisilicon/sec/sec_algs.c | 8 +++ drivers/crypto/hisilicon/sec2/sec_crypto.c | 2 + drivers/crypto/inside-secure/safexcel.h | 17 +++--- drivers/crypto/inside-secure/safexcel_cipher.c | 55 ++++++++++++++++-- drivers/crypto/inside-secure/safexcel_hash.c | 26 +++++++++ drivers/crypto/ixp4xx_crypto.c | 2 + drivers/crypto/keembay/keembay-ocs-aes-core.c | 12 ++++ drivers/crypto/keembay/keembay-ocs-hcu-core.c | 30 ++++++---- drivers/crypto/marvell/cesa/cipher.c | 6 ++ drivers/crypto/marvell/cesa/hash.c | 6 ++ drivers/crypto/marvell/octeontx/otx_cptvf_algs.c | 60 +++++++++++++++----- drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c | 52 ++++++++++++----- drivers/crypto/mxs-dcp.c | 8 ++- drivers/crypto/n2_core.c | 1 + drivers/crypto/omap-aes.c | 5 ++ drivers/crypto/omap-des.c | 4 ++ drivers/crypto/omap-sham.c | 12 ++++ drivers/crypto/qce/aead.c | 1 + drivers/crypto/qce/sha.c | 3 +- drivers/crypto/qce/skcipher.c | 1 + drivers/crypto/rockchip/rk3288_crypto_ahash.c | 3 + drivers/crypto/rockchip/rk3288_crypto_skcipher.c | 18 ++++-- drivers/crypto/s5p-sss.c | 6 ++ drivers/crypto/sa2ul.c | 9 +++ drivers/crypto/sahara.c | 14 +++-- drivers/crypto/stm32/stm32-cryp.c | 27 ++++++--- drivers/crypto/stm32/stm32-hash.c | 8 +++ drivers/crypto/talitos.c | 40 +++++++++++++ drivers/crypto/ux500/cryp/cryp_core.c | 21 ++++--- drivers/crypto/ux500/hash/hash_core.c | 12 ++-- drivers/crypto/xilinx/zynqmp-aes-gcm.c | 1 + include/crypto/aead.h | 2 +- include/crypto/hash.h | 5 +- include/crypto/internal/aead.h | 13 ++++- include/crypto/internal/hash.h | 10 +++- include/crypto/internal/skcipher.h | 13 ++++- include/crypto/skcipher.h | 8 +-- include/linux/crypto.h | 21 +++++++ 63 files changed, 568 insertions(+), 134 deletions(-)
Signature · 1 line
2.30.2
PATCH 2/8 crypto: safexcel - take request size after setting TFM
The skcipher, aead and ahash request structure types will no longer be aligned for DMA, and the padding and re-alignment of the context buffer region will be taken care of at runtime.
This means that we need to update the stack representation accordingly, to ensure that the context pointer doesn't point past the allocation after rounding.
Also, as getting at the context pointer of a skcipher_request will involve a check of the underlying algo's cra_flags field, as it may need to be aligned for DMA, defer grabbing the context pointer until after setting the TFM.
--- drivers/crypto/inside-secure/safexcel.h | 15 +++++++++------ drivers/crypto/inside-secure/safexcel_cipher.c | 8 ++++---- 2 files changed, 13 insertions(+), 10 deletions(-)
drivers/crypto/inside-secure/safexcel.h +9 -6
@@ -56,12 +56,15 @@ GFP_KERNEL : GFP_ATOMIC)/* Custom on-stack requests (for invalidation) */ -#define EIP197_SKCIPHER_REQ_SIZE sizeof(struct skcipher_request) + \ - sizeof(struct safexcel_cipher_req) -#define EIP197_AHASH_REQ_SIZE sizeof(struct ahash_request) + \ - sizeof(struct safexcel_ahash_req) -#define EIP197_AEAD_REQ_SIZE sizeof(struct aead_request) + \ - sizeof(struct safexcel_cipher_req) +#define EIP197_SKCIPHER_REQ_SIZE (ALIGN(sizeof(struct skcipher_request), \ + CRYPTO_MINALIGN) + \ + sizeof(struct safexcel_cipher_req)) +#define EIP197_AHASH_REQ_SIZE (ALIGN(sizeof(struct ahash_request), \ + CRYPTO_MINALIGN) + \ + sizeof(struct safexcel_ahash_req)) +#define EIP197_AEAD_REQ_SIZE (ALIGN(sizeof(struct aead_request), \ + CRYPTO_MINALIGN) + \ + sizeof(struct safexcel_cipher_req))#define EIP197_REQUEST_ON_STACK (name, type, size) \ char __##name##_desc[size] CRYPTO_MINALIGN_ATTR; \ struct type##_request *name = (void *)__##name##_desc
drivers/crypto/inside-secure/safexcel_cipher.c +4 -4
@@ -1108,7 +1108,6 @@ static int safexcel_cipher_exit_inv(struct crypto_tfm *tfm,static int safexcel_skcipher_exit_inv (struct crypto_tfm *tfm) {EIP197_REQUEST_ON_STACK (req, skcipher,EIP197_SKCIPHER_REQ_SIZE ); -struct safexcel_cipher_req *sreq =skcipher_request_ctx (req);struct safexcel_inv_result result = {};memset (req,0 ,sizeof (struct skcipher_request ));
@@ -1117,13 +1116,13 @@ static int safexcel_skcipher_exit_inv(struct crypto_tfm *tfm) safexcel_inv_complete, &result);skcipher_request_set_tfm (req,__crypto_skcipher_cast (tfm)); -return safexcel_cipher_exit_inv (tfm, &req->base, sreq, &result); +return safexcel_cipher_exit_inv (tfm, &req->base, +skcipher_request_ctx (req), &result); }static int safexcel_aead_exit_inv (struct crypto_tfm *tfm) {EIP197_REQUEST_ON_STACK (req, aead,EIP197_AEAD_REQ_SIZE ); -struct safexcel_cipher_req *sreq =aead_request_ctx (req);struct safexcel_inv_result result = {};memset (req,0 ,sizeof (struct aead_request ));
@@ -1132,7 +1131,8 @@ static int safexcel_aead_exit_inv(struct crypto_tfm *tfm) safexcel_inv_complete, &result);aead_request_set_tfm (req,__crypto_aead_cast (tfm)); -return safexcel_cipher_exit_inv (tfm, &req->base, sreq, &result); +return safexcel_cipher_exit_inv (tfm, &req->base,aead_request_ctx (req), + &result); } static int safexcel_queue_req(struct crypto_async_request *base,
Signature · 1 line
2.30.2
PATCH 5/8 crypto: skcipher - avoid rounding up request size to DMA alignment
In reply to Ard BiesheuvelNow that algo implementations that require it will set the new CRYPTO_ALG_NEED_DMA_ALIGNMENT flag, we can reduce the static footprint of the skcipher request structure, and increase the request context size to include the headroom needed for DMA alignment, but only when it is actually needed.
On arm64, this (and reordering the 'base' field) reduces the size of the skcipher request structure from 128 bytes to 72 bytes. It also reduces the minimum alignment of the struct type, which is relevant because the SYNC_SKCIPHER_ON_STACK() macro places such a struct on the stack, and here, the alignment requirement increases the memory footprint substantially, given that the stack pointer can only be assumed to be aligned to 16 bytes, as per the AArch64 ABI.
Since DMA to the stack is never allowed, we can ignore the flag here, and simply align the whole allocation to CRYPTO_REQ_MINALIGN.
--- include/crypto/internal/skcipher.h | 13 +++++++++++-- include/crypto/skcipher.h | 8 ++++---- include/linux/crypto.h | 14 ++++++++++++++ 3 files changed, 29 insertions(+), 6 deletions(-)
include/crypto/internal/skcipher.h +11 -2
@@ -119,7 +119,15 @@ static inline struct crypto_skcipher *crypto_spawn_skcipher(static inline void crypto_skcipher_set_reqsize (struct crypto_skcipher *skcipher,unsigned int reqsize) { - skcipher->reqsize = reqsize; +unsigned int align =crypto_tfm_alg_req_alignmask (&skcipher->base) +1 ; + +/* +* The request structure itself is only aligned to CRYPTO_REQ_MINALIGN, +* so we need to add some headroom, allowing us to return a suitably +* aligned context buffer pointer. We also need to round up the size so +* we don't end up sharing a cacheline at the end of the buffer. +*/ + skcipher->reqsize =ALIGN (reqsize, align) + align -CRYPTO_REQ_MINALIGN ; }int crypto_register_skcipher (struct skcipher_alg *alg);
@@ -153,7 +161,8 @@ static inline void *crypto_skcipher_ctx(struct crypto_skcipher *tfm)static inline void *skcipher_request_ctx (struct skcipher_request *req) { -return req->__ctx; +return PTR_ALIGN (&req->__ctx, +crypto_tfm_alg_req_alignmask (req->base.tfm) +1 ); }static inline u32 skcipher_request_flags (struct skcipher_request *req)
include/crypto/skcipher.h +4 -4
@@ -26,6 +26,8 @@ struct scatterlist;* @__ctx: Start of private context data */ struct skcipher_request { +struct crypto_async_request base; +unsigned int cryptlen;u8 *iv;
@@ -33,9 +35,7 @@ struct skcipher_request {struct scatterlist *src;struct scatterlist *dst; -struct crypto_async_request base; - - void *__ctx[] CRYPTO_MINALIGN_ATTR; + void *__ctx[] CRYPTO_REQ_MINALIGN_ATTR; };struct crypto_skcipher {
@@ -132,7 +132,7 @@ struct skcipher_alg {MAX_SYNC_SKCIPHER_REQSIZE + \ (!(sizeof ((struct crypto_sync_skcipher *)1 == \ (typeof (tfm ))1 ))) \ - ] CRYPTO_MINALIGN_ATTR; \ + ] CRYPTO_REQ_MINALIGN_ATTR;\ struct skcipher_request *name = (void *)__##name##_desc/**
include/linux/crypto.h +14 -0
@@ -178,6 +178,10 @@#define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN))) +/* minimum alignment for request structures */ +#define CRYPTO_REQ_MINALIGN ARCH_SLAB_MINALIGN +#define CRYPTO_REQ_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_REQ_MINALIGN))) +struct scatterlist ;struct crypto_async_request ;struct crypto_tfm ;
@@ -706,6 +710,16 @@ static inline unsigned int crypto_tfm_alg_alignmask(struct crypto_tfm *tfm)return tfm->__crt_alg->cra_alignmask; } +static inline unsigned int crypto_tfm_alg_req_alignmask (struct crypto_tfm *tfm) +{ +#ifdef ARCH_DMA_MINALIGN +if (ARCH_DMA_MINALIGN >CRYPTO_REQ_MINALIGN && + (tfm->__crt_alg->cra_flags &CRYPTO_ALG_NEED_DMA_ALIGNMENT )) +return ARCH_DMA_MINALIGN -1 ; +#endif +return CRYPTO_REQ_MINALIGN -1 ; +} +static inline u32 crypto_tfm_get_flags (struct crypto_tfm *tfm) {return tfm->crt_flags;
Signature · 1 line
2.30.2
PATCH 3/8 crypto: drivers - set CRYPTO_ALG_NEED_DMA_ALIGNMENT where needed
In reply to Ard BiesheuvelSet the CRYPTO_ALG_NEED_DMA_ALIGNMENT for all algo implementations that might need it, i.e., the CRYPTO_ALG_ASYNC ones that can be built for !X86. (Except CCP, which is used on arm64 but only in a cache coherent manner)
Note that we are erring firmly on the side of caution there: most implementations likely do not DMA into the request context buffer directly, and so the flag might be unnecessary. However, given that setting this flag basically preserves the status quo for these drivers, it is a safe and reasonable default.
--- drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c | 20 +++---- drivers/crypto/amcc/crypto4xx_core.c | 8 +++ drivers/crypto/amlogic/amlogic-gxl-core.c | 2 + drivers/crypto/atmel-aes.c | 2 +- drivers/crypto/atmel-sha.c | 2 +- drivers/crypto/atmel-tdes.c | 2 +- drivers/crypto/axis/artpec6_crypto.c | 8 +++ drivers/crypto/bcm/cipher.c | 23 +++++++- drivers/crypto/caam/caamalg.c | 4 +- drivers/crypto/caam/caamalg_qi.c | 2 + drivers/crypto/caam/caamalg_qi2.c | 4 +- drivers/crypto/caam/caamhash.c | 3 +- drivers/crypto/cavium/cpt/cptvf_algs.c | 6 ++ drivers/crypto/cavium/nitrox/nitrox_aead.c | 6 +- drivers/crypto/cavium/nitrox/nitrox_skcipher.c | 24 +++++--- drivers/crypto/ccree/cc_aead.c | 3 + drivers/crypto/ccree/cc_cipher.c | 3 + drivers/crypto/ccree/cc_hash.c | 6 ++ drivers/crypto/chelsio/chcr_algo.c | 5 +- drivers/crypto/gemini/sl3516-ce-core.c | 1 + drivers/crypto/hifn_795x.c | 4 +- drivers/crypto/hisilicon/sec/sec_algs.c | 8 +++ drivers/crypto/hisilicon/sec2/sec_crypto.c | 2 + drivers/crypto/inside-secure/safexcel_cipher.c | 47 +++++++++++++++ drivers/crypto/inside-secure/safexcel_hash.c | 26 +++++++++ drivers/crypto/ixp4xx_crypto.c | 2 + drivers/crypto/keembay/keembay-ocs-aes-core.c | 12 ++++ drivers/crypto/keembay/keembay-ocs-hcu-core.c | 30 ++++++---- drivers/crypto/marvell/cesa/cipher.c | 6 ++ drivers/crypto/marvell/cesa/hash.c | 6 ++ drivers/crypto/marvell/octeontx/otx_cptvf_algs.c | 60 +++++++++++++++----- drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c | 52 ++++++++++++----- drivers/crypto/mxs-dcp.c | 8 ++- drivers/crypto/n2_core.c | 1 + drivers/crypto/omap-aes.c | 5 ++ drivers/crypto/omap-des.c | 4 ++ drivers/crypto/omap-sham.c | 12 ++++ drivers/crypto/qce/aead.c | 1 + drivers/crypto/qce/sha.c | 3 +- drivers/crypto/qce/skcipher.c | 1 + drivers/crypto/rockchip/rk3288_crypto_ahash.c | 3 + drivers/crypto/rockchip/rk3288_crypto_skcipher.c | 18 ++++-- drivers/crypto/s5p-sss.c | 6 ++ drivers/crypto/sa2ul.c | 9 +++ drivers/crypto/sahara.c | 14 +++-- drivers/crypto/stm32/stm32-cryp.c | 27 ++++++--- drivers/crypto/stm32/stm32-hash.c | 8 +++ drivers/crypto/talitos.c | 40 +++++++++++++ drivers/crypto/ux500/cryp/cryp_core.c | 21 ++++--- drivers/crypto/ux500/hash/hash_core.c | 12 ++-- drivers/crypto/xilinx/zynqmp-aes-gcm.c | 1 + 51 files changed, 482 insertions(+), 101 deletions(-)
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c +10 -10
@@ -284,7 +284,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), .cra_module = THIS_MODULE, .cra_alignmask = 0xf,
@@ -311,7 +311,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), .cra_module = THIS_MODULE, .cra_alignmask = 0xf,
@@ -337,7 +337,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), .cra_module = THIS_MODULE, .cra_alignmask = 0xf,
@@ -364,7 +364,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), .cra_module = THIS_MODULE, .cra_alignmask = 0xf,
@@ -399,7 +399,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = MD5_HMAC_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
@@ -429,7 +429,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
@@ -459,7 +459,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
@@ -489,7 +489,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
@@ -519,7 +519,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA384_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
@@ -549,7 +549,7 @@ static struct sun8i_ce_alg_template ce_algs[] = { .cra_alignmask = 3, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, + CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA512_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sun8i_ce_hash_tfm_ctx), .cra_module = THIS_MODULE,
drivers/crypto/amcc/crypto4xx_core.c +8 -0
@@ -1197,6 +1197,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "cbc-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1217,6 +1218,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "cfb-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1238,6 +1240,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1258,6 +1261,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "rfc3686-ctr-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1278,6 +1282,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "ecb-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1297,6 +1302,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "ofb-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct crypto4xx_ctx),
@@ -1327,6 +1333,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "ccm-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_NEED_FALLBACK | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1,
@@ -1348,6 +1355,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = { .cra_driver_name = "gcm-aes-ppc4xx", .cra_priority = CRYPTO4XX_CRYPTO_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_NEED_FALLBACK | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1,
drivers/crypto/amlogic/amlogic-gxl-core.c +2 -0
@@ -55,6 +55,7 @@ static struct meson_alg_template mc_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx), .cra_module = THIS_MODULE,
@@ -81,6 +82,7 @@ static struct meson_alg_template mc_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx), .cra_module = THIS_MODULE,
drivers/crypto/atmel-aes.c +1 -1
@@ -2422,7 +2422,7 @@ static void atmel_aes_unregister_algs(struct atmel_aes_dev *dd)static void atmel_aes_crypto_alg_init (struct crypto_alg *alg) { - alg->cra_flags |=CRYPTO_ALG_ASYNC ; + alg->cra_flags |=CRYPTO_ALG_ASYNC |CRYPTO_ALG_NEED_DMA_ALIGNMENT ; alg->cra_alignmask =0xf ; alg->cra_priority =ATMEL_AES_PRIORITY ; alg->cra_module =THIS_MODULE ;
drivers/crypto/atmel-sha.c +1 -1
@@ -1254,7 +1254,7 @@ static int atmel_sha_cra_init(struct crypto_tfm *tfm)static void atmel_sha_alg_init (struct ahash_alg *alg) { alg->halg.base.cra_priority =ATMEL_SHA_PRIORITY ; - alg->halg.base.cra_flags =CRYPTO_ALG_ASYNC ; + alg->halg.base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_NEED_DMA_ALIGNMENT ; alg->halg.base.cra_ctxsize =sizeof (struct atmel_sha_ctx ); alg->halg.base.cra_module =THIS_MODULE ; alg->halg.base.cra_init = atmel_sha_cra_init;
drivers/crypto/atmel-tdes.c +1 -1
@@ -903,7 +903,7 @@ static int atmel_tdes_init_tfm(struct crypto_skcipher *tfm)static void atmel_tdes_skcipher_alg_init (struct skcipher_alg *alg) { alg->base.cra_priority =ATMEL_TDES_PRIORITY ; - alg->base.cra_flags =CRYPTO_ALG_ASYNC ; + alg->base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_NEED_DMA_ALIGNMENT ; alg->base.cra_ctxsize =sizeof (struct atmel_tdes_ctx ); alg->base.cra_module =THIS_MODULE ;
drivers/crypto/axis/artpec6_crypto.c +8 -0
@@ -2632,6 +2632,7 @@ static struct ahash_alg hash_algos[] = { .cra_driver_name = "artpec-sha1", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct artpec6_hashalg_context),
@@ -2656,6 +2657,7 @@ static struct ahash_alg hash_algos[] = { .cra_driver_name = "artpec-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct artpec6_hashalg_context),
@@ -2681,6 +2683,7 @@ static struct ahash_alg hash_algos[] = { .cra_driver_name = "artpec-hmac-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct artpec6_hashalg_context),
@@ -2701,6 +2704,7 @@ static struct skcipher_alg crypto_algos[] = { .cra_driver_name = "artpec6-ecb-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct artpec6_cryptotfm_context),
@@ -2722,6 +2726,7 @@ static struct skcipher_alg crypto_algos[] = { .cra_driver_name = "artpec6-ctr-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = 1,
@@ -2745,6 +2750,7 @@ static struct skcipher_alg crypto_algos[] = { .cra_driver_name = "artpec6-cbc-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct artpec6_cryptotfm_context),
@@ -2767,6 +2773,7 @@ static struct skcipher_alg crypto_algos[] = { .cra_driver_name = "artpec6-xts-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct artpec6_cryptotfm_context),
@@ -2798,6 +2805,7 @@ static struct aead_alg aead_algos[] = { .cra_driver_name = "artpec-gcm-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1,
drivers/crypto/bcm/cipher.c +22 -1
@@ -3183,6 +3183,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3208,6 +3209,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3233,6 +3235,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3258,6 +3261,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3283,6 +3287,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3308,6 +3313,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3333,6 +3339,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3358,6 +3365,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3383,6 +3391,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3408,6 +3417,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3433,6 +3443,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3458,6 +3469,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3483,6 +3495,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3508,6 +3521,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3533,6 +3547,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_NEED_FALLBACK |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY }, .setkey = aead_authenc_setkey,
@@ -3771,6 +3786,7 @@ static struct iproc_alg_s driver_algs[] = { .cra_driver_name = "md5-iproc", .cra_blocksize = MD5_BLOCK_WORDS * 4, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -4270,6 +4286,7 @@ static int aead_cra_init(struct crypto_aead *aead) ctx->fallback_cipher =crypto_alloc_aead (alg->cra_name,0 ,CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_NEED_FALLBACK );if (IS_ERR (ctx->fallback_cipher)) { pr_err("%s() Error: failed to allocate fallback for %s\n",
@@ -4463,6 +4480,7 @@ static int spu_register_skcipher(struct iproc_alg_s *driver_alg) crypto->base.cra_alignmask =0 ; crypto->base.cra_ctxsize =sizeof (struct iproc_ctx_s ); crypto->base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_KERN_DRIVER_ONLY ;
@@ -4504,6 +4522,7 @@ static int spu_register_ahash(struct iproc_alg_s *driver_alg) hash->halg.base.cra_init = ahash_cra_init; hash->halg.base.cra_exit = generic_cra_exit; hash->halg.base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY ; hash->halg.statesize =sizeof (struct spu_hash_export_s );
@@ -4548,7 +4567,9 @@ static int spu_register_aead(struct iproc_alg_s *driver_alg) aead->base.cra_alignmask =0 ; aead->base.cra_ctxsize =sizeof (struct iproc_ctx_s ); - aead->base.cra_flags |=CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY ; + aead->base.cra_flags |=CRYPTO_ALG_ASYNC | +CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT ;/* setkey set in alg initialization */ aead->setauthsize = aead_setauthsize; aead->encrypt = aead_encrypt;
drivers/crypto/caam/caamalg.c +3 -1
@@ -3493,7 +3493,8 @@ static void caam_skcipher_alg_init(struct caam_skcipher_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags |= (CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | -CRYPTO_ALG_KERN_DRIVER_ONLY ); +CRYPTO_ALG_NEED_DMA_ALIGNMENT | +CRYPTO_ALG_KERN_DRIVER_ONLY ); alg->init = caam_cra_init; alg->exit = caam_cra_exit;
@@ -3507,6 +3508,7 @@ static void caam_aead_alg_init(struct caam_aead_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY ; alg->init = caam_aead_init;
drivers/crypto/caam/caamalg_qi.c +2 -0
@@ -2581,6 +2581,7 @@ static void caam_skcipher_alg_init(struct caam_skcipher_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags |= (CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY ); alg->init = caam_cra_init;
@@ -2595,6 +2596,7 @@ static void caam_aead_alg_init(struct caam_aead_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY ; alg->init = caam_aead_init;
drivers/crypto/caam/caamalg_qi2.c +3 -1
@@ -3010,7 +3010,8 @@ static void caam_skcipher_alg_init(struct caam_skcipher_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags |= (CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | -CRYPTO_ALG_KERN_DRIVER_ONLY ); +CRYPTO_ALG_NEED_DMA_ALIGNMENT | +CRYPTO_ALG_KERN_DRIVER_ONLY ); alg->init = caam_cra_init_skcipher; alg->exit = caam_cra_exit;
@@ -3024,6 +3025,7 @@ static void caam_aead_alg_init(struct caam_aead_alg *t_alg) alg->base.cra_priority =CAAM_CRA_PRIORITY ; alg->base.cra_ctxsize =sizeof (struct caam_ctx ); alg->base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY ; alg->init = caam_cra_init_aead;
drivers/crypto/caam/caamhash.c +2 -1
@@ -1930,7 +1930,8 @@ caam_hash_alloc(struct caam_hash_template *template, alg->cra_priority =CAAM_CRA_PRIORITY ; alg->cra_blocksize = template->blocksize; alg->cra_alignmask =0 ; - alg->cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY ; + alg->cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; t_alg->alg_type = template->alg_type;
drivers/crypto/cavium/cpt/cptvf_algs.c +6 -0
@@ -342,6 +342,7 @@ static int cvm_enc_dec_init(struct crypto_skcipher *tfm)static struct skcipher_alg algs[] = { { .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY , .base.cra_blocksize =AES_BLOCK_SIZE , .base.cra_ctxsize =sizeof (struct cvm_enc_ctx ),
@@ -360,6 +361,7 @@ static struct skcipher_alg algs[] = { { .init = cvm_enc_dec_init, }, { .base.cra_flags = CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cvm_enc_ctx),
@@ -378,6 +380,7 @@ static struct skcipher_alg algs[] = { { .init = cvm_enc_dec_init, }, { .base.cra_flags = CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cvm_enc_ctx),
@@ -395,6 +398,7 @@ static struct skcipher_alg algs[] = { { .init = cvm_enc_dec_init, }, { .base.cra_flags = CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cvm_enc_ctx),
@@ -413,6 +417,7 @@ static struct skcipher_alg algs[] = { { .init = cvm_enc_dec_init, }, { .base.cra_flags = CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cvm_des3_ctx),
@@ -431,6 +436,7 @@ static struct skcipher_alg algs[] = { { .init = cvm_enc_dec_init, }, { .base.cra_flags = CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cvm_des3_ctx),
drivers/crypto/cavium/nitrox/nitrox_aead.c +4 -2
@@ -521,7 +521,8 @@ static struct aead_alg nitrox_aeads[] = { { .cra_name = "gcm(aes)", .cra_driver_name = "n5_aes_gcm", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -540,7 +541,8 @@ static struct aead_alg nitrox_aeads[] = { { .cra_name = "rfc4106(gcm(aes))", .cra_driver_name = "n5_rfc4106", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
drivers/crypto/cavium/nitrox/nitrox_skcipher.c +16 -8
@@ -388,7 +388,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "cbc(aes)", .cra_driver_name = "n5_cbc(aes)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -407,7 +408,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "ecb(aes)", .cra_driver_name = "n5_ecb(aes)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -426,7 +428,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "cfb(aes)", .cra_driver_name = "n5_cfb(aes)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -445,7 +448,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "xts(aes)", .cra_driver_name = "n5_xts(aes)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -464,7 +468,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "rfc3686(ctr(aes))", .cra_driver_name = "n5_rfc3686(ctr(aes))", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -483,7 +488,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "cts(cbc(aes))", .cra_driver_name = "n5_cts(cbc(aes))", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -502,7 +508,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "cbc(des3_ede)", .cra_driver_name = "n5_cbc(des3_ede)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
@@ -521,7 +528,8 @@ static struct skcipher_alg nitrox_skciphers[] = { { .cra_name = "ecb(des3_ede)", .cra_driver_name = "n5_ecb(des3_ede)", .cra_priority = PRIO, - .cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY, + .cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct nitrox_crypto_ctx), .cra_alignmask = 0,
drivers/crypto/ccree/cc_aead.c +3 -0
@@ -2642,6 +2642,9 @@ int cc_aead_alloc(struct cc_drvdata *drvdata) aead_algs[alg].driver_name); goto fail1; } +if (!drvdata->coherent) + t_alg->aead_alg.base.cra_flags |= +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; t_alg->drvdata = drvdata; rc =crypto_register_aead (&t_alg->aead_alg);if (rc) {
drivers/crypto/ccree/cc_cipher.c +3 -0
@@ -1484,6 +1484,9 @@ int cc_cipher_alloc(struct cc_drvdata *drvdata) skcipher_algs[alg].driver_name); goto fail0; } +if (!drvdata->coherent) + t_alg->skcipher_alg.base.cra_flags |= +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; t_alg->drvdata = drvdata; dev_dbg(dev, "registering %s\n",
drivers/crypto/ccree/cc_hash.c +6 -0
@@ -2013,6 +2013,9 @@ int cc_hash_alloc(struct cc_drvdata *drvdata) driver_hash[alg].driver_name); goto fail; } +if (!drvdata->coherent) + t_alg->ahash_alg.halg.base.cra_flags |= +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; t_alg->drvdata = drvdata; rc =crypto_register_ahash (&t_alg->ahash_alg);
@@ -2036,6 +2039,9 @@ int cc_hash_alloc(struct cc_drvdata *drvdata) driver_hash[alg].driver_name); goto fail; } +if (!drvdata->coherent) + t_alg->ahash_alg.halg.base.cra_flags |= +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; t_alg->drvdata = drvdata; rc =crypto_register_ahash (&t_alg->ahash_alg);
drivers/crypto/chelsio/chcr_algo.c +4 -1
@@ -4441,6 +4441,7 @@ static int chcr_register_alg(void) driver_algs[i].alg.skcipher.base.cra_module =THIS_MODULE ; driver_algs[i].alg.skcipher.base.cra_flags =CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_NEED_FALLBACK ; driver_algs[i].alg.skcipher.base.cra_ctxsize =
@@ -4454,6 +4455,7 @@ static int chcr_register_alg(void)case CRYPTO_ALG_TYPE_AEAD : driver_algs[i].alg.aead.base.cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_NEED_FALLBACK | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY ; driver_algs[i].alg.aead.encrypt = chcr_aead_encrypt; driver_algs[i].alg.aead.decrypt = chcr_aead_decrypt;
@@ -4475,7 +4477,8 @@ static int chcr_register_alg(void) a_hash->halg.base.cra_priority =CHCR_CRA_PRIORITY ; a_hash->halg.base.cra_module =THIS_MODULE ; a_hash->halg.base.cra_flags = -CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY ; +CRYPTO_ALG_ASYNC |CRYPTO_ALG_ALLOCATES_MEMORY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; a_hash->halg.base.cra_alignmask =0 ; a_hash->halg.base.cra_exit =NULL ;
drivers/crypto/gemini/sl3516-ce-core.c +1 -0
@@ -224,6 +224,7 @@ static struct sl3516_ce_alg_template ce_algs[] = { .cra_priority = 400, .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_TYPE_SKCIPHER | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sl3516_ce_cipher_tfm_ctx), .cra_module = THIS_MODULE,
drivers/crypto/hifn_795x.c +3 -1
@@ -2398,7 +2398,9 @@ static int hifn_alg_alloc(struct hifn_device *dev, const struct hifn_alg_templat t->drv_name, dev->name); alg->alg.base.cra_priority =300 ; - alg->alg.base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC ; + alg->alg.base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; alg->alg.base.cra_blocksize = t->bsize; alg->alg.base.cra_ctxsize =sizeof (struct hifn_context ); alg->alg.base.cra_alignmask =0 ;
drivers/crypto/hisilicon/sec/sec_algs.c +8 -0
@@ -935,6 +935,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_aes_ecb", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -955,6 +956,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_aes_cbc", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -975,6 +977,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_aes_ctr", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -995,6 +998,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_aes_xts", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -1016,6 +1020,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_des_ecb", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -1036,6 +1041,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_des_cbc", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -1056,6 +1062,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_3des_cbc", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
@@ -1076,6 +1083,7 @@ static struct skcipher_alg sec_algs[] = { .cra_driver_name = "hisi_sec_3des_ecb", .cra_priority = 4001, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sec_alg_tfm_ctx),
drivers/crypto/hisilicon/sec2/sec_crypto.c +2 -0
@@ -2113,6 +2113,7 @@ static int sec_skcipher_decrypt(struct skcipher_request *sk_req) .cra_driver_name = "hisi_sec_"sec_cra_name,\ .cra_priority = SEC_PRIORITY,\ .cra_flags =CRYPTO_ALG_ASYNC |\ +CRYPTO_ALG_NEED_DMA_ALIGNMENT |\CRYPTO_ALG_ALLOCATES_MEMORY |\ CRYPTO_ALG_NEED_FALLBACK,\ .cra_blocksize = blk_size,\
@@ -2366,6 +2367,7 @@ static int sec_aead_decrypt(struct aead_request *a_req) .cra_driver_name = "hisi_sec_"sec_cra_name,\ .cra_priority = SEC_PRIORITY,\ .cra_flags =CRYPTO_ALG_ASYNC |\ +CRYPTO_ALG_NEED_DMA_ALIGNMENT |\CRYPTO_ALG_ALLOCATES_MEMORY |\ CRYPTO_ALG_NEED_FALLBACK,\ .cra_blocksize = blk_size,\
drivers/crypto/inside-secure/safexcel_cipher.c +47 -0
329 lines not on this page · raw message
drivers/crypto/inside-secure/safexcel_hash.c +26 -0
@@ -997,6 +997,7 @@ struct safexcel_alg_template safexcel_alg_sha1 = { .cra_driver_name = "safexcel-sha1", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE,
@@ -1249,6 +1250,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha1 = { .cra_driver_name = "safexcel-hmac-sha1", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE,
@@ -1306,6 +1308,7 @@ struct safexcel_alg_template safexcel_alg_sha256 = { .cra_driver_name = "safexcel-sha256", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE,
@@ -1363,6 +1366,7 @@ struct safexcel_alg_template safexcel_alg_sha224 = { .cra_driver_name = "safexcel-sha224", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA224_BLOCK_SIZE,
@@ -1435,6 +1439,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha224 = { .cra_driver_name = "safexcel-hmac-sha224", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA224_BLOCK_SIZE,
@@ -1507,6 +1512,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha256 = { .cra_driver_name = "safexcel-hmac-sha256", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE,
@@ -1564,6 +1570,7 @@ struct safexcel_alg_template safexcel_alg_sha512 = { .cra_driver_name = "safexcel-sha512", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA512_BLOCK_SIZE,
@@ -1621,6 +1628,7 @@ struct safexcel_alg_template safexcel_alg_sha384 = { .cra_driver_name = "safexcel-sha384", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA384_BLOCK_SIZE,
@@ -1693,6 +1701,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha512 = { .cra_driver_name = "safexcel-hmac-sha512", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA512_BLOCK_SIZE,
@@ -1765,6 +1774,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha384 = { .cra_driver_name = "safexcel-hmac-sha384", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA384_BLOCK_SIZE,
@@ -1822,6 +1832,7 @@ struct safexcel_alg_template safexcel_alg_md5 = { .cra_driver_name = "safexcel-md5", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
@@ -1895,6 +1906,7 @@ struct safexcel_alg_template safexcel_alg_hmac_md5 = { .cra_driver_name = "safexcel-hmac-md5", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
@@ -1977,6 +1989,7 @@ struct safexcel_alg_template safexcel_alg_crc32 = { .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_OPTIONAL_KEY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1,
@@ -2067,6 +2080,7 @@ struct safexcel_alg_template safexcel_alg_cbcmac = { .cra_driver_name = "safexcel-cbcmac-aes", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1,
@@ -2162,6 +2176,7 @@ struct safexcel_alg_template safexcel_alg_xcbcmac = { .cra_driver_name = "safexcel-xcbc-aes", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = AES_BLOCK_SIZE,
@@ -2258,6 +2273,7 @@ struct safexcel_alg_template safexcel_alg_cmac = { .cra_driver_name = "safexcel-cmac-aes", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = AES_BLOCK_SIZE,
@@ -2315,6 +2331,7 @@ struct safexcel_alg_template safexcel_alg_sm3 = { .cra_driver_name = "safexcel-sm3", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SM3_BLOCK_SIZE,
@@ -2387,6 +2404,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sm3 = { .cra_driver_name = "safexcel-hmac-sm3", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SM3_BLOCK_SIZE,
@@ -2580,6 +2598,7 @@ struct safexcel_alg_template safexcel_alg_sha3_224 = { .cra_driver_name = "safexcel-sha3-224", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_224_BLOCK_SIZE,
@@ -2638,6 +2657,7 @@ struct safexcel_alg_template safexcel_alg_sha3_256 = { .cra_driver_name = "safexcel-sha3-256", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_256_BLOCK_SIZE,
@@ -2696,6 +2716,7 @@ struct safexcel_alg_template safexcel_alg_sha3_384 = { .cra_driver_name = "safexcel-sha3-384", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_384_BLOCK_SIZE,
@@ -2754,6 +2775,7 @@ struct safexcel_alg_template safexcel_alg_sha3_512 = { .cra_driver_name = "safexcel-sha3-512", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_512_BLOCK_SIZE,
@@ -2917,6 +2939,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha3_224 = { .cra_driver_name = "safexcel-hmac-sha3-224", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_224_BLOCK_SIZE,
@@ -2988,6 +3011,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha3_256 = { .cra_driver_name = "safexcel-hmac-sha3-256", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_256_BLOCK_SIZE,
@@ -3059,6 +3083,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha3_384 = { .cra_driver_name = "safexcel-hmac-sha3-384", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_384_BLOCK_SIZE,
@@ -3129,6 +3154,7 @@ struct safexcel_alg_template safexcel_alg_hmac_sha3_512 = { .cra_driver_name = "safexcel-hmac-sha3-512", .cra_priority = SAFEXCEL_CRA_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA3_512_BLOCK_SIZE,
drivers/crypto/ixp4xx_crypto.c +2 -0
@@ -1503,6 +1503,7 @@ static int ixp_crypto_probe(struct platform_device *_pdev)/* block ciphers */ cra->base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_NEED_FALLBACK ;if (!cra->setkey)
@@ -1538,6 +1539,7 @@ static int ixp_crypto_probe(struct platform_device *_pdev)/* authenc */ cra->base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY ; cra->setkey = cra->setkey ?: aead_setkey; cra->setauthsize = aead_setauthsize;
drivers/crypto/keembay/keembay-ocs-aes-core.c +12 -0
@@ -1287,6 +1287,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ecb-aes-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE,
@@ -1308,6 +1309,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "cbc-aes-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE,
@@ -1329,6 +1331,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ctr-aes-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = 1,
@@ -1351,6 +1354,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "cts-aes-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE,
@@ -1374,6 +1378,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ecb-sm4-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct ocs_aes_tctx),
@@ -1394,6 +1399,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "cbc-sm4-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct ocs_aes_tctx),
@@ -1414,6 +1420,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ctr-sm4-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = 1, .base.cra_ctxsize = sizeof(struct ocs_aes_tctx),
@@ -1435,6 +1442,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "cts-sm4-keembay-ocs", .base.cra_priority = KMB_OCS_PRIORITY, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct ocs_aes_tctx),
@@ -1460,6 +1468,7 @@ static struct aead_alg algs_aead[] = { .cra_driver_name = "gcm-aes-keembay-ocs", .cra_priority = KMB_OCS_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = 1,
@@ -1482,6 +1491,7 @@ static struct aead_alg algs_aead[] = { .cra_driver_name = "ccm-aes-keembay-ocs", .cra_priority = KMB_OCS_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = 1,
@@ -1504,6 +1514,7 @@ static struct aead_alg algs_aead[] = { .cra_driver_name = "gcm-sm4-keembay-ocs", .cra_priority = KMB_OCS_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct ocs_aes_tctx),
@@ -1525,6 +1536,7 @@ static struct aead_alg algs_aead[] = { .cra_driver_name = "ccm-sm4-keembay-ocs", .cra_priority = KMB_OCS_PRIORITY, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct ocs_aes_tctx),
drivers/crypto/keembay/keembay-ocs-hcu-core.c +20 -10
@@ -900,7 +900,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "sha224", .cra_driver_name = "sha224-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -925,7 +926,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "hmac(sha224)", .cra_driver_name = "hmac-sha224-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -951,7 +953,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "sha256", .cra_driver_name = "sha256-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -976,7 +979,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "hmac(sha256)", .cra_driver_name = "hmac-sha256-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1001,7 +1005,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "sm3", .cra_driver_name = "sm3-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SM3_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1026,7 +1031,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "hmac(sm3)", .cra_driver_name = "hmac-sm3-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SM3_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1051,7 +1057,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "sha384", .cra_driver_name = "sha384-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA384_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1076,7 +1083,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "hmac(sha384)", .cra_driver_name = "hmac-sha384-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA384_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1101,7 +1109,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "sha512", .cra_driver_name = "sha512-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA512_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
@@ -1126,7 +1135,8 @@ static struct ahash_alg ocs_hcu_algs[] = { .cra_name = "hmac(sha512)", .cra_driver_name = "hmac-sha512-keembay-ocs", .cra_priority = 255, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA512_BLOCK_SIZE, .cra_ctxsize = sizeof(struct ocs_hcu_ctx), .cra_alignmask = 0,
drivers/crypto/marvell/cesa/cipher.c +6 -0
@@ -520,6 +520,7 @@ struct skcipher_alg mv_cesa_ecb_des_alg = { .cra_driver_name = "mv-ecb-des", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_des_ctx),
@@ -571,6 +572,7 @@ struct skcipher_alg mv_cesa_cbc_des_alg = { .cra_driver_name = "mv-cbc-des", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_des_ctx),
@@ -630,6 +632,7 @@ struct skcipher_alg mv_cesa_ecb_des3_ede_alg = { .cra_driver_name = "mv-ecb-des3-ede", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_des3_ctx),
@@ -684,6 +687,7 @@ struct skcipher_alg mv_cesa_cbc_des3_ede_alg = { .cra_driver_name = "mv-cbc-des3-ede", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_des3_ctx),
@@ -757,6 +761,7 @@ struct skcipher_alg mv_cesa_ecb_aes_alg = { .cra_driver_name = "mv-ecb-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_aes_ctx),
@@ -807,6 +812,7 @@ struct skcipher_alg mv_cesa_cbc_aes_alg = { .cra_driver_name = "mv-cbc-aes", .cra_priority = 300, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .cra_blocksize = AES_BLOCK_SIZE, .cra_ctxsize = sizeof(struct mv_cesa_aes_ctx),
drivers/crypto/marvell/cesa/hash.c +6 -0
@@ -949,6 +949,7 @@ struct ahash_alg mv_md5_alg = { .cra_driver_name = "mv-md5", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
@@ -1020,6 +1021,7 @@ struct ahash_alg mv_sha1_alg = { .cra_driver_name = "mv-sha1", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE,
@@ -1094,6 +1096,7 @@ struct ahash_alg mv_sha256_alg = { .cra_driver_name = "mv-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE,
@@ -1329,6 +1332,7 @@ struct ahash_alg mv_ahmac_md5_alg = { .cra_driver_name = "mv-hmac-md5", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE,
@@ -1400,6 +1404,7 @@ struct ahash_alg mv_ahmac_sha1_alg = { .cra_driver_name = "mv-hmac-sha1", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE,
@@ -1471,6 +1476,7 @@ struct ahash_alg mv_ahmac_sha256_alg = { .cra_driver_name = "mv-hmac-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE,
drivers/crypto/marvell/octeontx/otx_cptvf_algs.c +45 -15
150 lines not on this page · raw message
drivers/crypto/marvell/octeontx2/otx2_cptvf_algs.c +38 -14
136 lines not on this page · raw message
drivers/crypto/mxs-dcp.c +6 -2
@@ -858,6 +858,7 @@ static struct skcipher_alg dcp_aes_algs[] = { .base.cra_priority = 400, .base.cra_alignmask = 15, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct dcp_async_ctx),
@@ -876,6 +877,7 @@ static struct skcipher_alg dcp_aes_algs[] = { .base.cra_priority = 400, .base.cra_alignmask = 15, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct dcp_async_ctx),
@@ -909,7 +911,8 @@ static struct ahash_alg dcp_sha1_alg = { .cra_driver_name = "sha1-dcp", .cra_priority = 400, .cra_alignmask = 63, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct dcp_async_ctx), .cra_module = THIS_MODULE,
@@ -936,7 +939,8 @@ static struct ahash_alg dcp_sha256_alg = { .cra_driver_name = "sha256-dcp", .cra_priority = 400, .cra_alignmask = 63, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct dcp_async_ctx), .cra_module = THIS_MODULE,
drivers/crypto/n2_core.c +1 -0
@@ -1338,6 +1338,7 @@ static int __n2_register_one_skcipher(const struct n2_skcipher_tmpl *tmpl)snprintf (alg->base.cra_driver_name,CRYPTO_MAX_ALG_NAME ,"%s-n2" , tmpl->drv_name); alg->base.cra_priority =N2_CRA_PRIORITY ; alg->base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY ; alg->base.cra_blocksize = tmpl->block_size; p->enc_type = tmpl->enc_type;
drivers/crypto/omap-aes.c +5 -0
@@ -675,6 +675,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_aes_ctx),
@@ -694,6 +695,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_aes_ctx),
@@ -717,6 +719,7 @@ static struct skcipher_alg algs_ctr[] = { .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = 1, .base.cra_ctxsize = sizeof(struct omap_aes_ctx),
@@ -747,6 +750,7 @@ static struct aead_alg algs_aead_gcm[] = { .cra_driver_name = "gcm-aes-omap", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct omap_aes_gcm_ctx),
@@ -767,6 +771,7 @@ static struct aead_alg algs_aead_gcm[] = { .cra_driver_name = "rfc4106-gcm-aes-omap", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct omap_aes_gcm_ctx),
drivers/crypto/omap-des.c +4 -0
@@ -737,6 +737,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_driver_name = "ecb-des-omap", .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ASYNC, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_des_ctx),
@@ -754,6 +755,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_driver_name = "cbc-des-omap", .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ASYNC, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_des_ctx),
@@ -772,6 +774,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_driver_name = "ecb-des3-omap", .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ASYNC, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_des_ctx),
@@ -789,6 +792,7 @@ static struct skcipher_alg algs_ecb_cbc[] = { .base.cra_driver_name = "cbc-des3-omap", .base.cra_priority = 300, .base.cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ASYNC, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct omap_des_ctx),
drivers/crypto/omap-sham.c +12 -0
@@ -1437,6 +1437,7 @@ static struct ahash_alg algs_sha1_md5[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1459,6 +1460,7 @@ static struct ahash_alg algs_sha1_md5[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1482,6 +1484,7 @@ static struct ahash_alg algs_sha1_md5[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
@@ -1506,6 +1509,7 @@ static struct ahash_alg algs_sha1_md5[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
@@ -1533,6 +1537,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1555,6 +1560,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1578,6 +1584,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
@@ -1602,6 +1609,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
@@ -1628,6 +1636,7 @@ static struct ahash_alg algs_sha384_sha512[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA384_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1650,6 +1659,7 @@ static struct ahash_alg algs_sha384_sha512[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA512_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx),
@@ -1673,6 +1683,7 @@ static struct ahash_alg algs_sha384_sha512[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA384_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
@@ -1697,6 +1708,7 @@ static struct ahash_alg algs_sha384_sha512[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA512_BLOCK_SIZE, .cra_ctxsize = sizeof(struct omap_sham_ctx) +
drivers/crypto/qce/aead.c +1 -0
@@ -788,6 +788,7 @@ static int qce_aead_register_one(const struct qce_aead_def *def, struct qce_devi alg->base.cra_priority =300 ; alg->base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_NEED_FALLBACK ;
drivers/crypto/qce/sha.c +2 -1
@@ -481,7 +481,8 @@ static int qce_ahash_register_one(const struct qce_ahash_def *def, base = &alg->halg.base; base->cra_blocksize = def->blocksize; base->cra_priority =300 ; - base->cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_KERN_DRIVER_ONLY ; + base->cra_flags =CRYPTO_ALG_ASYNC |CRYPTO_ALG_KERN_DRIVER_ONLY | +CRYPTO_ALG_NEED_DMA_ALIGNMENT ; base->cra_ctxsize =sizeof (struct qce_sha_ctx ); base->cra_alignmask =0 ; base->cra_module =THIS_MODULE ;
drivers/crypto/qce/skcipher.c +1 -0
@@ -463,6 +463,7 @@ static int qce_skcipher_register_one(const struct qce_skcipher_def *def, alg->base.cra_priority =300 ; alg->base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_KERN_DRIVER_ONLY ; alg->base.cra_ctxsize =sizeof (struct qce_cipher_ctx );
drivers/crypto/rockchip/rk3288_crypto_ahash.c +3 -0
@@ -330,6 +330,7 @@ struct rk_crypto_tmp rk_ahash_sha1 = { .cra_driver_name = "rk-sha1", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct rk_ahash_ctx),
@@ -360,6 +361,7 @@ struct rk_crypto_tmp rk_ahash_sha256 = { .cra_driver_name = "rk-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct rk_ahash_ctx),
@@ -390,6 +392,7 @@ struct rk_crypto_tmp rk_ahash_md5 = { .cra_driver_name = "rk-md5", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct rk_ahash_ctx),
drivers/crypto/rockchip/rk3288_crypto_skcipher.c +12 -6
@@ -406,7 +406,8 @@ struct rk_crypto_tmp rk_ecb_aes_alg = { .base.cra_name = "ecb(aes)", .base.cra_driver_name = "ecb-aes-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x0f,
@@ -428,7 +429,8 @@ struct rk_crypto_tmp rk_cbc_aes_alg = { .base.cra_name = "cbc(aes)", .base.cra_driver_name = "cbc-aes-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x0f,
@@ -451,7 +453,8 @@ struct rk_crypto_tmp rk_ecb_des_alg = { .base.cra_name = "ecb(des)", .base.cra_driver_name = "ecb-des-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x07,
@@ -473,7 +476,8 @@ struct rk_crypto_tmp rk_cbc_des_alg = { .base.cra_name = "cbc(des)", .base.cra_driver_name = "cbc-des-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x07,
@@ -496,7 +500,8 @@ struct rk_crypto_tmp rk_ecb_des3_ede_alg = { .base.cra_name = "ecb(des3_ede)", .base.cra_driver_name = "ecb-des3-ede-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x07,
@@ -518,7 +523,8 @@ struct rk_crypto_tmp rk_cbc_des3_ede_alg = { .base.cra_name = "cbc(des3_ede)", .base.cra_driver_name = "cbc-des3-ede-rk", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct rk_cipher_ctx), .base.cra_alignmask = 0x07,
drivers/crypto/s5p-sss.c +6 -0
@@ -1744,6 +1744,7 @@ static struct ahash_alg algs_sha1_md5_sha256[] = { .cra_priority = 100, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = HASH_BLOCK_SIZE, .cra_ctxsize = sizeof(struct s5p_hash_ctx),
@@ -1769,6 +1770,7 @@ static struct ahash_alg algs_sha1_md5_sha256[] = { .cra_priority = 100, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = HASH_BLOCK_SIZE, .cra_ctxsize = sizeof(struct s5p_hash_ctx),
@@ -1794,6 +1796,7 @@ static struct ahash_alg algs_sha1_md5_sha256[] = { .cra_priority = 100, .cra_flags =CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = HASH_BLOCK_SIZE, .cra_ctxsize = sizeof(struct s5p_hash_ctx),
@@ -2100,6 +2103,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ecb-aes-s5p", .base.cra_priority = 100, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct s5p_aes_ctx),
@@ -2118,6 +2122,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "cbc-aes-s5p", .base.cra_priority = 100, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct s5p_aes_ctx),
@@ -2137,6 +2142,7 @@ static struct skcipher_alg algs[] = { .base.cra_driver_name = "ctr-aes-s5p", .base.cra_priority = 100, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .base.cra_blocksize = 1, .base.cra_ctxsize = sizeof(struct s5p_aes_ctx),
drivers/crypto/sa2ul.c +9 -0
@@ -1979,6 +1979,7 @@ static struct sa_alg_tmpl sa_algs[] = { .base.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sa_tfm_ctx),
@@ -2002,6 +2003,7 @@ static struct sa_alg_tmpl sa_algs[] = { .base.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sa_tfm_ctx),
@@ -2024,6 +2026,7 @@ static struct sa_alg_tmpl sa_algs[] = { .base.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sa_tfm_ctx),
@@ -2047,6 +2050,7 @@ static struct sa_alg_tmpl sa_algs[] = { .base.cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sa_tfm_ctx),
@@ -2069,6 +2073,7 @@ static struct sa_alg_tmpl sa_algs[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA1_BLOCK_SIZE,
@@ -2098,6 +2103,7 @@ static struct sa_alg_tmpl sa_algs[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA256_BLOCK_SIZE,
@@ -2127,6 +2133,7 @@ static struct sa_alg_tmpl sa_algs[] = { .cra_priority = 400, .cra_flags =CRYPTO_ALG_TYPE_AHASH |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK, .cra_blocksize = SHA512_BLOCK_SIZE,
@@ -2158,6 +2165,7 @@ static struct sa_alg_tmpl sa_algs[] = { .cra_flags = CRYPTO_ALG_TYPE_AEAD |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sa_tfm_ctx), .cra_module = THIS_MODULE,
@@ -2185,6 +2193,7 @@ static struct sa_alg_tmpl sa_algs[] = { .cra_flags = CRYPTO_ALG_TYPE_AEAD |CRYPTO_ALG_KERN_DRIVER_ONLY |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_NEED_FALLBACK, .cra_ctxsize = sizeof(struct sa_tfm_ctx), .cra_module = THIS_MODULE,
drivers/crypto/sahara.c +10 -4
@@ -1182,7 +1182,9 @@ static struct skcipher_alg aes_algs[] = { .base.cra_name = "ecb(aes)", .base.cra_driver_name = "sahara-ecb-aes", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_FALLBACK | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sahara_ctx), .base.cra_alignmask = 0x0,
@@ -1199,7 +1201,9 @@ static struct skcipher_alg aes_algs[] = { .base.cra_name = "cbc(aes)", .base.cra_driver_name = "sahara-cbc-aes", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_FALLBACK | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct sahara_ctx), .base.cra_alignmask = 0x0,
@@ -1232,7 +1236,8 @@ static struct ahash_alg sha_v3_algs[] = { .cra_driver_name = "sahara-sha1", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, +CRYPTO_ALG_NEED_FALLBACK | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sahara_ctx), .cra_alignmask = 0,
@@ -1258,7 +1263,8 @@ static struct ahash_alg sha_v4_algs[] = { .cra_driver_name = "sahara-sha256", .cra_priority = 300, .cra_flags =CRYPTO_ALG_ASYNC | - CRYPTO_ALG_NEED_FALLBACK, +CRYPTO_ALG_NEED_FALLBACK | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct sahara_ctx), .cra_alignmask = 0,
drivers/crypto/stm32/stm32-cryp.c +18 -9
@@ -1558,7 +1558,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "ecb(aes)", .base.cra_driver_name = "stm32-ecb-aes", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1575,7 +1576,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "cbc(aes)", .base.cra_driver_name = "stm32-cbc-aes", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1593,7 +1595,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "ctr(aes)", .base.cra_driver_name = "stm32-ctr-aes", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = 1, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1611,7 +1614,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "ecb(des)", .base.cra_driver_name = "stm32-ecb-des", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1628,7 +1632,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "cbc(des)", .base.cra_driver_name = "stm32-cbc-des", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1646,7 +1651,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "ecb(des3_ede)", .base.cra_driver_name = "stm32-ecb-des3", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1663,7 +1669,8 @@ static struct skcipher_alg crypto_algs[] = { .base.cra_name = "cbc(des3_ede)", .base.cra_driver_name = "stm32-cbc-des3", .base.cra_priority = 200, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct stm32_cryp_ctx), .base.cra_alignmask = 0,
@@ -1693,7 +1700,8 @@ static struct aead_alg aead_algs[] = { .cra_name = "gcm(aes)", .cra_driver_name = "stm32-gcm-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct stm32_cryp_ctx), .cra_alignmask = 0,
@@ -1713,7 +1721,8 @@ static struct aead_alg aead_algs[] = { .cra_name = "ccm(aes)", .cra_driver_name = "stm32-ccm-aes", .cra_priority = 200, - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = 1, .cra_ctxsize = sizeof(struct stm32_cryp_ctx), .cra_alignmask = 0,
drivers/crypto/stm32/stm32-hash.c +8 -0
@@ -1138,6 +1138,7 @@ static struct ahash_alg algs_md5_sha1[] = { .cra_driver_name = "stm32-md5", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1164,6 +1165,7 @@ static struct ahash_alg algs_md5_sha1[] = { .cra_driver_name = "stm32-hmac-md5", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = MD5_HMAC_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1189,6 +1191,7 @@ static struct ahash_alg algs_md5_sha1[] = { .cra_driver_name = "stm32-sha1", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1215,6 +1218,7 @@ static struct ahash_alg algs_md5_sha1[] = { .cra_driver_name = "stm32-hmac-sha1", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1243,6 +1247,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_driver_name = "stm32-sha224", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1269,6 +1274,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_driver_name = "stm32-hmac-sha224", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA224_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1294,6 +1300,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_driver_name = "stm32-sha256", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
@@ -1320,6 +1327,7 @@ static struct ahash_alg algs_sha224_sha256[] = { .cra_driver_name = "stm32-hmac-sha256", .cra_priority = 200, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_KERN_DRIVER_ONLY, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct stm32_hash_ctx),
drivers/crypto/talitos.c +40 -0
@@ -2270,6 +2270,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2292,6 +2293,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos-hsna", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2314,6 +2316,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2339,6 +2342,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos-hsna", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2362,6 +2366,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2384,6 +2389,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos-hsna", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2406,6 +2412,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2431,6 +2438,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos-hsna", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2454,6 +2462,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2476,6 +2485,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos-hsna", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2498,6 +2508,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2523,6 +2534,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos-hsna", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2546,6 +2558,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2568,6 +2581,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2591,6 +2605,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2613,6 +2628,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2636,6 +2652,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2658,6 +2675,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-aes-talitos-hsna", .cra_blocksize = AES_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = AES_BLOCK_SIZE,
@@ -2679,6 +2697,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2703,6 +2722,7 @@ static struct talitos_alg_template driver_algs[] = { "cbc-3des-talitos-hsna", .cra_blocksize = DES3_EDE_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, }, .ivsize = DES3_EDE_BLOCK_SIZE,
@@ -2725,6 +2745,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "ecb-aes-talitos", .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = AES_MIN_KEY_SIZE, .max_keysize = AES_MAX_KEY_SIZE,
@@ -2739,6 +2760,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "cbc-aes-talitos", .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = AES_MIN_KEY_SIZE, .max_keysize = AES_MAX_KEY_SIZE,
@@ -2755,6 +2777,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "ctr-aes-talitos", .base.cra_blocksize = 1, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = AES_MIN_KEY_SIZE, .max_keysize = AES_MAX_KEY_SIZE,
@@ -2771,6 +2794,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "ctr-aes-talitos", .base.cra_blocksize = 1, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = AES_MIN_KEY_SIZE, .max_keysize = AES_MAX_KEY_SIZE,
@@ -2787,6 +2811,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "ecb-des-talitos", .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = DES_KEY_SIZE, .max_keysize = DES_KEY_SIZE,
@@ -2801,6 +2826,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "cbc-des-talitos", .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = DES_KEY_SIZE, .max_keysize = DES_KEY_SIZE,
@@ -2817,6 +2843,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "ecb-3des-talitos", .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = DES3_EDE_KEY_SIZE, .max_keysize = DES3_EDE_KEY_SIZE,
@@ -2832,6 +2859,7 @@ static struct talitos_alg_template driver_algs[] = { .base.cra_driver_name = "cbc-3des-talitos", .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, .min_keysize = DES3_EDE_KEY_SIZE, .max_keysize = DES3_EDE_KEY_SIZE,
@@ -2853,6 +2881,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "md5-talitos", .cra_blocksize = MD5_HMAC_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2869,6 +2898,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "sha1-talitos", .cra_blocksize = SHA1_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2885,6 +2915,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "sha224-talitos", .cra_blocksize = SHA224_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2901,6 +2932,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "sha256-talitos", .cra_blocksize = SHA256_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2917,6 +2949,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "sha384-talitos", .cra_blocksize = SHA384_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2933,6 +2966,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "sha512-talitos", .cra_blocksize = SHA512_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2949,6 +2983,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-md5-talitos", .cra_blocksize = MD5_HMAC_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2965,6 +3000,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-sha1-talitos", .cra_blocksize = SHA1_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2981,6 +3017,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-sha224-talitos", .cra_blocksize = SHA224_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -2997,6 +3034,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-sha256-talitos", .cra_blocksize = SHA256_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -3013,6 +3051,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-sha384-talitos", .cra_blocksize = SHA384_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
@@ -3029,6 +3068,7 @@ static struct talitos_alg_template driver_algs[] = { .cra_driver_name = "hmac-sha512-talitos", .cra_blocksize = SHA512_BLOCK_SIZE, .cra_flags =CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT | CRYPTO_ALG_ALLOCATES_MEMORY, } },
drivers/crypto/ux500/cryp/cryp_core.c +14 -7
@@ -1072,7 +1072,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "ecb(aes)", .base.cra_driver_name = "ecb-aes-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1092,7 +1093,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "cbc(aes)", .base.cra_driver_name = "cbc-aes-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = AES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1113,7 +1115,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "ctr(aes)", .base.cra_driver_name = "ctr-aes-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = 1, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1135,7 +1138,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "ecb(des)", .base.cra_driver_name = "ecb-des-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1155,7 +1159,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "ecb(des3_ede)", .base.cra_driver_name = "ecb-des3_ede-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1175,7 +1180,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "cbc(des)", .base.cra_driver_name = "cbc-des-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
@@ -1196,7 +1202,8 @@ static struct cryp_algo_template cryp_algs[] = { .base.cra_name = "cbc(des3_ede)", .base.cra_driver_name = "cbc-des3_ede-ux500", .base.cra_priority = 300, - .base.cra_flags = CRYPTO_ALG_ASYNC, + .base.cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .base.cra_blocksize = DES3_EDE_BLOCK_SIZE, .base.cra_ctxsize = sizeof(struct cryp_ctx), .base.cra_alignmask = 3,
drivers/crypto/ux500/hash/hash_core.c +8 -4
@@ -1536,7 +1536,8 @@ static struct hash_algo_template hash_algs[] = { .halg.base = { .cra_name = "sha1", .cra_driver_name = "sha1-ux500", - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct hash_ctx), .cra_init = hash_cra_init,
@@ -1559,7 +1560,8 @@ static struct hash_algo_template hash_algs[] = { .halg.base = { .cra_name = "sha256", .cra_driver_name = "sha256-ux500", - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct hash_ctx), .cra_init = hash_cra_init,
@@ -1583,7 +1585,8 @@ static struct hash_algo_template hash_algs[] = { .halg.base = { .cra_name = "hmac(sha1)", .cra_driver_name = "hmac-sha1-ux500", - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_ctxsize = sizeof(struct hash_ctx), .cra_init = hash_cra_init,
@@ -1607,7 +1610,8 @@ static struct hash_algo_template hash_algs[] = { .halg.base = { .cra_name = "hmac(sha256)", .cra_driver_name = "hmac-sha256-ux500", - .cra_flags = CRYPTO_ALG_ASYNC, + .cra_flags =CRYPTO_ALG_ASYNC | + CRYPTO_ALG_NEED_DMA_ALIGNMENT, .cra_blocksize = SHA256_BLOCK_SIZE, .cra_ctxsize = sizeof(struct hash_ctx), .cra_init = hash_cra_init,
drivers/crypto/xilinx/zynqmp-aes-gcm.c +1 -0
@@ -365,6 +365,7 @@ static struct zynqmp_aead_drv_ctx aes_drv_ctx = { .cra_priority = 200, .cra_flags =CRYPTO_ALG_TYPE_AEAD |CRYPTO_ALG_ASYNC | +CRYPTO_ALG_NEED_DMA_ALIGNMENT |CRYPTO_ALG_ALLOCATES_MEMORY |CRYPTO_ALG_KERN_DRIVER_ONLY | CRYPTO_ALG_NEED_FALLBACK,
Signature · 1 line
2.30.2
PATCH 1/8 crypto: add flag for algos that need DMA aligned context buffers
In reply to Ard BiesheuvelOn architectures that support non-coherent DMA, we align and round up all dynamically allocated request and TFM structures to the worst case DMA alignment, which is 128 bytes on arm64, even though most systems only have 64 byte cachelines, are cache coherent for DMA, don't use use accelerators for crypto, or the driver for the accelerator does not DMA into the request context buffer to begin with.
We can relax this requirement, by only performing this rounding for algorithms that are backed by an implementation that actually requires it. So introduce CRYPTO_ALG_NEED_DMA_ALIGNMENT for this purpose, which will be wired up in subsequent patches.
--- include/linux/crypto.h | 7 +++++++ 1 file changed, 7 insertions(+)
include/linux/crypto.h +7 -0
@@ -100,6 +100,13 @@*/ #define CRYPTO_NOLOAD 0x00008000 +/* +* Whether context buffers require DMA alignment. This is the case for +* drivers that perform non-coherent inbound DMA on the context buffer +* directly, but should not be needed otherwise. +*/ +#define CRYPTO_ALG_NEED_DMA_ALIGNMENT 0x00010000 +/* * The algorithm may allocate memory during request processing, i.e. during * encryption, decryption, or hashing. Users can request an algorithm with this
Signature · 1 line
2.30.2
PATCH 8/8 crypto: safexcel - reduce alignment of stack buffer
In reply to Ard BiesheuvelNow that skcipher, AEAD and ahash have all been updated to use a request structure that itself is not aligned for DMA, we can relax the alignment requirement of the EIP197 stack buffers as well.
--- drivers/crypto/inside-secure/safexcel.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/crypto/inside-secure/safexcel.h +1 -1
@@ -66,7 +66,7 @@ CRYPTO_MINALIGN) + \ sizeof(struct safexcel_cipher_req))#define EIP197_REQUEST_ON_STACK (name, type, size) \ - char __##name##_desc[size] CRYPTO_MINALIGN_ATTR; \ + char __##name##_desc[size] CRYPTO_REQ_MINALIGN_ATTR; \ struct type##_request *name = (void *)__##name##_desc/* Xilinx dev board base offsets */
Signature · 1 line
2.30.2
PATCH 4/8 crypto: drivers - avoid setting skcipher TFM reqsize directly
In reply to Ard BiesheuvelThe skcipher TFM reqsize field will be rounded up to DMA alignment and padded so that the resulting request allocations contain context buffers that are safe for DMA if the algo implementation requires it. So avoid setting the field directly, and use the appropriate setter instead.
--- drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c | 6 +++--- drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c | 6 +++--- drivers/crypto/amlogic/amlogic-gxl-cipher.c | 5 +++-- drivers/crypto/gemini/sl3516-ce-cipher.c | 5 +++-- 4 files changed, 12 insertions(+), 10 deletions(-)
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c +3 -3
@@ -395,9 +395,9 @@ int sun8i_ce_cipher_init(struct crypto_tfm *tfm)return PTR_ERR (op->fallback_tfm); } - sktfm->reqsize =sizeof (struct sun8i_cipher_req_ctx ) + -crypto_skcipher_reqsize (op->fallback_tfm); - +crypto_skcipher_set_reqsize (sktfm, +sizeof (struct sun8i_cipher_req_ctx ) + +crypto_skcipher_reqsize (op->fallback_tfm)); dev_info(op->ce->dev, "Fallback for %s is %s\n", crypto_tfm_alg_driver_name(&sktfm->base),
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c +3 -3
@@ -345,9 +345,9 @@ int sun8i_ss_cipher_init(struct crypto_tfm *tfm)return PTR_ERR (op->fallback_tfm); } - sktfm->reqsize =sizeof (struct sun8i_cipher_req_ctx ) + -crypto_skcipher_reqsize (op->fallback_tfm); - +crypto_skcipher_set_reqsize (sktfm, +sizeof (struct sun8i_cipher_req_ctx ) + +crypto_skcipher_reqsize (op->fallback_tfm)); dev_info(op->ss->dev, "Fallback for %s is %s\n", crypto_tfm_alg_driver_name(&sktfm->base),
drivers/crypto/amlogic/amlogic-gxl-cipher.c +3 -2
@@ -328,8 +328,9 @@ int meson_cipher_init(struct crypto_tfm *tfm)return PTR_ERR (op->fallback_tfm); } - sktfm->reqsize =sizeof (struct meson_cipher_req_ctx ) + -crypto_skcipher_reqsize (op->fallback_tfm); +crypto_skcipher_set_reqsize (sktfm, +sizeof (struct meson_cipher_req_ctx ) + +crypto_skcipher_reqsize (op->fallback_tfm)); op->enginectx.op.do_one_request = meson_handle_cipher_request; op->enginectx.op.prepare_request =NULL ;
drivers/crypto/gemini/sl3516-ce-cipher.c +3 -2
@@ -328,8 +328,9 @@ int sl3516_ce_cipher_init(struct crypto_tfm *tfm)return PTR_ERR (op->fallback_tfm); } - sktfm->reqsize =sizeof (struct sl3516_ce_cipher_req_ctx ) + -crypto_skcipher_reqsize (op->fallback_tfm); +crypto_skcipher_set_reqsize (sktfm, +sizeof (struct sl3516_ce_cipher_req_ctx ) + +crypto_skcipher_reqsize (op->fallback_tfm)); dev_info(op->ce->dev, "Fallback for %s is %s\n", crypto_tfm_alg_driver_name(&sktfm->base),
Signature · 1 line
2.30.2
PATCH 6/8 crypto: aead - avoid DMA alignment for request structures unless needed
In reply to Ard BiesheuvelAEAD request structures are currently aligned to minimal DMA alignment for the arcitecture, which defaults to 128 bytes on arm64. This is excessive, and rarely needed, i.e., only when doing non-coherent inbound DMA on the contents of the request context buffer. So let's relax this requirement, and only use this alignment if the CRYPTO_ALG_NEED_DMA_ALIGNMENT flag is set by the implementation.
--- include/crypto/aead.h | 2 +- include/crypto/internal/aead.h | 13 +++++++++++-- 2 files changed, 12 insertions(+), 3 deletions(-)
include/crypto/aead.h +1 -1
@@ -97,7 +97,7 @@ struct aead_request {struct scatterlist *src;struct scatterlist *dst; - void *__ctx[] CRYPTO_MINALIGN_ATTR; + void *__ctx[] CRYPTO_REQ_MINALIGN_ATTR; };/**
include/crypto/internal/aead.h +11 -2
@@ -62,7 +62,8 @@ static inline void *aead_instance_ctx(struct aead_instance *inst)static inline void *aead_request_ctx (struct aead_request *req) { -return req->__ctx; +return PTR_ALIGN (&req->__ctx, +crypto_tfm_alg_req_alignmask (req->base.tfm) +1 ); }static inline void aead_request_complete (struct aead_request *req,int err)
@@ -105,7 +106,15 @@ static inline struct crypto_aead *crypto_spawn_aead(static inline void crypto_aead_set_reqsize (struct crypto_aead *aead,unsigned int reqsize) { - aead->reqsize = reqsize; +unsigned int align =crypto_tfm_alg_req_alignmask (&aead->base) +1 ; + +/* +* The request structure itself is only aligned to CRYPTO_REQ_MINALIGN, +* so we need to add some headroom, allowing us to return a suitably +* aligned context buffer pointer. We also need to round up the size so +* we don't end up sharing a cacheline at the end of the buffer. +*/ + aead->reqsize =ALIGN (reqsize, align) + align -CRYPTO_REQ_MINALIGN ; } static inline void aead_init_queue(struct aead_queue *queue,
Signature · 1 line
2.30.2
PATCH 7/8 crypto: ahash - avoid DMA alignment for request structures unless needed
In reply to Ard BiesheuvelAHASH request structures are currently aligned to minimal DMA alignment for the arcitecture, which defaults to 128 bytes on arm64. This is excessive, and rarely needed, i.e., only when doing non-coherent inbound DMA on the contents of the request context buffer. So let's relax this requirement, and only use this alignment if the CRYPTO_ALG_NEED_DMA_ALIGNMENT flag is set by the implementation.
--- include/crypto/hash.h | 5 +++-- include/crypto/internal/hash.h | 10 +++++++++- 2 files changed, 12 insertions(+), 3 deletions(-)
include/crypto/hash.h +3 -2
@@ -56,7 +56,7 @@ struct ahash_request {/* This field may only be used by the ahash API code. */ void *priv; - void *__ctx[] CRYPTO_MINALIGN_ATTR; + void *__ctx[] CRYPTO_REQ_MINALIGN_ATTR; };/**
@@ -417,7 +417,8 @@ static inline unsigned int crypto_ahash_reqsize(struct crypto_ahash *tfm)static inline void *ahash_request_ctx (struct ahash_request *req) { -return req->__ctx; +return PTR_ALIGN (&req->__ctx, +crypto_tfm_alg_req_alignmask (req->base.tfm) +1 ); }/**
include/crypto/internal/hash.h +9 -1
@@ -143,7 +143,15 @@ static inline struct ahash_alg *__crypto_ahash_alg(struct crypto_alg *alg)static inline void crypto_ahash_set_reqsize (struct crypto_ahash *tfm,unsigned int reqsize) { - tfm->reqsize = reqsize; +unsigned int align =crypto_tfm_alg_req_alignmask (crypto_ahash_tfm (tfm)) +1 ; + +/* +* The request structure itself is only aligned to CRYPTO_REQ_MINALIGN, +* so we need to add some headroom, allowing us to return a suitably +* aligned context buffer pointer. We also need to round up the size so +* we don't end up sharing a cacheline at the end of the buffer. +*/ + tfm->reqsize =ALIGN (reqsize, align) + align -CRYPTO_REQ_MINALIGN ; } static inline struct crypto_instance *ahash_crypto_instance(
Signature · 1 line
2.30.2
kernel test robot
build botRe: [PATCH 7/8] crypto: ahash - avoid DMA alignment for request structures unless needed
I love your patch! Perhaps something to improve:
Hi Ard,
I love your patch! Perhaps something to improve:
[auto build test WARNING on herbert-cryptodev-2.6/master]
[also build test WARNING on herbert-crypto-2.6/master sunxi/sunxi/for-next v5.18-rc1 next-20220406] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Ard-Biesheuvel/crypto-avoid-DMA-padding-for-request-structures/20220407-010855 base: https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master config: m68k-allyesconfig (https://download.01.org/0day-ci/archive/20220407/[email protected]/config) compiler: m68k-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/072f0a01b9f98cd545d8179b1591b9f37be5cb2a
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Ard-Biesheuvel/crypto-avoid-DMA-padding-for-request-structures/20220407-010855 git checkout 072f0a01b9f98cd545d8179b1591b9f37be5cb2a
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=m68k SHELL=/bin/bash drivers/crypto/inside-secure/
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
In file included from include/vdso/const.h:5,
from include/linux/const.h:4,
from include/linux/list.h:9,
from include/linux/preempt.h:11,
from arch/m68k/include/asm/irqflags.h:6,
from include/linux/irqflags.h:16,
from arch/m68k/include/asm/atomic.h:6,
from include/linux/atomic.h:7,
from include/linux/crypto.h:15,
from include/crypto/aes.h:10,
from drivers/crypto/inside-secure/safexcel_hash.c:8:
In function 'ahash_request_ctx',
inlined from 'safexcel_ahash_exit_inv' at drivers/crypto/inside-secure/safexcel_hash.c:627:36:
>> include/crypto/hash.h:421:26: warning: '*(struct ahash_request *)(&__req_desc[0]).base.tfm' is used uninitialized [-Wuninitialized]
421 | crypto_tfm_alg_req_alignmask(req->base.tfm) + 1);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/uapi/linux/const.h:32:50: note: in definition of macro '__ALIGN_KERNEL_MASK'
32 | #define __ALIGN_KERNEL_MASK(x, mask) (((x) + (mask)) & ~(mask))
| ^~~~
include/linux/align.h:8:33: note: in expansion of macro '__ALIGN_KERNEL'
8 | #define ALIGN(x, a) __ALIGN_KERNEL((x), (a))
| ^~~~~~~~~~~~~~
include/linux/align.h:11:45: note: in expansion of macro 'ALIGN'
11 | #define PTR_ALIGN(p, a) ((typeof(p))ALIGN((unsigned long)(p), (a)))
| ^~~~~
include/crypto/hash.h:420:16: note: in expansion of macro 'PTR_ALIGN'
420 | return PTR_ALIGN(&req->__ctx,
| ^~~~~~~~~
In file included from drivers/crypto/inside-secure/safexcel_hash.c:21:
drivers/crypto/inside-secure/safexcel_hash.c: In function 'safexcel_ahash_exit_inv':
drivers/crypto/inside-secure/safexcel.h:69:14: note: '__req_desc' declared here
69 | char __##name##_desc[size] CRYPTO_MINALIGN_ATTR; \
| ^~
drivers/crypto/inside-secure/safexcel_hash.c:626:9: note: in expansion of macro 'EIP197_REQUEST_ON_STACK'
626 | EIP197_REQUEST_ON_STACK(req, ahash, EIP197_AHASH_REQ_SIZE);
| ^~~~~~~~~~~~~~~~~~~~~~~vim +421 include/crypto/hash.h
417
418 static inline void *ahash_request_ctx(struct ahash_request *req)
419 {
420 return PTR_ALIGN(&req->__ctx,
> 421 crypto_tfm_alg_req_alignmask(req->base.tfm) + 1);
422 }
423
Signature · 2 lines
0-DAY CI Kernel Test Service https://01.org/lkp
Herbert Xu
maintainerRe: [PATCH 2/8] crypto: safexcel - take request size after setting TFM
In reply to Ard BiesheuvelReview
Quoted 3 lines · Ard Biesheuvel
(quoted diff) +2
-0+#define EIP197_SKCIPHER_REQ_SIZE (ALIGN(sizeof(struct skcipher_request), \ + CRYPTO_MINALIGN) + \
The whole point of CRYPTO_MINALIGN is that it comes for free via kmalloc.
If you need alignment over and above that of kmalloc, you need to do it explicitly in the driver.
Cheers,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Re: [PATCH 2/8] crypto: safexcel - take request size after setting TFM
Quoted 9 lines · Herbert Xu
Review
Quoted 3 lines · Ard Biesheuvel
(quoted diff) +2
-0+#define EIP197_SKCIPHER_REQ_SIZE (ALIGN(sizeof(struct skcipher_request), \ + CRYPTO_MINALIGN) + \The whole point of
CRYPTO_MINALIGNis that it comes for free via kmalloc.
Yes, but kmalloc allocates the entire block at once, and so all the concatenated structures need to use the same alignment, which results in substantial padding overhead.
Quoted 3 lines
If you need alignment over and above that of kmalloc, you need to do it explicitly in the driver.
But none of the drivers currently do so, and rely on the API to produce ctx struct pointers that are suitably aligned for DMA.
Note that the main issue is not the alignment, but the rounding up of the size due to that. For instance, looking at crypto/adiantum.c:
struct adiantum_request_ctx {
... other fields ...
/* Sub-requests, must be last */
union {
struct shash_desc hash_desc;
struct skcipher_request streamcipher_req;
} u;
};So on arm64, every skcipher_request that gets allocated will be:
128 bytes for the outer skcipher_request + padding
128 bytes for the adiantum request context + padding 128 bytes for the inner skcipher_request + padding
n bytes for the inner context
Given that the skcipher_request only needs 72 bytes on 64-bit architectures, and Adiantum's reqctx size of ~50 bytes, this results in an overhead of ~200 bytes for every allocation, which is rather wasteful.
I think permitting DMA directly into these buffers was a mistake, but it is the situation we are in today. I am only trying to reduce the memory overhead for cases where it is not needed.
Re: [PATCH 2/8] crypto: safexcel - take request size after setting TFM
Quoted 12 lines · Ard Biesheuvel
Quoted 9 lines · Herbert Xu
Review
Quoted 3 lines · Ard Biesheuvel
(quoted diff) +2
-0+#define EIP197_SKCIPHER_REQ_SIZE (ALIGN(sizeof(struct skcipher_request), \ + CRYPTO_MINALIGN) + \The whole point of
CRYPTO_MINALIGNis that it comes for free via kmalloc.
BTW the definition above is only used for request allocations on the stack.