Skip to contentSkip to conversation

arm64: crypto: increase AES interleave to 4x

PATCH New Ard Biesheuvel 2 participants 3 messages Feb 19, 2015 – Feb 20, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm64: crypto: increase AES interleave to 4x

This patch increases the interleave factor for parallel AES modes to 4x. This improves performance on Cortex-A57 by ~35%. This is due to the 3-cycle latency of AES instructions on the A57's relatively deep pipeline (compared to Cortex-A53 where the AES instruction latency is only 2 cycles).

At the same time, disable inline expansion of the core AES functions, as the performance benefit of this feature is negligible.

Measured on AMD Seattle (using tcrypt.ko mode=500 sec=1):

Baseline (2x interleave, inline expansion)
------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 95545 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 68496 operations in 1 seconds

This patch (4x interleave, no inline expansion)
-----------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 124735 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 92328 operations in 1 seconds

---
 arch/arm64/crypto/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/crypto/Makefile +1 -1
index 5720608c50b1..abb79b3cfcfe 100644
@@ -29,7 +29,7 @@ aes-ce-blk-y := aes-glue-ce.o aes-ce.o
 obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
 aes-neon-blk-y := aes-glue-neon.o aes-neon.o
 
-AFLAGS_aes-ce.o		:= -DINTERLEAVE=2 -DINTERLEAVE_INLINE
+AFLAGS_aes-ce.o		:= -DINTERLEAVE=4
 AFLAGS_aes-neon.o	:= -DINTERLEAVE=4
 
 CFLAGS_aes-glue-ce.o	:= -DUSE_V8_CRYPTO_EXTENSIONS
Signature · 1 line
1.8.3.2
#1 lore raw

PATCH 1/1 arm64: crypto: increase AES interleave to 4x

Quoted 17 lines · Ard Biesheuvel

This patch increases the interleave factor for parallel AES modes to 4x. This improves performance on Cortex-A57 by ~35%. This is due to the 3-cycle latency of AES instructions on the A57's relatively deep pipeline (compared to Cortex-A53 where the AES instruction latency is only 2 cycles).

At the same time, disable inline expansion of the core AES functions, as the performance benefit of this feature is negligible.

Measured on AMD Seattle (using tcrypt.ko mode=500 sec=1):

Baseline (2x interleave, inline expansion)
------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 95545 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 68496 operations in 1 seconds

Quoted 5 lines

This patch (4x interleave, no inline expansion)
-----------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 124735 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 92328 operations in 1 seconds

Fine by me. Shall I queue this via the arm64 tree?

Will

Quoted 22 lines
---
 arch/arm64/crypto/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/crypto/Makefile +1 -1
index 5720608c50b1..abb79b3cfcfe 100644
@@ -29,7 +29,7 @@ aes-ce-blk-y := aes-glue-ce.o aes-ce.o
 obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
 aes-neon-blk-y := aes-glue-neon.o aes-neon.o
 
-AFLAGS_aes-ce.o		:= -DINTERLEAVE=2 -DINTERLEAVE_INLINE
+AFLAGS_aes-ce.o		:= -DINTERLEAVE=4
 AFLAGS_aes-neon.o	:= -DINTERLEAVE=4
 
 CFLAGS_aes-glue-ce.o	:= -DUSE_V8_CRYPTO_EXTENSIONS
Signature · 1 line
1.8.3.2
#2 lore raw

PATCH 1/1 arm64: crypto: increase AES interleave to 4x

Quoted 18 lines · Will Deacon
Quoted 17 lines · Ard Biesheuvel

This patch increases the interleave factor for parallel AES modes to 4x. This improves performance on Cortex-A57 by ~35%. This is due to the 3-cycle latency of AES instructions on the A57's relatively deep pipeline (compared to Cortex-A53 where the AES instruction latency is only 2 cycles).

At the same time, disable inline expansion of the core AES functions, as the performance benefit of this feature is negligible.

Measured on AMD Seattle (using tcrypt.ko mode=500 sec=1):

Baseline (2x interleave, inline expansion)
------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 95545 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 68496 operations in 1 seconds

Quoted 8 lines
Quoted 5 lines

This patch (4x interleave, no inline expansion)
-----------------------------------------------
testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 124735 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 92328 operations in 1 seconds

Fine by me. Shall I queue this via the arm64 tree?

Yes, please.

Quoted 22 lines
Quoted 22 lines
---
 arch/arm64/crypto/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/crypto/Makefile +1 -2
index 5720608c50b1..abb79b3cfcfe 100644
@@ -29,7 +29,7 @@ aes-ce-blk-y := aes-glue-ce.o aes-ce.o
 obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
 aes-neon-blk-y := aes-glue-neon.o aes-neon.o
 
-AFLAGS_aes-ce.o              := -DINTERLEAVE=2 -DINTERLEAVE_INLINE
+AFLAGS_aes-ce.o              := -DINTERLEAVE=4
 AFLAGS_aes-neon.o    := -DINTERLEAVE=4
 
 CFLAGS_aes-glue-ce.o := -DUSE_V8_CRYPTO_EXTENSIONS
--

1.8.3.2