crypto: SHA1 transform x86_64 AVX2
Subsystems (2)
PATCH v5 1/1 crypto: SHA1 transform x86_64 AVX2
This git patch adds x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.
On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.
sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The Makefile change therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix and Kconfig support and leaves the configuration build support as is.
--- arch/x86/crypto/Makefile | 3 + arch/x86/crypto/sha1_avx2_x86_64_asm.S | 702 +++++++++++++++++++++++++++++++++ arch/x86/crypto/sha1_ssse3_glue.c | 50 ++- crypto/Kconfig | 4 +- 4 files changed, 750 insertions(+), 9 deletions(-) create mode 100644 arch/x86/crypto/sha1_avx2_x86_64_asm.S
arch/x86/crypto/Makefile +3 -0
@@ -79,6 +79,9 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o +ifeq ($(avx2_supported),yes) +sha1-ssse3-y += sha1_avx2_x86_64_asm.o +endif crc32c-intel-y := crc32c-intel_glue.o crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o
arch/x86/crypto/sha1_avx2_x86_64_asm.S new file +702 -0
702 lines not on this page · raw message
arch/x86/crypto/sha1_ssse3_glue.c +43 -7
91 lines not on this page · raw message
crypto/Kconfig +2 -2
@@ -491,14 +491,14 @@ config CRYPTO_SHA1 SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2). config CRYPTO_SHA1_SSSE3 - tristate "SHA1 digest algorithm (SSSE3/AVX)" + tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2)" depends on X86 && 64BIT select CRYPTO_SHA1 select CRYPTO_HASH help SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2) implemented using Supplemental SSE3 (SSSE3) instructions or Advanced Vector - Extensions (AVX), when available. + Extensions (AVX/AVX2), when available. config CRYPTO_SHA256_SSSE3 tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2)"
Signature · 1 line
1.8.2.1
Quoted 24 lines · chandramouli narayanan
This git patch adds
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds
sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The
Makefilechange therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix andKconfigsupport and leaves the configuration build support as is.Signed-off-by Chandramouli Narayanan <[email protected]>
Quoted 8 lines
--- arch/x86/crypto/Makefile | 3 + arch/x86/crypto/sha1_avx2_x86_64_asm.S | 702 +++++++++++++++++++++++++++++++++ arch/x86/crypto/sha1_ssse3_glue.c | 50 ++- crypto/Kconfig | 4 +- 4 files changed, 750 insertions(+), 9 deletions(-) create mode 100644 arch/x86/crypto/sha1_avx2_x86_64_asm.S
The changelog is missing completely now ;-)
[...]
Quoted 11 lines
arch/x86/crypto/sha1_avx2_x86_64_asm.S new file +11
-0+#include <linux/linkage.h> + +#define CTX %rdi /* arg1 */ +#define BUF %rsi /* arg2 */ +#define CNT %rdx /* arg3 */ + +#define REG_A %ecx +#define REG_B %esi +#define REG_C %edi +#define REG_D %eax +#define REG_E %edx
arch/x86/crypto/sha1_avx2_x86_64_asm.S line 93
Quoted 10 lines
arch/x86/crypto/sha1_avx2_x86_64_asm.S new file +10
-0+#define REG_TB %ebx +#define REG_TA %r12d +#define REG_RA %rcx +#define REG_RB %rsi +#define REG_RC %rdi +#define REG_RD %rax +#define REG_RE %rdx +#define REG_RTA %r12 +#define REG_RTB %rbx +#define REG_T1 %ebp
You're still mixing spaces and tabs here ...
[...]
arch/x86/crypto/sha1_avx2_x86_64_asm.S line 637
Quoted 11 lines
arch/x86/crypto/sha1_avx2_x86_64_asm.S new file +11
-0+ /* Align stack */ + mov %rsp, %rbx + and $(0x1000-1), %rbx + sub $(8+32), %rbx + sub %rbx, %rsp + push %rbx + sub $RESERVE_STACK, %rsp + + avx2_zeroupper + + lea K_XMM_AR(%rip), K_BASE
The indent here is really flying all around ;-)
Why don't you just check for "^ \+" and replace them with tabs ? That'd solve your indent problem rather quickly. Moreover, you can just use:
[TAB]<insn>[TAB]arg1, arg2...
This would solve the problem where your instruction arguments are not well indented.
Uh guys, Peter or Herbert, please stop me if I'm pushing too much.
[...]