SHA1 transform: x86_64 AVX2 optimization - glue & build - resend with email correction
Subsystems (2)
PATCH 2/2 SHA1 transform: x86_64 AVX2 optimization - glue & build - resend with email correction
This git patch adds the glue, build and configuration changes to include x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.
On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
arch/x86/crypto/Makefile +3 -0
@@ -79,6 +79,9 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o +ifeq ($(avx2_supported),yes) +sha1-ssse3-y += sha1_avx2_x86_64_asm.o +endif crc32c-intel-y := crc32c-intel_glue.o crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o
arch/x86/crypto/sha1_ssse3_glue.c +43 -7
91 lines not on this page · raw message
crypto/Kconfig +2 -2
@@ -491,14 +491,14 @@ config CRYPTO_SHA1 SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2). config CRYPTO_SHA1_SSSE3 - tristate "SHA1 digest algorithm (SSSE3/AVX)" + tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2)" depends on X86 && 64BIT select CRYPTO_SHA1 select CRYPTO_HASH help SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2) implemented using Supplemental SSE3 (SSSE3) instructions or Advanced Vector - Extensions (AVX), when available. + Extensions (AVX/AVX2), when available. config CRYPTO_SHA256_SSSE3 tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2)"
Quoted 12 lines · chandramouli narayanan
This git patch adds the glue, build and configuration changes to include
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
Signed-off-by Chandramouli Narayanan <[email protected]>
<..snip..>
Quoted 11 lines
static int
__initsha1_ssse3_mod_init(void)
{arch/x86/crypto/sha1_ssse3_glue.c +4
-1+char *algo_name;/* test for SSSE3 first */ -if (cpu_has_ssse3) +if (cpu_has_ssse3) { sha1_transform_asm = sha1_transform_ssse3; + algo_name ="SSSE3" ; + }#ifdef CONFIG_AS_AVX
arch/x86/crypto/sha1_ssse3_glue.c static bool __init avx_usable(void) · line 247
Quoted 10 lines
arch/x86/crypto/sha1_ssse3_glue.c +7
-2/* allow AVX to override SSSE3, it's a little faster */ -if (avx_usable ()) - sha1_transform_asm = sha1_transform_avx; +if (avx_usable ()) { +if (cpu_has_avx) { + sha1_transform_asm = sha1_transform_avx; + algo_name ="AVX" ; + } +#ifdef CONFIG_AS_AVX2 +if (cpu_has_avx2) {
Wouldn't you need to check also for BMI2 as __sha1_transform_avx2 uses 'rorx'?
For example, commit 16c0c4e1656c14ef9deac189a4240b5ca19c6919 added BMI2 check for SHA-256.
-Jussi
Quoted 7 lines
arch/x86/crypto/sha1_ssse3_glue.c +6
-0+/* allow AVX2 to override AVX, it's a little faster */ + sha1_transform_asm = __sha1_transform_avx2; + algo_name ="AVX2" ; + } +#endif + }#endif
Quoted 38 lines · Jussi Kivilinna
Quoted 12 lines · chandramouli narayanan
This git patch adds the glue, build and configuration changes to include
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
Signed-off-by Chandramouli Narayanan <[email protected]><..snip..>
Quoted 11 lines
static int
__initsha1_ssse3_mod_init(void)
{arch/x86/crypto/sha1_ssse3_glue.c +4
-1+char *algo_name;/* test for SSSE3 first */ -if (cpu_has_ssse3) +if (cpu_has_ssse3) { sha1_transform_asm = sha1_transform_ssse3; + algo_name ="SSSE3" ; + }#ifdef CONFIG_AS_AVX arch/x86/crypto/sha1_ssse3_glue.c static bool __init avx_usable(void) · line 247
Quoted 10 lines
arch/x86/crypto/sha1_ssse3_glue.c +7
-2/* allow AVX to override SSSE3, it's a little faster */ -if (avx_usable ()) - sha1_transform_asm = sha1_transform_avx; +if (avx_usable ()) { +if (cpu_has_avx) { + sha1_transform_asm = sha1_transform_avx; + algo_name ="AVX" ; + } +#ifdef CONFIG_AS_AVX2 +if (cpu_has_avx2) {Wouldn't you need to check also for BMI2 as
__sha1_transform_avx2uses 'rorx'?
Quoted 3 lines
For example, commit
16c0c4e1656c14ef9deac189a4240b5ca19c6919added BMI2 check for SHA-256.-Jussi
Good catch! I will add the check for AVX and BMI2.
- mouli
Quoted 8 lines
Quoted 8 lines
arch/x86/crypto/sha1_ssse3_glue.c +6
-0+/* allow AVX2 to override AVX, it's a little faster */ + sha1_transform_asm = __sha1_transform_avx2; + algo_name ="AVX2" ; + } +#endif + }#endif