crypto: SHA1 transform x86_64 AVX2
Subsystems (2)
PATCH v6 1/1 crypto: SHA1 transform x86_64 AVX2
This git patch adds x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.
On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.
sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The Makefile change therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix and Kconfig support and leaves the configuration build support as is.
--- arch/x86/crypto/Makefile | 3 + arch/x86/crypto/sha1_avx2_x86_64_asm.S | 712 +++++++++++++++++++++++++++++++++ arch/x86/crypto/sha1_ssse3_glue.c | 49 ++- crypto/Kconfig | 4 +- 4 files changed, 759 insertions(+), 9 deletions(-) create mode 100644 arch/x86/crypto/sha1_avx2_x86_64_asm.S
Changes noted from the initial version of this patch are based on the feedback from the community:
a) check for BMI2 in addition to AVX2 support since __sha1_transform_avx2() uses rorx
b) Since the module build has dependency on 64bit, it is redundant to check it in the code here.
c) coding style cleanup
d) simplification of the assembly code where macros are repetitively used.
arch/x86/crypto/Makefile +3 -0
@@ -79,6 +79,9 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o +ifeq ($(avx2_supported),yes) +sha1-ssse3-y += sha1_avx2_x86_64_asm.o +endif crc32c-intel-y := crc32c-intel_glue.o crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o
arch/x86/crypto/sha1_avx2_x86_64_asm.S new file +712 -0
712 lines not on this page · raw message
arch/x86/crypto/sha1_ssse3_glue.c +42 -7
90 lines not on this page · raw message
crypto/Kconfig +2 -2
@@ -491,14 +491,14 @@ config CRYPTO_SHA1 SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2). config CRYPTO_SHA1_SSSE3 - tristate "SHA1 digest algorithm (SSSE3/AVX)" + tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2)" depends on X86 && 64BIT select CRYPTO_SHA1 select CRYPTO_HASH help SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2) implemented using Supplemental SSE3 (SSSE3) instructions or Advanced Vector - Extensions (AVX), when available. + Extensions (AVX/AVX2), when available. config CRYPTO_SHA256_SSSE3 tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2)"
Signature · 1 line
1.8.2.1
H. Peter Anvin
reviewerAgain... Herbert, David... yours or mine?
-hpa
Herbert Xu
maintainerQuoted 1 line · H. Peter Anvin
Again... Herbert, David... yours or mine?
Yes I will be taking this through cryptodev.
Thanks,
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Quoted 5 lines · Herbert Xu
Quoted 1 line · H. Peter Anvin
Again... Herbert, David... yours or mine?
Yes I will be taking this through cryptodev.
Thanks!
-hpa
Quoted 23 lines · chandramouli narayanan
This git patch adds
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds
sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The
Makefilechange therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix andKconfigsupport and leaves the configuration build support as is.
Quoted 1 line
Signed-off-by Chandramouli Narayanan <[email protected]>
Very nice, good job. Thank you!
Best regards,
Marek Vasut
Quoted 24 lines · Marek Vasut
Quoted 23 lines · chandramouli narayanan
This git patch adds
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds
sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The
Makefilechange therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix andKconfigsupport and leaves the configuration build support as is.
Quoted 8 lines
Quoted 1 line
Signed-off-by Chandramouli Narayanan <[email protected]>Very nice, good job. Thank you!
Reviewed-by Marek Vasut <[email protected]>Best regards,
Marek Vasut
Thank you for reviewing the churn of patches!
Best regards,
- mouli
Herbert Xu
maintainerQuoted 24 lines · Marek Vasut
Quoted 23 lines · chandramouli narayanan
This git patch adds
x86_64AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.
This patch adds
sha1_avx2_transform(), the glue, build and configuration changes needed for AVX2 optimization of SHA1 transform to crypto support.sha1-ssse3 is one module which adds the necessary optimization support (SSSE3/AVX/AVX2) for the low-level SHA1 transform function. With better optimization support, transform function is overridden as the case may be. In the case of AVX2, due to performance reasons across datablock sizes, the AVX or AVX2 transform function is used at run-time as it suits best. The
Makefilechange therefore appends the necessary objects to the linkage. Due to this, the patch merely appends AVX2 transform to the existing build mix andKconfigsupport and leaves the configuration build support as is.
Quoted 5 lines
Quoted 1 line
Signed-off-by Chandramouli Narayanan <[email protected]>Very nice, good job. Thank you!
Reviewed-by Marek Vasut <[email protected]>
Patch applied. Thanks for reviewing.
Signature · 3 lines
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt