Skip to contentSkip to conversation

SHA1 transform: x86_64 AVX2 optimization - glue & build - resend with email correction

PATCH In review chandramouli narayanan 2 participants 3 messages Feb 27, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 2/2 SHA1 transform: x86_64 AVX2 optimization - glue & build - resend with email correction

This git patch adds the glue, build and configuration changes to include x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.

On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.

arch/x86/crypto/Makefile +3 -0
index 6ba54d6..61d6e28 100644
@@ -79,6 +79,9 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o
 aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o
 ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o
 sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o
+ifeq ($(avx2_supported),yes)
+sha1-ssse3-y += sha1_avx2_x86_64_asm.o
+endif
 crc32c-intel-y := crc32c-intel_glue.o
 crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o
 crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o
arch/x86/crypto/sha1_ssse3_glue.c +43 -7

91 lines not on this page · raw message

crypto/Kconfig +2 -2
index 7bcb70d..ce4012a 100644
@@ -491,14 +491,14 @@ config CRYPTO_SHA1
 	  SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2).
 
 config CRYPTO_SHA1_SSSE3
-	tristate "SHA1 digest algorithm (SSSE3/AVX)"
+	tristate "SHA1 digest algorithm (SSSE3/AVX/AVX2)"
 	depends on X86 && 64BIT
 	select CRYPTO_SHA1
 	select CRYPTO_HASH
 	help
 	  SHA-1 secure hash standard (FIPS 180-1/DFIPS 180-2) implemented
 	  using Supplemental SSE3 (SSSE3) instructions or Advanced Vector
-	  Extensions (AVX), when available.
+	  Extensions (AVX/AVX2), when available.
 
 config CRYPTO_SHA256_SSSE3
 	tristate "SHA256 digest algorithm (SSSE3/AVX/AVX2)"
#1 lore raw
Quoted 12 lines · chandramouli narayanan

This git patch adds the glue, build and configuration changes to include x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.

On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.

<..snip..>

Quoted 11 lines

static int __init sha1_ssse3_mod_init(void)
{

arch/x86/crypto/sha1_ssse3_glue.c +4 -1
+	char *algo_name;
 	/* test for SSSE3 first */
-	if (cpu_has_ssse3)
+	if (cpu_has_ssse3) {
 		sha1_transform_asm = sha1_transform_ssse3;
+		algo_name = "SSSE3";
+	}
 
 #ifdef CONFIG_AS_AVX

arch/x86/crypto/sha1_ssse3_glue.c static bool __init avx_usable(void) · line 247

Quoted 10 lines
arch/x86/crypto/sha1_ssse3_glue.c +7 -2
 	/* allow AVX to override SSSE3, it's a little faster */
-	if (avx_usable())
-		sha1_transform_asm = sha1_transform_avx;
+	if (avx_usable()) {
+		if (cpu_has_avx) {
+			sha1_transform_asm = sha1_transform_avx;
+			algo_name = "AVX";
+		}
+#ifdef CONFIG_AS_AVX2
+		if (cpu_has_avx2) {

Wouldn't you need to check also for BMI2 as __sha1_transform_avx2 uses 'rorx'?

For example, commit 16c0c4e1656c14ef9deac189a4240b5ca19c6919 added BMI2 check for SHA-256.

-Jussi

Quoted 7 lines
arch/x86/crypto/sha1_ssse3_glue.c +6 -0
+			/* allow AVX2 to override AVX, it's a little faster */
+			sha1_transform_asm = __sha1_transform_avx2;
+			algo_name = "AVX2";
+		}
+#endif
+	}
 #endif
#2 lore raw
Quoted 38 lines · Jussi Kivilinna
Quoted 12 lines · chandramouli narayanan

This git patch adds the glue, build and configuration changes to include x86_64 AVX2 optimization of SHA1 transform to crypto support. The patch has been tested with 3.14.0-rc1 kernel.

On a Haswell desktop, with turbo disabled and all cpus running at maximum frequency, tcrypt shows AVX2 performance improvement from 3% for 256 bytes update to 16% for 1024 bytes update over AVX implementation.

<..snip..>

Quoted 11 lines

static int __init sha1_ssse3_mod_init(void)
{

arch/x86/crypto/sha1_ssse3_glue.c +4 -1
+	char *algo_name;
 	/* test for SSSE3 first */
-	if (cpu_has_ssse3)
+	if (cpu_has_ssse3) {
 		sha1_transform_asm = sha1_transform_ssse3;
+		algo_name = "SSSE3";
+	}
 
 #ifdef CONFIG_AS_AVX

arch/x86/crypto/sha1_ssse3_glue.c static bool __init avx_usable(void) · line 247

Quoted 10 lines
arch/x86/crypto/sha1_ssse3_glue.c +7 -2
 	/* allow AVX to override SSSE3, it's a little faster */
-	if (avx_usable())
-		sha1_transform_asm = sha1_transform_avx;
+	if (avx_usable()) {
+		if (cpu_has_avx) {
+			sha1_transform_asm = sha1_transform_avx;
+			algo_name = "AVX";
+		}
+#ifdef CONFIG_AS_AVX2
+		if (cpu_has_avx2) {

Wouldn't you need to check also for BMI2 as __sha1_transform_avx2 uses 'rorx'?

Quoted 3 lines

For example, commit 16c0c4e1656c14ef9deac189a4240b5ca19c6919 added BMI2 check for SHA-256.

-Jussi

Good catch! I will add the check for AVX and BMI2.
- mouli

Quoted 8 lines
Quoted 8 lines
arch/x86/crypto/sha1_ssse3_glue.c +6 -0
+			/* allow AVX2 to override AVX, it's a little faster */
+			sha1_transform_asm = __sha1_transform_avx2;
+			algo_name = "AVX2";
+		}
+#endif
+	}
 #endif