crypto api: add crc32 pclmulqdq implementation and wrappers for table implementation

PATCH In review Alexander Boyko 4 participants 9 messages Jan 10, 2013 – Jan 20, 2013 1 message not in the archive

Subsystems (2)

x86 Crypto API

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Alexander Boyko

author

2013-01-10 14:55 UTC #1 lore raw

PATCH v4 1/1 crypto api: add crc32 pclmulqdq implementation and wrappers for table implementation

From: Alexander Boyko <[email protected]>

This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.

For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation.

Signed-off-by Alexander Boyko <[email protected]>

---
Changes from V3 to V4:
- Fix email format.

 arch/x86/crypto/Makefile            |    2 
 arch/x86/crypto/crc32-pclmul_asm.S  |  247 ++++++++++++++++++++++++++++++++++++
 arch/x86/crypto/crc32-pclmul_glue.c |  201 +++++++++++++++++++++++++++++
 crypto/Kconfig                      |   21 +++
 crypto/Makefile                     |    1 
 crypto/crc32.c                      |  158 +++++++++++++++++++++++
 6 files changed, 630 insertions(+)

arch/x86/crypto/Makefile +2 -0

index e0ca7c9..63947a8 100644
@@ -27,6 +27,7 @@ obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o
 
 obj-$(CONFIG_CRYPTO_CRC32C_INTEL) += crc32c-intel.o
 obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o
+obj-$(CONFIG_CRYPTO_CRC32_PCLMUL) += crc32-pclmul.o
 
 aes-i586-y := aes-i586-asm_32.o aes_glue.o
 twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o

@@ -52,3 +53,4 @@ ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o
 sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o
 crc32c-intel-y := crc32c-intel_glue.o
 crc32c-intel-$(CONFIG_CRYPTO_CRC32C_X86_64) += crc32c-pcl-intel-asm_64.o
+crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o

arch/x86/crypto/crc32-pclmul_asm.S new file +247 -0

247 lines not on this page · raw message

arch/x86/crypto/crc32-pclmul_glue.c new file +201 -0

201 lines not on this page · raw message

crypto/Kconfig +21 -0

index 4641d95..e8b51e0 100644
@@ -355,6 +355,27 @@ config CRYPTO_CRC32C_SPARC64
 	  CRC32c CRC algorithm implemented using sparc64 crypto instructions,
 	  when available.
 
+config CRYPTO_CRC32
+	tristate "CRC32 CRC algorithm"
+	select CRYPTO_HASH
+	select CRC32
+	help
+	  CRC-32-IEEE 802.3 cyclic redundancy-check algorithm.
+	  Shash crypto api wrappers to crc32_le function.
+
+config CRYPTO_CRC32_PCLMUL
+	tristate "CRC32 PCLMULQDQ hardware acceleration"
+	depends on X86
+	select CRYPTO_HASH
+	select CRC32
+	help
+	  From Intel Westmere and AMD Bulldozer processor with SSE4.2
+	  and PCLMULQDQ supported, the processor will support
+	  CRC32 PCLMULQDQ implementation using hardware accelerated PCLMULQDQ
+	  instruction. This option will create 'crc32-plcmul' module,
+	  which will enable any routine to use the CRC-32-IEEE 802.3 checksum
+	  and gain better performance as compared with the table implementation.
+
 config CRYPTO_GHASH
 	tristate "GHASH digest algorithm"
 	select CRYPTO_GF128MUL

crypto/Makefile +1 -0

index d59dec7..be1a1be 100644
@@ -81,6 +81,7 @@ obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o
 obj-$(CONFIG_CRYPTO_ZLIB) += zlib.o
 obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o
 obj-$(CONFIG_CRYPTO_CRC32C) += crc32c.o
+obj-$(CONFIG_CRYPTO_CRC32) += crc32.o
 obj-$(CONFIG_CRYPTO_AUTHENC) += authenc.o authencesn.o
 obj-$(CONFIG_CRYPTO_LZO) += lzo.o
 obj-$(CONFIG_CRYPTO_842) += 842.o

crypto/crc32.c new file +158 -0

158 lines not on this page · raw message

Tim Chen

2013-01-10 17:55 UTC #2 lore raw

Quoted 9 lines · Alexander Boyko

From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation (

Alexander,

Wonder if you have a chance to test performance of our PCLMULQDQ implementation for crc32c that's in the current code (see crc32c-pcl-intel-asm_64.asm). The throughput will probably be comparable with your implementation.

Tim

Alexander Boyko

author

2013-01-10 19:26 UTC #3 lore raw

1/10/13 9:54 PM, Tim Chen пишет:

Quoted 11 lines · Alexander Boyko

Quoted 9 lines
From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation (

Quoted 11 lines

Alexander,
Wonder if you have a chance to test performance of our PCLMULQDQ implementation for crc32c that's in the current code (see crc32c-pcl-intel-asm_64.asm). The throughput will probably be comparable with your implementation.
Tim

I have no chance to test crc32c pclmul, but I tested previous crc32c implementation on crc32 instruction, the speed was about 2500 MB/s. So, I think, the newest version should be faster.

Tim Chen

2013-01-10 20:08 UTC #4 lore raw

Quoted 19 lines · Alexander Boyko

1/10/13 9:54 PM, Tim Chen пишет:
Quoted 18 lines · Alexander Boyko
Quoted 9 lines
From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation (
Alexander,
Wonder if you have a chance to test performance of our PCLMULQDQ implementation for crc32c that's in the current code (see crc32c-pcl-intel-asm_64.asm). The throughput will probably be comparable with your implementation.

Quoted 7 lines

Quoted 4 lines
Tim
I have no chance to test crc32c pclmul, but I tested previous crc32c implementation on crc32 instruction, the speed was about 2500 MB/s. So, I think, the newest version should be faster.

It will be troublesome to maintain two separate versions of PCLMUL crc32c code. So we should find out if there's performance benefit of your PCLMUL code over the one in the codebase. Testing should be straight forward by enabling the CRYPTO_CRC32C_INTEL option in kernel and inserting the crc32c-intel module.

You may also want to add check in your glue code for support of the PCLMUL feature before calling the pclmul version. You probably also don't want to use this feature if the data size is small, as kernel_fpu_begin and kernel_fpu_end takes significant time. In that case, using the crc32c hw instructions in a loop is faster (see crc32c-intel_glue.c).

Tim

Alexander Boyko

author

2013-01-10 20:26 UTC #5 lore raw

1/11/13 12:08 AM, Tim Chen пишет:

Quoted 32 lines · Alexander Boyko

Quoted 18 lines
1/10/13 9:54 PM, Tim Chen пишет:
Quoted 17 lines · Alexander Boyko
Quoted 9 lines
From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation (
Alexander,
Wonder if you have a chance to test performance of our PCLMULQDQ implementation for crc32c that's in the current code (see crc32c-pcl-intel-asm_64.asm). The throughput will probably be comparable with your implementation.
Quoted 7 lines
Quoted 4 lines
Tim
I have no chance to test crc32c pclmul, but I tested previous crc32c implementation on crc32 instruction, the speed was about 2500 MB/s. So, I think, the newest version should be faster.
It will be troublesome to maintain two separate versions of PCLMUL crc32c code. So we should find out if there's performance benefit of your PCLMUL code over the one in the codebase. Testing should be straight forward by enabling the CRYPTO_CRC32C_INTEL option in kernel and inserting the crc32c-intel module.

Quoted 9 lines

You may also want to add check in your glue code for support of the PCLMUL feature before calling the pclmul version. You probably also don't want to use this feature if the data size is small, as kernel_fpu_begin and kernel_fpu_end takes significant time. In that case, using the crc32c hw instructions in a loop is faster (see crc32c-intel_glue.c).
Tim

Sorry, may be I was miss understood, but I am trying to add CRC32 pclmul, not CRC32C. They use different polynomial.

Tim Chen

2013-01-10 21:57 UTC #6 lore raw

Quoted 3 lines · Alexander Boyko

Quoted 1 line
Sorry, may be I was miss understood, but I am trying to add CRC32 pclmul, not CRC32C. They use different polynomial.

Ah, I missed that. Thanks for clarifying.

Tim

Dilger, Andreas

2013-01-11 00:39 UTC #7 lore raw

In reply to Tim Chen

Quoted 29 lines · Tim Chen

Quoted 21 lines · Alexander Boyko
1/10/13 9:54 PM, Tim Chen пишет:
Quoted 5 lines · Alexander Boyko
Quoted 3 lines
From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper
to
Quoted 1 line
Quoted 1 line
gerneric crc32_le function. Second is crc32 pclmulqdq
implementation. It
Quoted 1 line
Quoted 1 line
use hardware provided PCLMULQDQ instruction to accelerate the CRC32
disposal.
Quoted 3 lines
Quoted 3 lines
This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and
2100MB/s
Quoted 6 lines
Quoted 1 line
for pclmulqdq implementation (
Alexander,
Wonder if you have a chance to test performance of our PCLMULQDQ implementation for crc32c that's in the current code (see crc32c-pcl-intel-asm_64.asm). The throughput will probably be
Quoted 6 lines
comparable
Quoted 1 line
with your implementation.
I have no chance to test crc32c pclmul, but I tested previous crc32c implementation on crc32 instruction, the speed was about 2500 MB/s. So, I think, the newest version should be faster.

Quoted 5 lines

It will be troublesome to maintain two separate versions of PCLMUL crc32c code. So we should find out if there's performance benefit of your PCLMUL code over the one in the codebase. Testing should be straight forward by enabling the CRYPTO_CRC32C_INTEL option in kernel and inserting the crc32c-intel module.

Maybe there is some confusion here? The submitted patch is for CRC32, while you are referring to CRC32C (note trailing "C")? Are they not different CRC functions, or can both CRCs be computed by the same code if there are different constants loaded?

Quoted 6 lines

You may also want to add check in your glue code for support of the PCLMUL feature before calling the pclmul version. You probably also don't want to use this feature if the data size is small, as kernel_fpu_begin and kernel_fpu_end takes significant time. In that case, using the crc32c hw instructions in a loop is faster (see crc32c-intel_glue.c).

Cheers, Andreas

Signature · 4 lines

Andreas Dilger

Lustre Software Architect
Intel High Performance Data Division

Tim Chen

2013-01-11 17:26 UTC #8 lore raw

Quoted 5 lines · Andreas

Maybe there is some confusion here? The submitted patch is for CRC32, while you are referring to CRC32C (note trailing "C")? Are they not different CRC functions, or can both CRCs be computed by the same code if there are different constants loaded?

Yeah, I missed the trailing C.

Tim

Herbert Xu

maintainer

2013-01-20 00:13 UTC #9 lore raw

In reply to Alexander Boyko

Quoted 11 lines · Alexander Boyko

From: Alexander Boyko <[email protected]>
This patch adds crc32 algorithms to shash crypto api. One is wrapper to gerneric crc32_le function. Second is crc32 pclmulqdq implementation. It use hardware provided PCLMULQDQ instruction to accelerate the CRC32 disposal. This instruction present from Intel Westmere and AMD Bulldozer CPUs.
For intel core i5 I got 450MB/s for table implementation and 2100MB/s for pclmulqdq implementation.
Signed-off-by Alexander Boyko <[email protected]>

Patch applied. Thanks!

Signature · 3 lines

Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt