Skip to content

Clean up and improve ARM/arm64 CRC-T10DIF code

Ard Biesheuvel linux-arm-kernel linux-crypto 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest6 patches 2 repliesIn review
  2. v16 patches 6 repliesSuperseded

Changes in v2 from the cover letter

- fix bug introduced in refactoring
- add asm comments to explain the fallback algorithm
- type 'u8 *out' parameter as 'u8 out[16]'
- avoid asm code for 16 byte inputs (a higher threshold might be more appropriate but 16 is nonsensical given that the folding routine returns a 16 byte output)

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6crypto: arm64/crct10dif - Remove obsolete chunking logicarch/arm64/crypto/crct10dif-ce-glue.c+6 -240 replies
2/6crypto: arm64/crct10dif - Use faster 16x64 bit polynomial multiplyarch/arm64/crypto/crct10dif-ce-core.S+104 -170 replies
3/6crypto: arm64/crct10dif - Remove remaining 64x64 PMULL fallback codearch/arm64/crypto/crct10dif-ce-core.S arch/arm64/crypto/crct10dif-ce-glue.c+68 -1940 replies
4/6crypto: arm/crct10dif - Use existing mov_l macro instead of __adrlarch/arm/crypto/crct10dif-ce-core.S+3 -80 replies
5/6crypto: arm/crct10dif - Macroify PMULL asm codearch/arm/crypto/crct10dif-ce-core.S arch/arm/crypto/crct10dif-ce-glue.c+83 -810 replies
6/6crypto: arm/crct10dif - Implement plain NEON variantarch/arm/crypto/crct10dif-ce-core.S arch/arm/crypto/crct10dif-ce-glue.c+134 -90 replies

Reviews trailers per patch

3 of 6 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchEric Biggers
SeriesR-b
1/6R-b
2/6
3/6
4/6R-b
5/6R-b
6/6

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/6] Clean up and improve ARM/arm64 CRC-T10DIF code

From: Ard Biesheuvel <[email protected]>

I realized that the generic sequence implementing 64x64 polynomial multiply using 8x8 PMULL instructions, which is used in the CRC-T10DIF code to implement a fallback version for cores that lack the 64x64 PMULL instruction, is not very efficient.

The folding coefficients that are used when processing the bulk of the data are only 16 bits wide, and so 3/4 of the partial results of all those 8x8->16 bit multiplications do not contribute anything to the end result.

This means we can use a much faster implementation, producing a speedup of 3.3x on Cortex-A72 without Crypto Extensions (Raspberry Pi 4).

The same logic can be ported to 32-bit ARM too, where it produces a speedup of 6.6x compared with the generic C implementation on the same platform.

Changes since v1:
- fix bug introduced in refactoring
- add asm comments to explain the fallback algorithm
- type 'u8 *out' parameter as 'u8 out[16]'
- avoid asm code for 16 byte inputs (a higher threshold might be more appropriate but 16 is nonsensical given that the folding routine returns a 16 byte output)

Ard Biesheuvel (6):
crypto: arm64/crct10dif - Remove obsolete chunking logic
crypto: arm64/crct10dif - Use faster 16x64 bit polynomial multiply
crypto: arm64/crct10dif - Remove remaining 64x64 PMULL fallback code
crypto: arm/crct10dif - Use existing mov_l macro instead of __adrl
crypto: arm/crct10dif - Macroify PMULL asm code
crypto: arm/crct10dif - Implement plain NEON variant

 arch/arm/crypto/crct10dif-ce-core.S   | 249 ++++++++++-----
 arch/arm/crypto/crct10dif-ce-glue.c   |  55 +++-
 arch/arm64/crypto/crct10dif-ce-core.S | 335 +++++++++-----------
 arch/arm64/crypto/crct10dif-ce-glue.c |  48 ++-
 4 files changed, 376 insertions(+), 311 deletions(-)
Signature · 1 line
2.47.0.199.ga7371fff76-goog