Skip to content

crypto: crypto API library interfaces for WireGuard

Ard Biesheuvel linux-crypto 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v5 Latest34 patches 8 repliesIn review

Changes in v5 from the cover letter

- Address most review feedback from Eric, with the exception of the remark about libraries being selectable by the user - this is something we need to revisit in the context of moving to weak references or static calls to make accelerated versions of libraries loadable at any time. (Currently, loading an accelerated version at runtime will not supersede calls to the generic routines in the kernel proper, which is counterintuitive, and this is currently being addressed by making the generic library versions only selectable as modules if the accelerated ones are selected as modules as well)
- Align the generic blake2s Kconfig symbols, filenames etc with the recently added blake2b driver.
- Rewrote the blake2s selftest for better coverage of key length and input length combinations, and added a HMAC selftest as well.
- Rename blake2s_hmac() to blake2s256_hmac(), and drop the digest length argument, which was not implemented correctly, and never deviates from the full length in practice anyway.
- Update to more recent version of the blake2s x86 Zinc code

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/34crypto: tidy up lib/crypto Kconfig and Makefilecrypto/Kconfig lib/crypto/Kconfig lib/crypto/Makefile+24 -200 replies
2/34crypto: chacha - move existing library code into lib/cryptoarch/arm/crypto/chacha-neon-glue.c arch/arm64/crypto/chacha-neon-glue.c arch/x86/crypto/chacha_glue.c +9 more+199 -860 replies
3/34crypto: x86/chacha - depend on generic chacha library instead of crypto driverarch/x86/crypto/chacha_glue.c crypto/Kconfig+36 -560 replies
4/34crypto: x86/chacha - expose SIMD ChaCha routine as library functionarch/x86/crypto/chacha_glue.c crypto/Kconfig include/crypto/chacha.h+73 -250 replies
5/34crypto: arm64/chacha - depend on generic chacha library instead of crypto driverarch/arm64/crypto/Kconfig arch/arm64/crypto/chacha-neon-glue.c+23 -190 replies
6/34crypto: arm64/chacha - expose arm64 ChaCha routine as library functionarch/arm64/crypto/Kconfig arch/arm64/crypto/chacha-neon-glue.c+43 -110 replies
7/34crypto: arm/chacha - import Eric Biggers's scalar accelerated ChaCha codearch/arm/crypto/chacha-scalar-core.S+461 -00 replies
8/34crypto: arm/chacha - remove dependency on generic ChaCha driverarch/arm/crypto/Kconfig arch/arm/crypto/Makefile arch/arm/crypto/chacha-glue.c +3 more+340 -2400 replies
9/34crypto: arm/chacha - expose ARM ChaCha routine as library functionarch/arm/crypto/Kconfig arch/arm/crypto/chacha-glue.c+41 -10 replies
10/34crypto: mips/chacha - import 32r2 ChaCha code from Zincarch/mips/crypto/chacha-core.S+424 -00 replies
11/34crypto: mips/chacha - wire up accelerated 32r2 code from Zincarch/mips/Makefile arch/mips/crypto/Makefile arch/mips/crypto/chacha-core.S +2 more+277 -440 replies
12/34crypto: chacha - unexport chacha_generic routinescrypto/chacha_generic.c include/crypto/internal/chacha.h+8 -280 replies
13/34crypto: poly1305 - move core routines into a separate libraryarch/x86/crypto/poly1305_glue.c crypto/Kconfig crypto/adiantum.c +7 more+248 -2160 replies
14/34crypto: x86/poly1305 - unify Poly1305 state struct with generic codearch/x86/crypto/poly1305_glue.c crypto/poly1305_generic.c include/crypto/internal/poly1305.h +1 more+43 -730 replies
15/34crypto: poly1305 - expose init/update/final library interfacecrypto/poly1305_generic.c include/crypto/poly1305.h lib/crypto/Kconfig +1 more+138 -220 replies
16/34crypto: x86/poly1305 - depend on generic library not generic shasharch/x86/crypto/poly1305_glue.c crypto/Kconfig crypto/poly1305_generic.c +1 more+60 -280 replies
17/34crypto: x86/poly1305 - expose existing driver as poly1305 libraryarch/x86/crypto/poly1305_glue.c crypto/Kconfig lib/crypto/Kconfig+43 -160 replies
18/34crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementationarch/arm64/crypto/Kconfig arch/arm64/crypto/Makefile arch/arm64/crypto/poly1305-armv8.pl +3 more+2001 -10 replies
19/34crypto: arm/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON implementationarch/arm/crypto/Kconfig arch/arm/crypto/Makefile arch/arm/crypto/poly1305-armv4.pl +3 more+2687 -20 replies
20/34crypto: mips/poly1305 - incorporate OpenSSL/CRYPTOGAMS optimized implementationarch/mips/crypto/Makefile arch/mips/crypto/poly1305-glue.c arch/mips/crypto/poly1305-mips.pl +2 more+1496 -00 replies
21/34int128: move __uint128_t compiler test to Kconfigarch/arm64/Kconfig arch/riscv/Kconfig arch/x86/Kconfig +4 more+10 -60 replies
22/34crypto: BLAKE2s - generic C library implementation and selftestinclude/crypto/blake2s.h include/crypto/internal/blake2s.h lib/crypto/Kconfig +4 more+1019 -00 replies
23/34crypto: testmgr - add test cases for Blake2scrypto/testmgr.c crypto/testmgr.h+280 -390 replies
24/34crypto: blake2s - implement generic shash drivercrypto/Kconfig crypto/Makefile crypto/blake2s_generic.c +1 more+195 -00 replies
25/34crypto: BLAKE2s - x86_64 SIMD implementationarch/x86/crypto/Makefile arch/x86/crypto/blake2s-core.S arch/x86/crypto/blake2s-glue.c +1 more+499 -00 replies
26/34crypto: Curve25519 - generic C library implementationsinclude/crypto/curve25519.h lib/crypto/Kconfig lib/crypto/Makefile +3 more+1778 -00 replies
27/34crypto: curve25519 - add kpp selftestcrypto/testmgr.c crypto/testmgr.h+1231 -00 replies
28/34crypto: curve25519 - implement generic KPP drivercrypto/Kconfig crypto/Makefile crypto/curve25519-generic.c+96 -00 replies
29/34crypto: lib/curve25519 - work around Clang stack spilling issuelib/crypto/curve25519-fiat32.c+5 -50 replies
30/34crypto: Curve25519 - x86_64 library and KPP implementationsarch/x86/crypto/Makefile arch/x86/crypto/curve25519-x86_64.c crypto/Kconfig+2482 -00 replies
31/34crypto: arm - import Bernstein and Schwabe's Curve25519 ARM implementationarch/arm/crypto/curve25519-core.S+2105 -00 replies
32/34crypto: arm/Curve25519 - wire up NEON implementationarch/arm/crypto/Kconfig arch/arm/crypto/Makefile arch/arm/crypto/curve25519-core.S +1 more+287 -1950 replies
33/34crypto: chacha20poly1305 - import construction and selftest from Zincinclude/crypto/chacha20poly1305.h lib/crypto/Kconfig lib/crypto/Makefile +2 more+7615 -00 replies
34/34crypto: lib/chacha20poly1305 - reimplement crypt_from_sg() routineinclude/crypto/chacha20poly1305.h lib/crypto/chacha20poly1305-selftest.c lib/crypto/chacha20poly1305.c+206 -00 replies

Reviews trailers per patch

None of the 34 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchJason A. Donenfeld
SeriesR-b
1/34
2/34
3/34
4/34
5/34
6/34
7/34
8/34
9/34
10/34
11/34
12/34
13/34
14/34
15/34
16/34
17/34
18/34
19/34
20/34
21/34
22/34
23/34
24/34
25/34
26/34
27/34
28/34
29/34
30/34
31/34
32/34
33/34
34/34

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v5 thread

Cover letter [PATCH v5 00/34] crypto: crypto API library interfaces for WireGuard

This series implements the crypto library abstractions that are needed to incorporate WireGuard into the mainline kernel.

Changes since v4:
- Address most review feedback from Eric, with the exception of the remark about libraries being selectable by the user - this is something we need to revisit in the context of moving to weak references or static calls to make accelerated versions of libraries loadable at any time. (Currently, loading an accelerated version at runtime will not supersede calls to the generic routines in the kernel proper, which is counterintuitive, and this is currently being addressed by making the generic library versions only selectable as modules if the accelerated ones are selected as modules as well)
- Align the generic blake2s Kconfig symbols, filenames etc with the recently added blake2b driver.
- Rewrote the blake2s selftest for better coverage of key length and input length combinations, and added a HMAC selftest as well.
- Rename blake2s_hmac() to blake2s256_hmac(), and drop the digest length argument, which was not implemented correctly, and never deviates from the full length in practice anyway.
- Update to more recent version of the blake2s x86 Zinc code

Changes since v3:
- Unify the way the generic vs arch libraries are organized between ChaCha20 and Poly1305 on the one hand and Curve25519 and Blake2s on the other. All are now made up of a generic library, a generic crypto API driver (skcipher for [X]ChaCha, shash for Poly1305 and Blake2s and kpp for Curve25519) and optional per-arch versions providing both the library and the crypto API interfaces while potentially relying on the generic *library* only as a fallback (and not on the generic crypto API driver). Implementations of the libary interface that don't require the fallback don't pull in the generic code at all, but the generic crypto API drivers are tied to the generic implementations directly (this is necessary since we fuzz test the accelerated implementations against the generic implementations)
- Provide testmgr test vectors for the Curve25519 and Blake2s crypto API drivers that were added in this revision. This also required some changes to the KPP test routines so we can test for failures as well.
- Update to the latest version of Andy Polyakov's Poly1305 implementation for MIPS that incorporates Rene's improvements for 32r2
- Remove logic in the x86 and ARM implementations of ChaCha and Poly1305 to prefer the non-SIMD path for short inputs. This is no longer necessary, and even undesirable since it forced ChaCha20Poly1305's ChaCha pass generating the Poly1305 nonce to always take the slower scalar path.

Changes since v2:
- Reduce the cc: audience a bit, since I assumed that not everyone is interested in discussing the details of this.
- Incorporate scalar ARM code for ChaCha, and the 64-bit MIPS code for Poly1305. NOTE: the Cryptogams MIPS code now supports 32-bit MIPS as well, and not just 32r2, so I omitted Rene's Poly1305 implementation for now, and used Andy's code for everything.
- Incorporate NEON opt-out for Cortex-A5/A7. Note that the code is still exposed via the crypto API, but with a low prioririty, so it is still available and still gets test coverage, but is not used by default.
- Use static keys (*not* static calls) in the SIMD and bmi2/adx drivers to keep track of which implementation is being used, to avoid the memory load on each call.
- Defer using weak references or static calls until the dust around this has settled. Instead, rely on Kconfig constraints and symbol dependencies to ensure that the arch code is always used when it is loaded. This means you can only opt out of using the arch code if you disable it in Kconfig but this is something I can live with for now.
- Refactor the Curve25519 glue code slightly so that the call sites branch to the arch or generic code directly.
- Split up the Poly1305 refactoring patches so they can be reviewed more easily.

Changes since RFC/v1:
- dropped the WireGuard patch itself, and the followup patches - since the purpose was to illustrate the extent of the required changes, there is no reason to keep including them.
- import the MIPS 32r2 versions of ChaCha and Poly1305, but expose both the crypto API and library interfaces so that not only WireGuard but also IPsec and Adiantum can benefit immediately. (The latter required adding support for the reduced round version of ChaCha to the MIPS asm code)
- fix up various minor kconfig/build issues found in randconfig testing (thanks Arnd!)

Patches can be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=wireguard-crypto-library-api-v5

Cc Herbert Xu <[email protected]>
Cc David Miller <[email protected]>
Cc Jason A. Donenfeld <[email protected]>
Cc Samuel Neves <[email protected]>
Cc Arnd Bergmann <[email protected]>
Cc Eric Biggers <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Martin Willi <[email protected]>
Cc Rene van Dorst <[email protected]>
Cc David Sterba <[email protected]>

Ard Biesheuvel (27):
crypto: tidy up lib/crypto Kconfig and Makefile
crypto: chacha - move existing library code into lib/crypto
crypto: x86/chacha - depend on generic chacha library instead of
crypto driver
crypto: x86/chacha - expose SIMD ChaCha routine as library function
crypto: arm64/chacha - depend on generic chacha library instead of
crypto driver
crypto: arm64/chacha - expose arm64 ChaCha routine as library function
crypto: arm/chacha - import Eric Biggers's scalar accelerated ChaCha
code
crypto: arm/chacha - remove dependency on generic ChaCha driver
crypto: arm/chacha - expose ARM ChaCha routine as library function
crypto: mips/chacha - wire up accelerated 32r2 code from Zinc
crypto: chacha - unexport chacha_generic routines
crypto: poly1305 - move core routines into a separate library
crypto: x86/poly1305 - unify Poly1305 state struct with generic code
crypto: poly1305 - expose init/update/final library interface
crypto: x86/poly1305 - depend on generic library not generic shash
crypto: x86/poly1305 - expose existing driver as poly1305 library
crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON
implementation
crypto: arm/poly1305 - incorporate OpenSSL/CRYPTOGAMS NEON
implementation
crypto: mips/poly1305 - incorporate OpenSSL/CRYPTOGAMS optimized
implementation
int128: move __uint128_t compiler test to Kconfig
crypto: testmgr - add test cases for Blake2s
crypto: blake2s - implement generic shash driver
crypto: curve25519 - add kpp selftest
crypto: curve25519 - implement generic KPP driver
crypto: lib/curve25519 - work around Clang stack spilling issue
crypto: chacha20poly1305 - import construction and selftest from Zinc
crypto: lib/chacha20poly1305 - reimplement crypt_from_sg() routine

Jason A. Donenfeld (7):
crypto: mips/chacha - import 32r2 ChaCha code from Zinc
crypto: BLAKE2s - generic C library implementation and selftest
crypto: BLAKE2s - x86_64 SIMD implementation
crypto: Curve25519 - generic C library implementations
crypto: Curve25519 - x86_64 library and KPP implementations
crypto: arm - import Bernstein and Schwabe's Curve25519 ARM
implementation
crypto: arm/Curve25519 - wire up NEON implementation

 arch/arm/crypto/Kconfig                   |   16 +-
 arch/arm/crypto/Makefile                  |   17 +-
 arch/arm/crypto/chacha-glue.c             |  343 +
 arch/arm/crypto/chacha-neon-glue.c        |  202 -
 arch/arm/crypto/chacha-scalar-core.S      |  460 ++
 arch/arm/crypto/curve25519-core.S         | 2062 ++++++
 arch/arm/crypto/curve25519-glue.c         |  127 +
 arch/arm/crypto/poly1305-armv4.pl         | 1236 ++++
 arch/arm/crypto/poly1305-core.S_shipped   | 1158 +++
 arch/arm/crypto/poly1305-glue.c           |  276 +
 arch/arm64/Kconfig                        |    2 +-
 arch/arm64/crypto/Kconfig                 |    9 +-
 arch/arm64/crypto/Makefile                |   10 +-
 arch/arm64/crypto/chacha-neon-glue.c      |   81 +-
 arch/arm64/crypto/poly1305-armv8.pl       |  913 +++
 arch/arm64/crypto/poly1305-core.S_shipped |  835 +++
 arch/arm64/crypto/poly1305-glue.c         |  237 +
 arch/mips/Makefile                        |    2 +-
 arch/mips/crypto/Makefile                 |   18 +
 arch/mips/crypto/chacha-core.S            |  497 ++
 arch/mips/crypto/chacha-glue.c            |  150 +
 arch/mips/crypto/poly1305-glue.c          |  203 +
 arch/mips/crypto/poly1305-mips.pl         | 1273 ++++
 arch/riscv/Kconfig                        |    2 +-
 arch/x86/Kconfig                          |    2 +-
 arch/x86/crypto/Makefile                  |    3 +
 arch/x86/crypto/blake2s-core.S            |  258 +
 arch/x86/crypto/blake2s-glue.c            |  233 +
 arch/x86/crypto/chacha_glue.c             |  181 +-
 arch/x86/crypto/curve25519-x86_64.c       | 2475 +++++++
 arch/x86/crypto/poly1305_glue.c           |  199 +-
 crypto/Kconfig                            |   71 +-
 crypto/Makefile                           |    2 +
 crypto/adiantum.c                         |    5 +-
 crypto/blake2s_generic.c                  |  171 +
 crypto/chacha_generic.c                   |   84 +-
 crypto/curve25519-generic.c               |   90 +
 crypto/ecc.c                              |    2 +-
 crypto/nhpoly1305.c                       |    3 +-
 crypto/poly1305_generic.c                 |  228 +-
 crypto/testmgr.c                          |   30 +
 crypto/testmgr.h                          | 1520 +++-
 include/crypto/blake2s.h                  |  106 +
 include/crypto/chacha.h                   |   83 +-
 include/crypto/chacha20poly1305.h         |   48 +
 include/crypto/curve25519.h               |   71 +
 include/crypto/internal/blake2s.h         |   24 +
 include/crypto/internal/chacha.h          |   43 +
 include/crypto/internal/poly1305.h        |   58 +
 include/crypto/poly1305.h                 |   69 +-
 init/Kconfig                              |    4 +
 lib/Makefile                              |    3 +-
 lib/crypto/Kconfig                        |  130 +
 lib/crypto/Makefile                       |   42 +-
 lib/crypto/blake2s-generic.c              |  111 +
 lib/crypto/blake2s-selftest.c             |  622 ++
 lib/crypto/blake2s.c                      |  126 +
 lib/{ => crypto}/chacha.c                 |   20 +-
 lib/crypto/chacha20poly1305-selftest.c    | 7393 ++++++++++++++++++++
 lib/crypto/chacha20poly1305.c             |  369 +
 lib/crypto/curve25519-fiat32.c            |  864 +++
 lib/crypto/curve25519-hacl64.c            |  788 +++
 lib/crypto/curve25519.c                   |   25 +
 lib/crypto/libchacha.c                    |   35 +
 lib/crypto/poly1305.c                     |  232 +
 lib/ubsan.c                               |    2 +-
 lib/ubsan.h                               |    2 +-
 67 files changed, 26148 insertions(+), 808 deletions(-)
 create mode 100644 arch/arm/crypto/chacha-glue.c
 delete mode 100644 arch/arm/crypto/chacha-neon-glue.c
 create mode 100644 arch/arm/crypto/chacha-scalar-core.S
 create mode 100644 arch/arm/crypto/curve25519-core.S
 create mode 100644 arch/arm/crypto/curve25519-glue.c
 create mode 100644 arch/arm/crypto/poly1305-armv4.pl
 create mode 100644 arch/arm/crypto/poly1305-core.S_shipped
 create mode 100644 arch/arm/crypto/poly1305-glue.c
 create mode 100644 arch/arm64/crypto/poly1305-armv8.pl
 create mode 100644 arch/arm64/crypto/poly1305-core.S_shipped
 create mode 100644 arch/arm64/crypto/poly1305-glue.c
 create mode 100644 arch/mips/crypto/chacha-core.S
 create mode 100644 arch/mips/crypto/chacha-glue.c
 create mode 100644 arch/mips/crypto/poly1305-glue.c
 create mode 100644 arch/mips/crypto/poly1305-mips.pl
 create mode 100644 arch/x86/crypto/blake2s-core.S
 create mode 100644 arch/x86/crypto/blake2s-glue.c
 create mode 100644 arch/x86/crypto/curve25519-x86_64.c
 create mode 100644 crypto/blake2s_generic.c
 create mode 100644 crypto/curve25519-generic.c
 create mode 100644 include/crypto/blake2s.h
 create mode 100644 include/crypto/chacha20poly1305.h
 create mode 100644 include/crypto/curve25519.h
 create mode 100644 include/crypto/internal/blake2s.h
 create mode 100644 include/crypto/internal/chacha.h
 create mode 100644 include/crypto/internal/poly1305.h
 create mode 100644 lib/crypto/Kconfig
 create mode 100644 lib/crypto/blake2s-generic.c
 create mode 100644 lib/crypto/blake2s-selftest.c
 create mode 100644 lib/crypto/blake2s.c
 rename lib/{ => crypto}/chacha.c (88%)
 create mode 100644 lib/crypto/chacha20poly1305-selftest.c
 create mode 100644 lib/crypto/chacha20poly1305.c
 create mode 100644 lib/crypto/curve25519-fiat32.c
 create mode 100644 lib/crypto/curve25519-hacl64.c
 create mode 100644 lib/crypto/curve25519.c
 create mode 100644 lib/crypto/libchacha.c
 create mode 100644 lib/crypto/poly1305.c
Signature · 1 line
2.20.1