crypto: HCTR2 support
Subsystems (6)
v7 cover crypto: HCTR2 support
HCTR2 is a length-preserving encryption mode that is efficient on processors with instructions to accelerate AES and carryless multiplication, e.g. x86 processors with AES-NI and CLMUL, and ARM processors with the ARMv8 Crypto Extensions.
HCTR2 is specified in https://ia.cr/2021/1441 "Length-preserving encryption with HCTR2" which shows that if AES is secure and HCTR2 is instantiated with AES, then HCTR2 is secure. Reference code and test vectors are at
https://github.com/google/hctr2.
As a length-preserving encryption mode, HCTR2 is suitable for applications such as storage encryption where ciphertext expansion is not possible, and thus authenticated encryption cannot be used. Currently, such applications usually use XTS, or in some cases Adiantum. XTS has the disadvantage that it is a narrow-block mode: a bitflip will only change 16 bytes in the resulting ciphertext or plaintext. This reveals more information to an attacker than necessary.
HCTR2 is a wide-block mode, so it provides a stronger security property: a bitflip will change the entire message. HCTR2 is somewhat similar to Adiantum, which is also a wide-block mode. However, HCTR2 is designed to take advantage of existing crypto instructions, while Adiantum targets devices without such hardware support. Adiantum is also designed with longer messages in mind, while HCTR2 is designed to be efficient even on short messages.
The first intended use of this mode in the kernel is for the encryption of filenames, where for efficiency reasons encryption must be fully deterministic (only one ciphertext for each plaintext) and the existing CBC solution leaks more information than necessary for filenames with common prefixes.
HCTR2 uses two passes of an ε-almost-∆-universal hash function called POLYVAL and one pass of a block cipher mode called XCTR. POLYVAL is a polynomial hash designed for efficiency on modern processors and was originally specified for use in AES-GCM-SIV (RFC 8452). XCTR mode is a variant of CTR mode that is more efficient on little-endian machines.
This patchset adds HCTR2 to Linux's crypto API, including generic implementations of XCTR and POLYVAL, hardware accelerated implementations of XCTR and POLYVAL for both x86-64 and ARM64, a templated implementation of HCTR2, and an fscrypt policy for using HCTR2 for filename encryption.
Changes in v7:
* Added/modified some comments in ARM64 XCTR/CTR implementation
* Various small style fixes
Changes in v6:
* Split ARM64 XCTR/CTR refactoring into separate patch
* Allow simd POLYVAL implementations to be preempted
* Fix uninitialized bug in HCTR2
* Fix streamcipher name handling bug in HCTR2
* Various small style fixes
Changes in v5:
* Refactor HCTR2 tweak hashing
* Remove non-AVX x86-64 XCTR implementation
* Combine arm64 CTR and XCTR modes
* Comment and alias CTR and XCTR modes
* Move generic fallback code for simd POLYVAL into polyval-generic.c
* Various small style fixes
Changes in v4:
* Small style fixes in generic POLYVAL and XCTR
* Move HCTR2 hash exporting/importing to helper functions
* Rewrite montgomery reduction for x86-64 POLYVAL
* Rewrite partial block handling for x86-64 POLYVAL
* Optimize x86-64 POLYVAL loop handling
* Remove ahash wrapper from x86-64 POLYVAL
* Add simd-unavailable handling to x86-64 POLYVAL
* Rewrite montgomery reduction for ARM64 POLYVAL
* Rewrite partial block handling for ARM64 POLYVAL
* Optimize ARM64 POLYVAL loop handling
* Remove ahash wrapper from ARM64 POLYVAL
* Add simd-unavailable handling to ARM64 POLYVAL
Changes in v3:
* Improve testvec coverage for XCTR, POLYVAL and HCTR2
* Fix endianness bug in xctr.c
* Fix alignment issues in polyval-generic.c
* Optimize hctr2.c by exporting/importing hash states
* Fix blockcipher name derivation in hctr2.c
* Move x86-64 XCTR implementation into aes_ctrby8_avx-x86_64.S
* Reuse ARM64 CTR mode tail handling in ARM64 XCTR
* Fix x86-64 POLYVAL comments
* Fix x86-64 POLYVAL key_powers type to match asm
* Fix ARM64 POLYVAL comments
* Fix ARM64 POLYVAL key_powers type to match asm
* Add XTS + HCTR2 policy to fscrypt
Nathan Huckleberry (9):
crypto: xctr - Add XCTR support
crypto: polyval - Add POLYVAL support
crypto: hctr2 - Add HCTR2 support
crypto: x86/aesni-xctr: Add accelerated implementation of XCTR
crypto: arm64/aes-xctr: Add accelerated implementation of XCTR
crypto: arm64/aes-xctr: Improve readability of XCTR and CTR modes
crypto: x86/polyval: Add PCLMULQDQ accelerated implementation of
POLYVAL
crypto: arm64/polyval: Add PMULL accelerated implementation of POLYVAL
fscrypt: Add HCTR2 support for filename encryption
Documentation/filesystems/fscrypt.rst | 22 +- arch/arm64/crypto/Kconfig | 9 +- arch/arm64/crypto/Makefile | 3 + arch/arm64/crypto/aes-glue.c | 82 +- arch/arm64/crypto/aes-modes.S | 338 +++-- arch/arm64/crypto/polyval-ce-core.S | 361 ++++++ arch/arm64/crypto/polyval-ce-glue.c | 193 +++ arch/x86/crypto/Makefile | 3 + arch/x86/crypto/aes_ctrby8_avx-x86_64.S | 232 ++-- arch/x86/crypto/aesni-intel_glue.c | 114 +- arch/x86/crypto/polyval-clmulni_asm.S | 321 +++++ arch/x86/crypto/polyval-clmulni_glue.c | 203 +++ crypto/Kconfig | 39 +- crypto/Makefile | 3 + crypto/hctr2.c | 581 +++++++++ crypto/polyval-generic.c | 245 ++++ crypto/tcrypt.c | 10 + crypto/testmgr.c | 20 + crypto/testmgr.h | 1536 +++++++++++++++++++++++ crypto/xctr.c | 191 +++ fs/crypto/fscrypt_private.h | 2 +- fs/crypto/keysetup.c | 7 + fs/crypto/policy.c | 14 +- include/crypto/polyval.h | 22 + include/uapi/linux/fscrypt.h | 3 +- 25 files changed, 4355 insertions(+), 199 deletions(-) create mode 100644 arch/arm64/crypto/polyval-ce-core.S create mode 100644 arch/arm64/crypto/polyval-ce-glue.c create mode 100644 arch/x86/crypto/polyval-clmulni_asm.S create mode 100644 arch/x86/crypto/polyval-clmulni_glue.c create mode 100644 crypto/hctr2.c create mode 100644 crypto/polyval-generic.c create mode 100644 crypto/xctr.c create mode 100644 include/crypto/polyval.h
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 1/9 crypto: xctr - Add XCTR support
Add a generic implementation of XCTR mode as a template. XCTR is a blockcipher mode similar to CTR mode. XCTR uses XORs and little-endian addition rather than big-endian arithmetic which has two advantages: It is slightly faster on little-endian CPUs and it is less likely to be implemented incorrect since integer overflows are not possible on practical input sizes. XCTR is used as a component to implement HCTR2.
More information on XCTR mode can be found in the HCTR2 paper:
https://eprint.iacr.org/2021/1441.pdf
--- crypto/Kconfig | 9 + crypto/Makefile | 1 + crypto/tcrypt.c | 1 + crypto/testmgr.c | 6 + crypto/testmgr.h | 693 +++++++++++++++++++++++++++++++++++++++++++++++ crypto/xctr.c | 191 +++++++++++++ 6 files changed, 901 insertions(+) create mode 100644 crypto/xctr.c
crypto/Kconfig +9 -0
@@ -460,6 +460,15 @@ config CRYPTO_PCBC PCBC: Propagating Cipher Block Chaining mode This block cipher algorithm is required for RxRPC. +config CRYPTO_XCTR + tristate + select CRYPTO_SKCIPHER + select CRYPTO_MANAGER + help + XCTR: XOR Counter mode. This blockcipher mode is a variant of CTR mode + using XORs and little-endian addition rather than big-endian arithmetic. + XCTR mode is used to implement HCTR2. + config CRYPTO_XTS tristate "XTS support" select CRYPTO_SKCIPHER
crypto/Makefile +1 -0
@@ -93,6 +93,7 @@ obj-$(CONFIG_CRYPTO_CTS) += cts.o obj-$(CONFIG_CRYPTO_LRW) += lrw.o obj-$(CONFIG_CRYPTO_XTS) += xts.o obj-$(CONFIG_CRYPTO_CTR) += ctr.o +obj-$(CONFIG_CRYPTO_XCTR) += xctr.o obj-$(CONFIG_CRYPTO_KEYWRAP) += keywrap.o obj-$(CONFIG_CRYPTO_ADIANTUM) += adiantum.o obj-$(CONFIG_CRYPTO_NHPOLY1305) += nhpoly1305.o
crypto/tcrypt.c +1 -0
@@ -1556,6 +1556,7 @@ static int do_test(const char *alg, u32 type, u32 mask, int m, u32 num_mb) ret +=tcrypt_test ("rfc3686(ctr(aes))" ); ret +=tcrypt_test ("ofb(aes)" ); ret +=tcrypt_test ("cfb(aes)" ); + ret +=tcrypt_test ("xctr(aes)" );break ;case 11 :
crypto/testmgr.c +6 -0
@@ -5490,6 +5490,12 @@ static const struct alg_test_desc alg_test_descs[] = { .suite = { .cipher = __VECS(xchacha20_tv_template) }, + }, { + .alg = "xctr(aes)", + .test = alg_test_skcipher, + .suite = { + .cipher = __VECS(aes_xctr_tv_template) + } }, { .alg = "xts(aes)", .generic_driver = "xts(ecb(aes-generic))",
crypto/testmgr.h +693 -0
697 lines not on this page · raw message
crypto/xctr.c new file +191 -0
191 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 2/9 crypto: polyval - Add POLYVAL support
In reply to Nathan HuckleberryAdd support for POLYVAL, an ε-Δ-universal hash function similar to GHASH. This patch only uses POLYVAL as a component to implement HCTR2 mode. It should be noted that POLYVAL was originally specified for use in AES-GCM-SIV (RFC 8452), but the kernel does not currently support this mode.
POLYVAL is implemented as an shash algorithm. The implementation is modified from ghash-generic.c.
For more information on POLYVAL see:
Length-preserving encryption with HCTR2:
https://eprint.iacr.org/2021/1441.pdf
AES-GCM-SIV: Nonce Misuse-Resistant Authenticated Encryption:
https://datatracker.ietf.org/doc/html/rfc8452
--- crypto/Kconfig | 8 ++ crypto/Makefile | 1 + crypto/polyval-generic.c | 205 +++++++++++++++++++++++++++++++++++++++ crypto/tcrypt.c | 4 + crypto/testmgr.c | 6 ++ crypto/testmgr.h | 171 ++++++++++++++++++++++++++++++++ include/crypto/polyval.h | 17 ++++ 7 files changed, 412 insertions(+) create mode 100644 crypto/polyval-generic.c create mode 100644 include/crypto/polyval.h
crypto/Kconfig +8 -0
@@ -768,6 +768,14 @@ config CRYPTO_GHASH GHASH is the hash function used in GCM (Galois/Counter Mode). It is not a general-purpose cryptographic hash function. +config CRYPTO_POLYVAL + tristate + select CRYPTO_GF128MUL + select CRYPTO_HASH + help + POLYVAL is the hash function used in HCTR2. It is not a general-purpose + cryptographic hash function. + config CRYPTO_POLY1305 tristate "Poly1305 authenticator algorithm" select CRYPTO_HASH
crypto/Makefile +1 -0
@@ -169,6 +169,7 @@ UBSAN_SANITIZE_jitterentropy.o = n jitterentropy_rng-y := jitterentropy.o jitterentropy-kcapi.o obj-$(CONFIG_CRYPTO_TEST) += tcrypt.o obj-$(CONFIG_CRYPTO_GHASH) += ghash-generic.o +obj-$(CONFIG_CRYPTO_POLYVAL) += polyval-generic.o obj-$(CONFIG_CRYPTO_USER_API) += af_alg.o obj-$(CONFIG_CRYPTO_USER_API_HASH) += algif_hash.o obj-$(CONFIG_CRYPTO_USER_API_SKCIPHER) += algif_skcipher.o
crypto/polyval-generic.c new file +205 -0
205 lines not on this page · raw message
crypto/tcrypt.c +4 -0
@@ -1730,6 +1730,10 @@ static int do_test(const char *alg, u32 type, u32 mask, int m, u32 num_mb) ret +=tcrypt_test ("ccm(sm4)" );break ; +case 57 : + ret +=tcrypt_test ("polyval" ); +break ; +case 100 : ret +=tcrypt_test ("hmac(md5)" );break ;
crypto/testmgr.c +6 -0
@@ -5284,6 +5284,12 @@ static const struct alg_test_desc alg_test_descs[] = { .suite = { .hash = __VECS(poly1305_tv_template) } + }, { + .alg = "polyval", + .test = alg_test_hash, + .suite = { + .hash = __VECS(polyval_tv_template) + } }, { .alg = "rfc3686(ctr(aes))", .test = alg_test_skcipher,
crypto/testmgr.h +171 -0
175 lines not on this page · raw message
include/crypto/polyval.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Common values for the Polyval hash algorithm +* +* Copyright 2021 Google LLC +*/ + +#ifndef _CRYPTO_POLYVAL_H +#define _CRYPTO_POLYVAL_H + +#include <linux/types.h> +#include <linux/crypto.h> + +#define POLYVAL_BLOCK_SIZE 16 +#define POLYVAL_DIGEST_SIZE 16 + +#endif
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 3/9 crypto: hctr2 - Add HCTR2 support
In reply to Nathan HuckleberryAdd support for HCTR2 as a template. HCTR2 is a length-preserving encryption mode that is efficient on processors with instructions to accelerate AES and carryless multiplication, e.g. x86 processors with AES-NI and CLMUL, and ARM processors with the ARMv8 Crypto Extensions.
As a length-preserving encryption mode, HCTR2 is suitable for applications such as storage encryption where ciphertext expansion is not possible, and thus authenticated encryption cannot be used. Currently, such applications usually use XTS, or in some cases Adiantum. XTS has the disadvantage that it is a narrow-block mode: a bitflip will only change 16 bytes in the resulting ciphertext or plaintext. This reveals more information to an attacker than necessary.
HCTR2 is a wide-block mode, so it provides a stronger security property:
a bitflip will change the entire message. HCTR2 is somewhat similar to Adiantum, which is also a wide-block mode. However, HCTR2 is designed to take advantage of existing crypto instructions, while Adiantum targets devices without such hardware support. Adiantum is also designed with longer messages in mind, while HCTR2 is designed to be efficient even on short messages.
HCTR2 requires POLYVAL and XCTR as components. More information on HCTR2 can be found here: "Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf
--- crypto/Kconfig | 11 + crypto/Makefile | 1 + crypto/hctr2.c | 581 ++++++++++++++++++++++++++++++++++++++++ crypto/tcrypt.c | 5 + crypto/testmgr.c | 8 + crypto/testmgr.h | 672 +++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1278 insertions(+) create mode 100644 crypto/hctr2.c
crypto/Kconfig +11 -0
@@ -532,6 +532,17 @@ config CRYPTO_ADIANTUM If unsure, say N. +config CRYPTO_HCTR2 + tristate "HCTR2 support" + select CRYPTO_XCTR + select CRYPTO_POLYVAL + select CRYPTO_MANAGER + help + HCTR2 is a length-preserving encryption mode for storage encryption that + is efficient on processors with instructions to accelerate AES and + carryless multiplication, e.g. x86 processors with AES-NI and CLMUL, and + ARM processors with the ARMv8 crypto extensions. + config CRYPTO_ESSIV tristate "ESSIV support for block encryption" select CRYPTO_AUTHENC
crypto/Makefile +1 -0
@@ -94,6 +94,7 @@ obj-$(CONFIG_CRYPTO_LRW) += lrw.o obj-$(CONFIG_CRYPTO_XTS) += xts.o obj-$(CONFIG_CRYPTO_CTR) += ctr.o obj-$(CONFIG_CRYPTO_XCTR) += xctr.o +obj-$(CONFIG_CRYPTO_HCTR2) += hctr2.o obj-$(CONFIG_CRYPTO_KEYWRAP) += keywrap.o obj-$(CONFIG_CRYPTO_ADIANTUM) += adiantum.o obj-$(CONFIG_CRYPTO_NHPOLY1305) += nhpoly1305.o
crypto/hctr2.c new file +581 -0
581 lines not on this page · raw message
crypto/tcrypt.c +5 -0
@@ -2191,6 +2191,11 @@ static int do_test(const char *alg, u32 type, u32 mask, int m, u32 num_mb) 16, 16, aead_speed_template_19, num_mb);break ; +case 226 : +test_cipher_speed ("hctr2(aes)" ,ENCRYPT , sec,NULL , +0 , speed_template_32); +break ; +case 300 :if (alg) {test_hash_speed (alg, sec, generic_hash_speed_template);
crypto/testmgr.c +8 -0
@@ -5030,6 +5030,14 @@ static const struct alg_test_desc alg_test_descs[] = { .suite = { .hash = __VECS(ghash_tv_template) } + }, { + .alg = "hctr2(aes)", + .generic_driver = + "hctr2_base(xctr(aes-generic),polyval-generic)", + .test = alg_test_skcipher, + .suite = { + .cipher = __VECS(aes_hctr2_tv_template) + } }, { .alg = "hmac(md5)", .test = alg_test_hash,
crypto/testmgr.h +672 -0
676 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 4/9 crypto: x86/aesni-xctr: Add accelerated implementation of XCTR
In reply to Nathan HuckleberryAdd hardware accelerated version of XCTR for x86-64 CPUs with AESNI support.
More information on XCTR can be found in the HCTR2 paper:
"Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf
--- arch/x86/crypto/aes_ctrby8_avx-x86_64.S | 232 ++++++++++++++++-------- arch/x86/crypto/aesni-intel_glue.c | 114 +++++++++++- crypto/Kconfig | 2 +- 3 files changed, 266 insertions(+), 82 deletions(-)
arch/x86/crypto/aes_ctrby8_avx-x86_64.S +152 -80
@@ -23,6 +23,11 @@ #define VMOVDQ vmovdqu +/* + * Note: the "x" prefix in these aliases means "this is an xmm register". The + * alias prefixes have no relation to XCTR where the "X" prefix means "XOR + * counter". + */ #define xdata0 %xmm0 #define xdata1 %xmm1 #define xdata2 %xmm2
@@ -31,8 +36,10 @@ #define xdata5 %xmm5 #define xdata6 %xmm6 #define xdata7 %xmm7 -#define xcounter %xmm8 -#define xbyteswap %xmm9 +#define xcounter %xmm8 // CTR mode only +#define xiv %xmm8 // XCTR mode only +#define xbyteswap %xmm9 // CTR mode only +#define xtmp %xmm9 // XCTR mode only #define xkey0 %xmm10 #define xkey4 %xmm11 #define xkey8 %xmm12
@@ -45,7 +52,7 @@ #define p_keys %rdx #define p_out %rcx #define num_bytes %r8 - +#define counter %r9 // XCTR mode only #define tmp %r10 #define DDQ_DATA 0 #define XDATA 1
@@ -102,7 +109,7 @@ ddq_add_8: * do_aes num_in_par load_keys key_len * This increments p_in, but not p_out */ -.macro do_aes b, k, key_len +.macro do_aes b, k, key_len, xctr .set by, \b .set load_keys, \k .set klen, \key_len
@@ -111,29 +118,48 @@ ddq_add_8: vmovdqa 0*16(p_keys), xkey0 .endif - vpshufb xbyteswap, xcounter, xdata0 - - .set i, 1 - .rept (by - 1) - club XDATA, i - vpaddq (ddq_add_1 + 16 * (i - 1))(%rip), xcounter, var_xdata - vptest ddq_low_msk(%rip), var_xdata - jnz 1f - vpaddq ddq_high_add_1(%rip), var_xdata, var_xdata - vpaddq ddq_high_add_1(%rip), xcounter, xcounter - 1: - vpshufb xbyteswap, var_xdata, var_xdata - .set i, (i +1) - .endr + .if \xctr + movq counter, xtmp + .set i, 0 + .rept (by) + club XDATA, i + vpaddq (ddq_add_1 + 16 * i)(%rip), xtmp, var_xdata + .set i, (i +1) + .endr + .set i, 0 + .rept (by) + club XDATA, i + vpxor xiv, var_xdata, var_xdata + .set i, (i +1) + .endr + .else + vpshufb xbyteswap, xcounter, xdata0 + .set i, 1 + .rept (by - 1) + club XDATA, i + vpaddq (ddq_add_1 + 16 * (i - 1))(%rip), xcounter, var_xdata + vptest ddq_low_msk(%rip), var_xdata + jnz 1f + vpaddq ddq_high_add_1(%rip), var_xdata, var_xdata + vpaddq ddq_high_add_1(%rip), xcounter, xcounter + 1: + vpshufb xbyteswap, var_xdata, var_xdata + .set i, (i +1) + .endr + .endif vmovdqa 1*16(p_keys), xkeyA vpxor xkey0, xdata0, xdata0 - vpaddq (ddq_add_1 + 16 * (by - 1))(%rip), xcounter, xcounter - vptest ddq_low_msk(%rip), xcounter - jnz 1f - vpaddq ddq_high_add_1(%rip), xcounter, xcounter - 1: + .if \xctr + add $by, counter + .else + vpaddq (ddq_add_1 + 16 * (by - 1))(%rip), xcounter, xcounter + vptest ddq_low_msk(%rip), xcounter + jnz 1f + vpaddq ddq_high_add_1(%rip), xcounter, xcounter + 1: + .endif .set i, 1 .rept (by - 1)
@@ -371,94 +397,99 @@ ddq_add_8: .endr .endm -.macro do_aes_load val, key_len - do_aes \val, 1, \key_len +.macro do_aes_load val, key_len, xctr + do_aes \val, 1, \key_len, \xctr .endm -.macro do_aes_noload val, key_len - do_aes \val, 0, \key_len +.macro do_aes_noload val, key_len, xctr + do_aes \val, 0, \key_len, \xctr .endm /* main body of aes ctr load */ -.macro do_aes_ctrmain key_len +.macro do_aes_ctrmain key_len, xctr cmp $16, num_bytes - jb .Ldo_return2\key_len + jb .Ldo_return2\xctr\key_len - vmovdqa byteswap_const(%rip), xbyteswap - vmovdqu (p_iv), xcounter - vpshufb xbyteswap, xcounter, xcounter + .if \xctr + shr $4, counter + vmovdqu (p_iv), xiv + .else + vmovdqa byteswap_const(%rip), xbyteswap + vmovdqu (p_iv), xcounter + vpshufb xbyteswap, xcounter, xcounter + .endif mov num_bytes, tmp and $(7*16), tmp - jz .Lmult_of_8_blks\key_len + jz .Lmult_of_8_blks\xctr\key_len /* 1 <= tmp <= 7 */ cmp $(4*16), tmp - jg .Lgt4\key_len - je .Leq4\key_len + jg .Lgt4\xctr\key_len + je .Leq4\xctr\key_len -.Llt4\key_len: +.Llt4\xctr\key_len: cmp $(2*16), tmp - jg .Leq3\key_len - je .Leq2\key_len + jg .Leq3\xctr\key_len + je .Leq2\xctr\key_len -.Leq1\key_len: - do_aes_load 1, \key_len +.Leq1\xctr\key_len: + do_aes_load 1, \key_len, \xctr add $(1*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Leq2\key_len: - do_aes_load 2, \key_len +.Leq2\xctr\key_len: + do_aes_load 2, \key_len, \xctr add $(2*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Leq3\key_len: - do_aes_load 3, \key_len +.Leq3\xctr\key_len: + do_aes_load 3, \key_len, \xctr add $(3*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Leq4\key_len: - do_aes_load 4, \key_len +.Leq4\xctr\key_len: + do_aes_load 4, \key_len, \xctr add $(4*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Lgt4\key_len: +.Lgt4\xctr\key_len: cmp $(6*16), tmp - jg .Leq7\key_len - je .Leq6\key_len + jg .Leq7\xctr\key_len + je .Leq6\xctr\key_len -.Leq5\key_len: - do_aes_load 5, \key_len +.Leq5\xctr\key_len: + do_aes_load 5, \key_len, \xctr add $(5*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Leq6\key_len: - do_aes_load 6, \key_len +.Leq6\xctr\key_len: + do_aes_load 6, \key_len, \xctr add $(6*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Leq7\key_len: - do_aes_load 7, \key_len +.Leq7\xctr\key_len: + do_aes_load 7, \key_len, \xctr add $(7*16), p_out and $(~7*16), num_bytes - jz .Ldo_return2\key_len - jmp .Lmain_loop2\key_len + jz .Ldo_return2\xctr\key_len + jmp .Lmain_loop2\xctr\key_len -.Lmult_of_8_blks\key_len: +.Lmult_of_8_blks\xctr\key_len: .if (\key_len != KEY_128) vmovdqa 0*16(p_keys), xkey0 vmovdqa 4*16(p_keys), xkey4
@@ -471,17 +502,19 @@ ddq_add_8: vmovdqa 9*16(p_keys), xkey12 .endif .align 16 -.Lmain_loop2\key_len: +.Lmain_loop2\xctr\key_len: /* num_bytes is a multiple of 8 and >0 */ - do_aes_noload 8, \key_len + do_aes_noload 8, \key_len, \xctr add $(8*16), p_out sub $(8*16), num_bytes - jne .Lmain_loop2\key_len + jne .Lmain_loop2\xctr\key_len -.Ldo_return2\key_len: - /* return updated IV */ - vpshufb xbyteswap, xcounter, xcounter - vmovdqu xcounter, (p_iv) +.Ldo_return2\xctr\key_len: + .if !\xctr + /* return updated IV */ + vpshufb xbyteswap, xcounter, xcounter + vmovdqu xcounter, (p_iv) + .endif RET .endm
@@ -494,7 +527,7 @@ ddq_add_8: */ SYM_FUNC_START(aes_ctr_enc_128_avx_by8) /* call the aes main loop */ - do_aes_ctrmain KEY_128 + do_aes_ctrmain KEY_128 0 SYM_FUNC_END(aes_ctr_enc_128_avx_by8)
@@ -507,7 +540,7 @@ SYM_FUNC_END(aes_ctr_enc_128_avx_by8) */ SYM_FUNC_START(aes_ctr_enc_192_avx_by8) /* call the aes main loop */ - do_aes_ctrmain KEY_192 + do_aes_ctrmain KEY_192 0 SYM_FUNC_END(aes_ctr_enc_192_avx_by8)
@@ -520,6 +553,45 @@ SYM_FUNC_END(aes_ctr_enc_192_avx_by8) */ SYM_FUNC_START(aes_ctr_enc_256_avx_by8) /* call the aes main loop */ - do_aes_ctrmain KEY_256 + do_aes_ctrmain KEY_256 0 SYM_FUNC_END(aes_ctr_enc_256_avx_by8) + +/* + * routine to do AES128 XCTR enc/decrypt "by8" + * XMM registers are clobbered. + * Saving/restoring must be done at a higher level + * aes_xctr_enc_128_avx_by8(const u8 *in, const u8 *iv, const void *keys, + * u8* out, unsigned int num_bytes, unsigned int byte_ctr) + */ +SYM_FUNC_START(aes_xctr_enc_128_avx_by8) + /* call the aes main loop */ + do_aes_ctrmain KEY_128 1 + +SYM_FUNC_END(aes_xctr_enc_128_avx_by8) + +/* + * routine to do AES192 XCTR enc/decrypt "by8" + * XMM registers are clobbered. + * Saving/restoring must be done at a higher level + * aes_xctr_enc_192_avx_by8(const u8 *in, const u8 *iv, const void *keys, + * u8* out, unsigned int num_bytes, unsigned int byte_ctr) + */ +SYM_FUNC_START(aes_xctr_enc_192_avx_by8) + /* call the aes main loop */ + do_aes_ctrmain KEY_192 1 + +SYM_FUNC_END(aes_xctr_enc_192_avx_by8) + +/* + * routine to do AES256 XCTR enc/decrypt "by8" + * XMM registers are clobbered. + * Saving/restoring must be done at a higher level + * aes_xctr_enc_256_avx_by8(const u8 *in, const u8 *iv, const void *keys, + * u8* out, unsigned int num_bytes, unsigned int byte_ctr) + */ +SYM_FUNC_START(aes_xctr_enc_256_avx_by8) + /* call the aes main loop */ + do_aes_ctrmain KEY_256 1 + +SYM_FUNC_END(aes_xctr_enc_256_avx_by8)
arch/x86/crypto/aesni-intel_glue.c +113 -1
152 lines not on this page · raw message
crypto/Kconfig +1 -1
@@ -1161,7 +1161,7 @@ config CRYPTO_AES_NI_INTEL In addition to AES cipher algorithm support, the acceleration for some popular block cipher mode is supported too, including ECB, CBC, LRW, XTS. The 64 bit version has additional - acceleration for CTR. + acceleration for CTR and XCTR. config CRYPTO_AES_SPARC64 tristate "AES cipher algorithms (SPARC64)"
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 5/9 crypto: arm64/aes-xctr: Add accelerated implementation of XCTR
In reply to Nathan HuckleberryAdd hardware accelerated version of XCTR for ARM64 CPUs with ARMv8 Crypto Extension support. This XCTR implementation is based on the CTR implementation in aes-modes.S.
More information on XCTR can be found in
the HCTR2 paper: "Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf
--- arch/arm64/crypto/Kconfig | 4 +- arch/arm64/crypto/aes-glue.c | 64 ++++++++++++- arch/arm64/crypto/aes-modes.S | 168 +++++++++++++++++++++------------- 3 files changed, 169 insertions(+), 67 deletions(-)
arch/arm64/crypto/Kconfig +2 -2
@@ -84,13 +84,13 @@ config CRYPTO_AES_ARM64_CE_CCM select CRYPTO_LIB_AES config CRYPTO_AES_ARM64_CE_BLK - tristate "AES in ECB/CBC/CTR/XTS modes using ARMv8 Crypto Extensions" + tristate "AES in ECB/CBC/CTR/XTS/XCTR modes using ARMv8 Crypto Extensions" depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER select CRYPTO_AES_ARM64_CE config CRYPTO_AES_ARM64_NEON_BLK - tristate "AES in ECB/CBC/CTR/XTS modes using NEON instructions" + tristate "AES in ECB/CBC/CTR/XTS/XCTR modes using NEON instructions" depends on KERNEL_MODE_NEON select CRYPTO_SKCIPHER select CRYPTO_LIB_AES
arch/arm64/crypto/aes-glue.c +62 -2
106 lines not on this page · raw message
arch/arm64/crypto/aes-modes.S +105 -63
226 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
PATCH v7 6/9 crypto: arm64/aes-xctr: Improve readability of XCTR and CTR modes
In reply to Nathan HuckleberryAdded some clarifying comments, changed the register allocations to make the code clearer, and added register aliases.
--- arch/arm64/crypto/aes-glue.c | 18 +++ arch/arm64/crypto/aes-modes.S | 226 ++++++++++++++++++++++++---------- 2 files changed, 176 insertions(+), 68 deletions(-)
arch/arm64/crypto/aes-glue.c +18 -0
@@ -464,6 +464,15 @@ static int __maybe_unused xctr_encrypt(struct skcipher_request *req)u8 *dst = walk.dst.virt.addr;u8 buf[AES_BLOCK_SIZE ]; +/* +* XCTR mode can only handle a partial block at the very end. +* Since chunksize=16 we know that we always get at least 16 +* bytes (except possibly at the very end). +* +* If given less than 16 bytes, we must copy the partial block +* into a temporary buffer of 16 bytes to avoid out of bounds +* reads and writes. +*/ if (unlikely (nbytes <AES_BLOCK_SIZE )) src = dst =memcpy (buf +sizeof (buf) - nbytes, src, nbytes);
@@ -501,6 +510,15 @@ static int __maybe_unused ctr_encrypt(struct skcipher_request *req)u8 *dst = walk.dst.virt.addr;u8 buf[AES_BLOCK_SIZE ]; +/* +* CTR mode can only handle a partial block at the very end. +* Since chunksize=16 we know that we always get at least 16 +* bytes (except possibly at the very end). +* +* If given less than 16 bytes, we must copy the partial block +* into a temporary buffer of 16 bytes to avoid out of bounds +* reads and writes. +*/ if (unlikely (nbytes <AES_BLOCK_SIZE )) src = dst =memcpy (buf +sizeof (buf) - nbytes, src, nbytes);
arch/arm64/crypto/aes-modes.S +158 -68
335 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Eric Biggers
maintainerRe: [PATCH v7 6/9] crypto: arm64/aes-xctr: Improve readability of XCTR and CTR modes
Quoted 5 lines · Nathan Huckleberry
Added some clarifying comments, changed the register allocations to make the code clearer, and added register aliases.
Signed-off-by Nathan Huckleberry <[email protected]>Reviewed-by Eric Biggers <[email protected]>
Did you mean to add Ard's Reviewed-by that he gave on v6 as well?
One comment about the v7 changes below:
arch/arm64/crypto/aes-modes.S ST5( eor v7.16b, v7.16b, v2.16b ) · line 595
Quoted 8 lines
arch/arm64/crypto/aes-modes.S +4
-0/* * aes_ctr_encrypt(u8 out[], u8 const in[], u8 const rk[], int rounds, * int bytes, u8 ctr[]) + * + * The input and output buffers must always be at least 16 bytes even if + * encrypting/decrypting less than 16 bytes. Otherwise out of bounds + * accesses will occur. */
This comment, along with the other similar ones you added, doesn't properly describe the behavior when bytes < 16, as it's not mentioned that the extra space needs to be before the pointed-to regions rather than after. That's the most unusual part of these functions, so it really should be mentioned.
Separately, applying this patch and the previous one causes the following whitespace errors to be reported:
Applying: crypto: arm64/aes-xctr: Add accelerated implementation of XCTR .git/rebase-apply/patch:299: space before tab in indent.
ld1 {v5.16b-v7.16b}, [x1], #48
warning: 1 line adds whitespace errors.
Applying: crypto: arm64/aes-xctr: Improve readability of XCTR and CTR modes
.git/rebase-apply/patch:216: space before tab in indent.
ld1 {v5.16b-v7.16b}, [IN], #48warning: 1 line adds whitespace errors.
- Eric
PATCH v7 7/9 crypto: x86/polyval: Add PCLMULQDQ accelerated implementation of POLYVAL
In reply to Nathan HuckleberryAdd hardware accelerated version of POLYVAL for x86-64 CPUs with PCLMULQDQ support.
This implementation is accelerated using PCLMULQDQ instructions to perform the finite field computations. For added efficiency, 8 blocks of the message are processed simultaneously by precomputing the first 8 powers of the key.
Schoolbook multiplication is used instead of Karatsuba multiplication because it was found to be slightly faster on x86-64 machines. Montgomery reduction must be used instead of Barrett reduction due to the difference in modulus between POLYVAL's field and other finite fields.
More information on POLYVAL can be found in the HCTR2 paper:
"Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf
--- arch/x86/crypto/Makefile | 3 + arch/x86/crypto/polyval-clmulni_asm.S | 321 +++++++++++++++++++++++++ arch/x86/crypto/polyval-clmulni_glue.c | 203 ++++++++++++++++ crypto/Kconfig | 9 + crypto/polyval-generic.c | 40 +++ include/crypto/polyval.h | 5 + 6 files changed, 581 insertions(+) create mode 100644 arch/x86/crypto/polyval-clmulni_asm.S create mode 100644 arch/x86/crypto/polyval-clmulni_glue.c
arch/x86/crypto/Makefile +3 -0
@@ -69,6 +69,9 @@ libblake2s-x86_64-y := blake2s-core.o blake2s-glue.o obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o +obj-$(CONFIG_CRYPTO_POLYVAL_CLMUL_NI) += polyval-clmulni.o +polyval-clmulni-y := polyval-clmulni_asm.o polyval-clmulni_glue.o + obj-$(CONFIG_CRYPTO_CRC32C_INTEL) += crc32c-intel.o crc32c-intel-y := crc32c-intel_glue.o crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o
arch/x86/crypto/polyval-clmulni_asm.S new file +321 -0
321 lines not on this page · raw message
arch/x86/crypto/polyval-clmulni_glue.c new file +203 -0
203 lines not on this page · raw message
crypto/Kconfig +9 -0
@@ -787,6 +787,15 @@ config CRYPTO_POLYVAL POLYVAL is the hash function used in HCTR2. It is not a general-purpose cryptographic hash function. +config CRYPTO_POLYVAL_CLMUL_NI + tristate "POLYVAL hash function (CLMUL-NI accelerated)" + depends on X86 && 64BIT + select CRYPTO_POLYVAL + help + This is the x86_64 CLMUL-NI accelerated implementation of POLYVAL. It is + used to efficiently implement HCTR2 on x86-64 processors that support + carry-less multiplication instructions. + config CRYPTO_POLY1305 tristate "Poly1305 authenticator algorithm" select CRYPTO_HASH
crypto/polyval-generic.c +40 -0
@@ -76,6 +76,46 @@ static void copy_and_reverse(u8 dst[POLYVAL_BLOCK_SIZE],put_unaligned (swab64 (b), (u64 *)&dst[0 ]); } +/* +* Performs multiplication in the POLYVAL field using the GHASH field as a +* subroutine. This function is used as a fallback for hardware accelerated +* implementations when simd registers are unavailable. +* +* Note: This function is not used for polyval-generic, instead we use the 4k +* lookup table implementation for finite field multiplication. +*/ +void polyval_mul_non4k (u8 *op1,const u8 *op2) +{ +be128 a, b; + +// Assume one argument is in Montgomery form and one is not. +copy_and_reverse ((u8 *)&a, op1); +copy_and_reverse ((u8 *)&b, op2); +gf128mul_x_lle (&a, &a); +gf128mul_lle (&a, &b); +copy_and_reverse (op1, (u8 *)&a); +} +EXPORT_SYMBOL_GPL (polyval_mul_non4k); + +/* +* Perform a POLYVAL update using non4k multiplication. This function is used +* as a fallback for hardware accelerated implementations when simd registers +* are unavailable. +* +* Note: This function is not used for polyval-generic, instead we use the 4k +* lookup table implementation of finite field multiplication. +*/ +void polyval_update_non4k (const u8 *key,const u8 *in, +size_t nblocks,u8 *accumulator) +{ +while (nblocks--) { +crypto_xor (accumulator, in,POLYVAL_BLOCK_SIZE ); +polyval_mul_non4k (accumulator, key); + in +=POLYVAL_BLOCK_SIZE ; + } +} +EXPORT_SYMBOL_GPL (polyval_update_non4k); +static int polyval_setkey (struct crypto_shash *tfm,const u8 *key,unsigned int keylen) {
include/crypto/polyval.h +5 -0
@@ -14,4 +14,9 @@#define POLYVAL_BLOCK_SIZE 16#define POLYVAL_DIGEST_SIZE 16 +void polyval_mul_non4k (u8 *op1,const u8 *op2); + +void polyval_update_non4k (const u8 *key,const u8 *in, +size_t nblocks,u8 *accumulator); +#endif
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Re: [PATCH v7 7/9] crypto: x86/polyval: Add PCLMULQDQ accelerated implementation of POLYVAL
Quoted 1 line · Nathan Huckleberry
arch/x86/crypto/polyval-clmulni_asm.S +0
-0
[...]
Quoted 10 lines
+/*
+ * Computes the product of two 128-bit polynomials at the memory locations
+ * specified by (MSG + 16*i) and (KEY_POWERS+ 16*i) and XORs the components of
+ * the 256-bit product into LO, MI, HI.
+ *
+ * Given:
+ * X = [X_1:X_0]
+ * Y = [Y_1:Y_0]
+ *
+ * We compute:
Quoted 3 lines
+ * LO += X_0 * Y_0 + * MI += (X_0 + X_1) * (Y_0 + Y_1) + * HI += X_1 * Y_1
The above comment (changed in v7) is describing Karatsuba multiplication, but the actual code is using schoolbook multiplication.
Otherwise this looks good:
- Eric
PATCH v7 8/9 crypto: arm64/polyval: Add PMULL accelerated implementation of POLYVAL
In reply to Nathan HuckleberryAdd hardware accelerated version of POLYVAL for ARM64 CPUs with Crypto Extensions support.
This implementation is accelerated using PMULL instructions to perform the finite field computations. For added efficiency, 8 blocks of the message are processed simultaneously by precomputing the first 8 powers of the key.
Karatsuba multiplication is used instead of Schoolbook multiplication because it was found to be slightly faster on ARM64 CPUs. Montgomery reduction must be used instead of Barrett reduction due to the difference in modulus between POLYVAL's field and other finite fields.
More information on POLYVAL can be found in the HCTR2 paper:
"Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdf
--- arch/arm64/crypto/Kconfig | 5 + arch/arm64/crypto/Makefile | 3 + arch/arm64/crypto/polyval-ce-core.S | 361 ++++++++++++++++++++++++++++ arch/arm64/crypto/polyval-ce-glue.c | 193 +++++++++++++++ 4 files changed, 562 insertions(+) create mode 100644 arch/arm64/crypto/polyval-ce-core.S create mode 100644 arch/arm64/crypto/polyval-ce-glue.c
arch/arm64/crypto/Kconfig +5 -0
@@ -60,6 +60,11 @@ config CRYPTO_GHASH_ARM64_CE select CRYPTO_GF128MUL select CRYPTO_LIB_AES +config CRYPTO_POLYVAL_ARM64_CE + tristate "POLYVAL using ARMv8 Crypto Extensions (for HCTR2)" + depends on KERNEL_MODE_NEON + select CRYPTO_POLYVAL + config CRYPTO_CRCT10DIF_ARM64_CE tristate "CRCT10DIF digest algorithm using PMULL instructions" depends on KERNEL_MODE_NEON && CRC_T10DIF
arch/arm64/crypto/Makefile +3 -0
@@ -26,6 +26,9 @@ sm4-ce-y := sm4-ce-glue.o sm4-ce-core.o obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o ghash-ce-y := ghash-ce-glue.o ghash-ce-core.o +obj-$(CONFIG_CRYPTO_POLYVAL_ARM64_CE) += polyval-ce.o +polyval-ce-y := polyval-ce-glue.o polyval-ce-core.o + obj-$(CONFIG_CRYPTO_CRCT10DIF_ARM64_CE) += crct10dif-ce.o crct10dif-ce-y := crct10dif-ce-core.o crct10dif-ce-glue.o
arch/arm64/crypto/polyval-ce-core.S new file +361 -0
361 lines not on this page · raw message
arch/arm64/crypto/polyval-ce-glue.c new file +193 -0
193 lines not on this page · raw message
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog
Re: [PATCH v7 8/9] crypto: arm64/polyval: Add PMULL accelerated implementation of POLYVAL
Quoted 19 lines · Nathan Huckleberry
Add hardware accelerated version of POLYVAL for ARM64 CPUs with Crypto Extensions support.
This implementation is accelerated using PMULL instructions to perform the finite field computations. For added efficiency, 8 blocks of the message are processed simultaneously by precomputing the first 8 powers of the key.
Karatsuba multiplication is used instead of Schoolbook multiplication because it was found to be slightly faster on ARM64 CPUs. Montgomery reduction must be used instead of Barrett reduction due to the difference in modulus between POLYVAL's field and other finite fields.
More information on POLYVAL can be found in the HCTR2 paper:
"Length-preserving encryption with HCTR2":
https://eprint.iacr.org/2021/1441.pdfSigned-off-by Nathan Huckleberry <[email protected]>Reviewed-by Ard Biesheuvel <[email protected]>
Quoted 8 lines
--- arch/arm64/crypto/Kconfig | 5 + arch/arm64/crypto/Makefile | 3 + arch/arm64/crypto/polyval-ce-core.S | 361 ++++++++++++++++++++++++++++ arch/arm64/crypto/polyval-ce-glue.c | 193 +++++++++++++++ 4 files changed, 562 insertions(+) create mode 100644 arch/arm64/crypto/polyval-ce-core.S create mode 100644 arch/arm64/crypto/polyval-ce-glue.c
- Eric
PATCH v7 9/9 fscrypt: Add HCTR2 support for filename encryption
In reply to Nathan HuckleberryHCTR2 is a tweakable, length-preserving encryption mode that is intended for use on CPUs with dedicated crypto instructions. HCTR2 has the property that a bitflip in the plaintext changes the entire ciphertext. This property fixes a known weakness with filename encryption: when two filenames in the same directory share a prefix of >= 16 bytes, with AES-CTS-CBC their encrypted filenames share a common substring, leaking information. HCTR2 does not have this problem.
More information on HCTR2 can be found here: "Length-preserving encryption with HCTR2": https://eprint.iacr.org/2021/1441.pdf
--- Documentation/filesystems/fscrypt.rst | 22 +++++++++++++++++----- fs/crypto/fscrypt_private.h | 2 +- fs/crypto/keysetup.c | 7 +++++++ fs/crypto/policy.c | 14 +++++++++++--- include/uapi/linux/fscrypt.h | 3 ++- 5 files changed, 38 insertions(+), 10 deletions(-)
Documentation/filesystems/fscrypt.rst +17 -5
@@ -337,6 +337,7 @@ Currently, the following pairs of encryption modes are supported: - AES-256-XTS for contents and AES-256-CTS-CBC for filenames - AES-128-CBC for contents and AES-128-CTS-CBC for filenames - Adiantum for both contents and filenames +- AES-256-XTS for contents and AES-256-HCTR2 for filenames (v2 policies only) If unsure, you should use the (AES-256-XTS, AES-256-CTS-CBC) pair.
@@ -357,6 +358,17 @@ To use Adiantum, CONFIG_CRYPTO_ADIANTUM must be enabled. Also, fast implementations of ChaCha and NHPoly1305 should be enabled, e.g. CONFIG_CRYPTO_CHACHA20_NEON and CONFIG_CRYPTO_NHPOLY1305_NEON for ARM. +AES-256-HCTR2 is another true wide-block encryption mode that is intended for +use on CPUs with dedicated crypto instructions. AES-256-HCTR2 has the property +that a bitflip in the plaintext changes the entire ciphertext. This property +makes it desirable for filename encryption since initialization vectors are +reused within a directory. For more details on AES-256-HCTR2, see the paper +"Length-preserving encryption with HCTR2" +(https://eprint.iacr.org/2021/1441.pdf). To use AES-256-HCTR2, +CONFIG_CRYPTO_HCTR2 must be enabled. Also, fast implementations of XCTR and +POLYVAL should be enabled, e.g. CRYPTO_POLYVAL_ARM64_CE and +CRYPTO_AES_ARM64_CE_BLK for ARM64. + New encryption modes can be added relatively easily, without changes to individual filesystems. However, authenticated encryption (AE) modes are not currently supported because of the difficulty of dealing
@@ -404,11 +416,11 @@ alternatively has the file's nonce (for `DIRECT_KEY policies`_) or inode number (for `IV_INO_LBLK_64 policies`_) included in the IVs. Thus, IV reuse is limited to within a single directory. -With CTS-CBC, the IV reuse means that when the plaintext filenames -share a common prefix at least as long as the cipher block size (16 -bytes for AES), the corresponding encrypted filenames will also share -a common prefix. This is undesirable. Adiantum does not have this -weakness, as it is a wide-block encryption mode. +With CTS-CBC, the IV reuse means that when the plaintext filenames share a +common prefix at least as long as the cipher block size (16 bytes for AES), the +corresponding encrypted filenames will also share a common prefix. This is +undesirable. Adiantum and HCTR2 do not have this weakness, as they are +wide-block encryption modes. All supported filenames encryption modes accept any plaintext length >= 16 bytes; cipher block alignment is not required. However,
fs/crypto/fscrypt_private.h +1 -1
@@ -31,7 +31,7 @@#define FSCRYPT_CONTEXT_V2 2/* Keep this in sync with include/uapi/linux/fscrypt.h */ -#define FSCRYPT_MODE_MAX FSCRYPT_MODE_ADIANTUM +#define FSCRYPT_MODE_MAX FSCRYPT_MODE_AES_256_HCTR2struct fscrypt_context_v1 {u8 version;/* FSCRYPT_CONTEXT_V1 */
fs/crypto/keysetup.c +7 -0
@@ -53,6 +53,13 @@ struct fscrypt_mode fscrypt_modes[] = { .ivsize = 32, .blk_crypto_mode = BLK_ENCRYPTION_MODE_ADIANTUM, }, + [FSCRYPT_MODE_AES_256_HCTR2] = { + .friendly_name = "AES-256-HCTR2", + .cipher_str = "hctr2(aes)", + .keysize = 32, + .security_strength = 32, + .ivsize = 32, + }, };static DEFINE_MUTEX (fscrypt_mode_key_setup_mutex);
fs/crypto/policy.c +11 -3
@@ -40,7 +40,7 @@ fscrypt_get_dummy_policy(struct super_block *sb)return sb->s_cop->get_dummy_policy (sb); } -static bool fscrypt_valid_enc_modes (u32 contents_mode,u32 filenames_mode) +static bool fscrypt_valid_enc_modes_v1 (u32 contents_mode,u32 filenames_mode) {if (contents_mode ==FSCRYPT_MODE_AES_256_XTS && filenames_mode ==FSCRYPT_MODE_AES_256_CTS )
@@ -57,6 +57,14 @@ static bool fscrypt_valid_enc_modes(u32 contents_mode, u32 filenames_mode)return false; } +static bool fscrypt_valid_enc_modes_v2 (u32 contents_mode,u32 filenames_mode) +{ +if (contents_mode ==FSCRYPT_MODE_AES_256_XTS && + filenames_mode ==FSCRYPT_MODE_AES_256_HCTR2 ) +return true; +return fscrypt_valid_enc_modes_v1 (contents_mode, filenames_mode); +} +static bool supported_direct_key_modes (const struct inode *inode,u32 contents_mode,u32 filenames_mode) {
@@ -130,7 +138,7 @@ static bool supported_iv_ino_lblk_policy(const struct fscrypt_policy_v2 *policy,static bool fscrypt_supported_v1_policy (const struct fscrypt_policy_v1 *policy,const struct inode *inode) { -if (!fscrypt_valid_enc_modes (policy->contents_encryption_mode, +if (!fscrypt_valid_enc_modes_v1 (policy->contents_encryption_mode, policy->filenames_encryption_mode)) { fscrypt_warn(inode, "Unsupported encryption modes (contents %d, filenames %d)",
@@ -166,7 +174,7 @@ static bool fscrypt_supported_v2_policy(const struct fscrypt_policy_v2 *policy, {int count =0 ; -if (!fscrypt_valid_enc_modes (policy->contents_encryption_mode, +if (!fscrypt_valid_enc_modes_v2 (policy->contents_encryption_mode, policy->filenames_encryption_mode)) { fscrypt_warn(inode, "Unsupported encryption modes (contents %d, filenames %d)",
include/uapi/linux/fscrypt.h +2 -1
@@ -27,7 +27,8 @@#define FSCRYPT_MODE_AES_128_CBC 5#define FSCRYPT_MODE_AES_128_CTS 6#define FSCRYPT_MODE_ADIANTUM 9 -/* If adding a mode number > 9, update FSCRYPT_MODE_MAX in fscrypt_private.h */ +#define FSCRYPT_MODE_AES_256_HCTR2 10 +/* If adding a mode number > 10, update FSCRYPT_MODE_MAX in fscrypt_private.h */ /* * Legacy policy version; ad-hoc KDF and no key verification.
Signature · 1 line
2.36.0.512.ge40c2bad7a-goog