lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA-512, SHAKE128, SHAKE256
Subsystems (5)
David Howells
maintainerPATCH v2 1/1 lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA-512, SHAKE128, SHAKE256
Add SHA3, providing SHA3-224, SHA3-256, SHA3-384, SHA-512, SHAKE128 and SHAKE256 to lib/crypto.
The state array handling is simplified from what's in crypto/sha3_generic.c by keeping the state array (a u64[25]) in LE form and byteswapping all the entries before and after applying the keccak function on a BE system. This means no byteswapping is required when XOR'ing data into the state array or when extracting the digest. Further, this is a no-op on LE systems.
Also:
- Perform a multistage shake256 hash check in the module initialisation.
- Add kunit tests for each algorithm based on the gen-hash-testvecs.
- The conflicting static s390x crypto function names are renamed to have an s390_ prefix.
- gen-hash-testvecs.py had to be modified to be able to generate SHAKE hashes because Python's hashlib requires the output digest size supplying for those two algorithms as they produce arbitrary length digests.
Notes:
(1) I've left hooks in sha3.c for asm-optimised variants, but as I don't entirely know what those might look like, not having implemented any, the hooks' usability is uncertain.
(2) The SHAKE algorithms will be required for ML-DSA.
---
Changes
=======
v2)
- Simplify the endianness handling.
- Rename sha3_final() to sha3_squeeze() and don't clear the context at the end as it's permitted to continue calling sha3_final() to extract continuations of the digest (needed by ML-DSA).
- Don't reapply the end marker to the hash state in continuation sha3_squeeze() unless sha3_update() gets called again (needed by ML-DSA).
- Give sha3_squeeze() the amount of digest to produce as a parameter
rather than using ctx->digest_size and don't return the amount digested.
- Reimplement sha3_final() as a wrapper around sha3_squeeze() that extracts ctx->digest_size amount of digest and then zeroes out the context. The latter is necessary to avoid upsetting hash-test-template.h.
- Provide a sha3_reinit() function to clear the state, but to leave the parameters that indicate the hash properties unaffected, allowing for reuse.
- Provide a sha3_set_digestsize() function to change the size of the digest to be extracted by sha3_final(). sha3_squeeze() takes a parameter for this instead.
- Don't pass the digest size as a parameter to shake128/256_init() but rather default to 128/256 bits as per the function name.
- Provide a sha3_clear() function to zero out the context.
arch/s390/crypto/sha3_256_s390.c | 26 - include/crypto/sha3.h | 160 +++++++- lib/crypto/Kconfig | 7 lib/crypto/Makefile | 6 lib/crypto/sha3.c | 597 ++++++++++++++++++++++++++++++ lib/crypto/tests/Kconfig | 12 lib/crypto/tests/Makefile | 7 lib/crypto/tests/sha3_224_kunit.c | 32 + lib/crypto/tests/sha3_224_testvecs.h | 231 +++++++++++ lib/crypto/tests/sha3_256_kunit.c | 32 + lib/crypto/tests/sha3_256_testvecs.h | 231 +++++++++++ lib/crypto/tests/sha3_384_kunit.c | 32 + lib/crypto/tests/sha3_384_testvecs.h | 281 ++++++++++++++ lib/crypto/tests/sha3_512_kunit.c | 32 + lib/crypto/tests/sha3_512_testvecs.h | 331 ++++++++++++++++ lib/crypto/tests/sha3_shake128_kunit.c | 37 + lib/crypto/tests/sha3_shake128_testvecs.h | 181 +++++++++ lib/crypto/tests/sha3_shake256_kunit.c | 37 + lib/crypto/tests/sha3_shake256_testvecs.h | 231 +++++++++++ scripts/crypto/gen-hash-testvecs.py | 8 20 files changed, 2495 insertions(+), 16 deletions(-)
arch/s390/crypto/sha3_256_s390.c +13 -13
@@ -19,7 +19,7 @@#include "sha.h" -static int sha3_256_init (struct shash_desc *desc) +static int s390_sha3_256_init (struct shash_desc *desc) {struct s390_sha_ctx *sctx =shash_desc_ctx (desc);
@@ -32,7 +32,7 @@ static int sha3_256_init(struct shash_desc *desc)return 0 ; } -static int sha3_256_export (struct shash_desc *desc,void *out) +static int s390_sha3_256_export (struct shash_desc *desc,void *out) {struct s390_sha_ctx *sctx =shash_desc_ctx (desc);union {
@@ -50,7 +50,7 @@ static int sha3_256_export(struct shash_desc *desc, void *out)return 0 ; } -static int sha3_256_import (struct shash_desc *desc,const void *in) +static int s390_sha3_256_import (struct shash_desc *desc,const void *in) {struct s390_sha_ctx *sctx =shash_desc_ctx (desc);union {
@@ -68,22 +68,22 @@ static int sha3_256_import(struct shash_desc *desc, const void *in)return 0 ; } -static int sha3_224_import (struct shash_desc *desc,const void *in) +static int s390_sha3_224_import (struct shash_desc *desc,const void *in) {struct s390_sha_ctx *sctx =shash_desc_ctx (desc); -sha3_256_import (desc, in); +s390_sha3_256_import (desc, in); sctx->func =CPACF_KIMD_SHA3_224 ;return 0 ; }static struct shash_alg sha3_256_alg = { .digestsize =SHA3_256_DIGEST_SIZE ,/* = 32 */ - .init = sha3_256_init, + .init = s390_sha3_256_init, .update = s390_sha_update_blocks, .finup = s390_sha_finup, - .export = sha3_256_export, - .import = sha3_256_import, + .export = s390_sha3_256_export, + .import = s390_sha3_256_import, .descsize =S390_SHA_CTX_SIZE , .statesize =SHA3_STATE_SIZE , .base = {
@@ -96,22 +96,22 @@ static struct shash_alg sha3_256_alg = { } }; -static int sha3_224_init (struct shash_desc *desc) +static int s390_sha3_224_init (struct shash_desc *desc) {struct s390_sha_ctx *sctx =shash_desc_ctx (desc); -sha3_256_init (desc); +s390_sha3_256_init (desc); sctx->func =CPACF_KIMD_SHA3_224 ;return 0 ; }static struct shash_alg sha3_224_alg = { .digestsize =SHA3_224_DIGEST_SIZE , - .init = sha3_224_init, + .init = s390_sha3_224_init, .update = s390_sha_update_blocks, .finup = s390_sha_finup, - .export = sha3_256_export,/* same as for 256 */ - .import = sha3_224_import,/* function code different! */ + .export = s390_sha3_256_export,/* same as for 256 */ + .import = s390_sha3_224_import,/* function code different! */ .descsize =S390_SHA_CTX_SIZE , .statesize =SHA3_STATE_SIZE , .base = {
include/crypto/sha3.h +159 -1
177 lines not on this page · raw message
lib/crypto/Kconfig +7 -0
@@ -191,6 +191,13 @@ config CRYPTO_LIB_SHA512_ARCH default y if SPARC64 default y if X86_64 +config CRYPTO_LIB_SHA3 + tristate + help + The SHA3 library functions. + Select this if your module uses any of these functions from + <crypto/sha3.h>. + config CRYPTO_LIB_SM3 tristate
lib/crypto/Makefile +6 -0
@@ -149,6 +149,12 @@ endif # CONFIG_CRYPTO_LIB_SHA512_ARCH ################################################################################ +#obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o +#libsha3-y := sha3.o +obj-$(CONFIG_CRYPTO_SHA3) += sha3.o + +################################################################################ + obj-$(CONFIG_MPILIB) += mpi/ obj-$(CONFIG_CRYPTO_SELFTESTS_FULL) += simd.o
lib/crypto/sha3.c new file +597 -0
597 lines not on this page · raw message
lib/crypto/tests/Kconfig +12 -0
@@ -43,6 +43,18 @@ config CRYPTO_LIB_SHA512_KUNIT_TEST KUnit tests for the SHA-384 and SHA-512 cryptographic hash functions and their corresponding HMACs. +config CRYPTO_LIB_SHA3_KUNIT_TEST + tristate "KUnit tests for SHA-3" if !KUNIT_ALL_TESTS + depends on KUNIT + default KUNIT_ALL_TESTS || CRYPTO_SELFTESTS + select CRYPTO_LIB_BENCHMARK_VISIBLE + select CRYPTO_LIB_SHA3 + help + KUnit tests for the SHA3 cryptographic hash functions, including + SHA3-224, SHA3-256, SHA3-386, SHA3-512, SHAKE128 and SHAKE256. Note + that whilst the SHAKE* hash functions can support arbitrary-length + digests, these tests only check the nominal digest sizes for now. + config CRYPTO_LIB_BENCHMARK_VISIBLE bool
lib/crypto/tests/Makefile +7 -0
@@ -4,3 +4,10 @@ obj-$(CONFIG_CRYPTO_LIB_POLY1305_KUNIT_TEST) += poly1305_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA1_KUNIT_TEST) += sha1_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA256_KUNIT_TEST) += sha224_kunit.o sha256_kunit.o obj-$(CONFIG_CRYPTO_LIB_SHA512_KUNIT_TEST) += sha384_kunit.o sha512_kunit.o +obj-$(CONFIG_CRYPTO_LIB_SHA3_KUNIT_TEST) += \ + sha3_224_kunit.o \ + sha3_256_kunit.o \ + sha3_384_kunit.o \ + sha3_512_kunit.o \ + sha3_shake128_kunit.o \ + sha3_shake256_kunit.o
lib/crypto/tests/sha3_224_kunit.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_224_testvecs.h" + +#define HASH sha3_224 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHA3_224_DIGEST_SIZE +#define HASH_INIT sha3_224_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="sha3_224" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHA3-224" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_224_testvecs.h new file +231 -0
231 lines not on this page · raw message
lib/crypto/tests/sha3_256_kunit.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_256_testvecs.h" + +#define HASH sha3_256 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHA3_256_DIGEST_SIZE +#define HASH_INIT sha3_256_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="sha3_256" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHA3-256" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_256_testvecs.h new file +231 -0
231 lines not on this page · raw message
lib/crypto/tests/sha3_384_kunit.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_384_testvecs.h" + +#define HASH sha3_384 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHA3_384_DIGEST_SIZE +#define HASH_INIT sha3_384_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="sha3_384" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHA3-384" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_384_testvecs.h new file +281 -0
281 lines not on this page · raw message
lib/crypto/tests/sha3_512_kunit.c new file +32 -0
@@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_512_testvecs.h" + +#define HASH sha3_512 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHA3_512_DIGEST_SIZE +#define HASH_INIT sha3_512_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="sha3_512" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHA-512 and HMAC-SHA512" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_512_testvecs.h new file +331 -0
331 lines not on this page · raw message
lib/crypto/tests/sha3_shake128_kunit.c new file +37 -0
@@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_shake128_testvecs.h" + +static void test_shake128_128 (const u8 *in,size_t in_len,u8 *out) +{ +shake128 (in, in_len, out,SHAKE128_DIGEST_SIZE ); +} + +#define HASH test_shake128_128 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHAKE128_DIGEST_SIZE +#define HASH_INIT shake128_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="shake128" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHAKE-128 with 128-bit digest" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_shake128_testvecs.h new file +181 -0
181 lines not on this page · raw message
lib/crypto/tests/sha3_shake256_kunit.c new file +37 -0
@@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2025 Red Hat, Inc. All Rights Reserved. +* Written by David Howells ([email protected]) +*/ +#include <crypto/sha3.h> +#include "sha3_shake256_testvecs.h" + +static void test_shake256_256 (const u8 *in,size_t in_len,u8 *out) +{ +shake256 (in, in_len, out,SHAKE256_DIGEST_SIZE ); +} + +#define HASH test_shake256_256 +#define HASH_CTX sha3_ctx +#define HASH_SIZE SHAKE256_DIGEST_SIZE +#define HASH_INIT shake256_init +#define HASH_UPDATE sha3_update +#define HASH_FINAL sha3_final +#include "hash-test-template.h" + +static struct kunit_case hash_test_cases[] = { +HASH_KUNIT_CASES , +KUNIT_CASE (benchmark_hash), + {}, +}; + +static struct kunit_suite hash_test_suite = { + .name ="shake256" , + .test_cases = hash_test_cases, + .suite_init = hash_suite_init, + .suite_exit = hash_suite_exit, +}; +kunit_test_suite (hash_test_suite); + +MODULE_DESCRIPTION ("KUnit tests and benchmark for SHAKE-256 with 256-bit digest" ); +MODULE_LICENSE ("GPL" );
lib/crypto/tests/sha3_shake256_testvecs.h new file +231 -0
231 lines not on this page · raw message
scripts/crypto/gen-hash-testvecs.py +6 -2
@@ -61,6 +61,10 @@ def hash_update(ctx, data): ctx.update(data) def hash_final(ctx): + if ctx.name == "shake_128": + return ctx.digest(16) + if ctx.name == "shake_256": + return ctx.digest(32) return ctx.digest() def compute_hash(alg, data):
@@ -117,7 +121,7 @@ def gen_hmac_testvecs(alg): ctx.update(mac) print_static_u8_array_definition( f'hmac_testvec_consolidated[{alg.upper()}_DIGEST_SIZE]', - ctx.digest()) + hash_final(ctx)) def gen_additional_poly1305_testvecs(): key = b'\xff' * POLY1305_KEY_SIZE
@@ -143,5 +147,5 @@ print(f'/* This file was generated by: {sys.argv[0]} {" ".join(sys.argv[1:])} */ gen_unkeyed_testvecs(alg) if alg == 'poly1305': gen_additional_poly1305_testvecs() -else: +elif alg != 'shake128' and alg != 'shake256': gen_hmac_testvecs(alg)
Eric Biggers
maintainerQuoted 89 lines · David Howells
Add SHA3, providing SHA3-224, SHA3-256, SHA3-384, SHA-512, SHAKE128 and SHAKE256 to lib/crypto.
The state array handling is simplified from what's in
crypto/sha3_generic.cby keeping the state array (a u64[25]) in LE form and byteswapping all the entries before and after applying the keccak function on a BE system. This means no byteswapping is required when XOR'ing data into the state array or when extracting the digest. Further, this is a no-op on LE systems.Also:
- Perform a multistage shake256 hash check in the module initialisation.
- Add kunit tests for each algorithm based on the gen-hash-testvecs.
- The conflicting static s390x crypto function names are renamed to have an
s390_prefix.-
gen-hash-testvecs.pyhad to be modified to be able to generate SHAKE hashes because Python's hashlib requires the output digest size supplying for those two algorithms as they produce arbitrary length digests.Notes:
(1) I've left hooks in
sha3.cfor asm-optimised variants, but as I don't entirely know what those might look like, not having implemented any, the hooks' usability is uncertain.(2) The SHAKE algorithms will be required for ML-DSA.
Signed-off-by David Howells <[email protected]>cc Eric Biggers <[email protected]>cc Jason A. Donenfeld <[email protected]>cc Ard Biesheuvel <[email protected]>cc Harald Freudenberger <[email protected]>cc Holger Dengler <[email protected]>cc Herbert Xu <[email protected]>cc Stephan Mueller <[email protected]>---
Changes
=======
v2)
- Simplify the endianness handling.- Rename
sha3_final()tosha3_squeeze()and don't clear the context at the end as it's permitted to continue callingsha3_final()to extract continuations of the digest (needed by ML-DSA).- Don't reapply the end marker to the hash state in continuation
sha3_squeeze()unlesssha3_update()gets called again (needed by ML-DSA).- Give
sha3_squeeze()the amount of digest to produce as a parameterrather than using ctx->digest_size and don't return the amount digested.- Reimplement
sha3_final()as a wrapper aroundsha3_squeeze()that extractsctx->digest_sizeamount of digest and then zeroes out the context. The latter is necessary to avoid upsettinghash-test-template.h.- Provide a
sha3_reinit()function to clear the state, but to leave the parameters that indicate the hash properties unaffected, allowing for reuse.- Provide a
sha3_set_digestsize()function to change the size of the digest to be extracted bysha3_final().sha3_squeeze()takes a parameter for this instead.- Don't pass the digest size as a parameter to shake128/256_init() but rather default to 128/256 bits as per the function name.
- Provide a
sha3_clear()function to zero out the context.arch/s390/crypto/sha3_256_s390.c | 26 - include/crypto/sha3.h | 160 +++++++- lib/crypto/Kconfig | 7 lib/crypto/Makefile | 6 lib/crypto/sha3.c | 597 ++++++++++++++++++++++++++++++ lib/crypto/tests/Kconfig | 12 lib/crypto/tests/Makefile | 7 lib/crypto/tests/sha3_224_kunit.c | 32 + lib/crypto/tests/sha3_224_testvecs.h | 231 +++++++++++ lib/crypto/tests/sha3_256_kunit.c | 32 + lib/crypto/tests/sha3_256_testvecs.h | 231 +++++++++++ lib/crypto/tests/sha3_384_kunit.c | 32 + lib/crypto/tests/sha3_384_testvecs.h | 281 ++++++++++++++
Quoted 8 lines
lib/crypto/tests/sha3_512_kunit.c | 32 + lib/crypto/tests/sha3_512_testvecs.h | 331 ++++++++++++++++ lib/crypto/tests/sha3_shake128_kunit.c | 37 + lib/crypto/tests/sha3_shake128_testvecs.h | 181 +++++++++ lib/crypto/tests/sha3_shake256_kunit.c | 37 + lib/crypto/tests/sha3_shake256_testvecs.h | 231 +++++++++++ scripts/crypto/gen-hash-testvecs.py | 8 20 files changed, 2495 insertions(+), 16 deletions(-)
Thanks for working on this! Some preliminary comments (it will take a few days for me to find time to fully review this):
This should be based on libcrypto-next. And as with any kernel patch, it should include a base-commit so that people know where it applies to.
This should be split into three patches: (1) the arch/s390/ changes, (2) adding the library functions themselves, and (3) adding the tests.
We'll also need to integrate the existing arch-optimized SHA-3 code, and reimplement the SHA-3 crypto_shash algorithms on top of the library. Let me know whether you're planning to do that to. If not, I can do it.
In kerneldoc comments, please make it clear that lengths are measured in bytes, and that the functions can be called in any context.
The testing situation looks odd. This patch adds six KUnit test suites:
one for each of the SHA-3 algorithms. But they only include the hash-test-template.h test cases, and they don't test the unique behavior of SHAKE. The KUnit tests need to fully test the library.
I see you also have a test in sha3_mod_init(), which doesn't make sense. The tests should be in the KUnit test suite(s). If you intended for the sha3_mod_init() test to be a FIPS pre-operational self-test, then (1) it would first need to be confirmed with the people doing FIPS certifications that a FIPS pre-operational self-test is actually necessary here, (2) it would need to be fixed to actually fulfill the requirements for that type of test such as panicing the kernel on failure, and (3) it would need to come in its own patch with its own explanation. But, unless you are sure you actually need the FIPS test, just omit it out for now and focus on the real tests.
I also think that splitting the SHA-3 tests into six KUnit test suites is awkward. I know I did something similar for SHA-2, but it made more sense for SHA-2 because (1) there are only four SHA-2 variants, (2) SHA-256 and SHA-512 don't share any code, and (3) there wasn't anything more to add on top of hash-test-template.h. In contrast, SHA-3 has six variants, which all share most of their code, and there will need to be SHA-3 specific tests (for the XOFs).
I think what I'd recommend is creating a single sha3_kunit test suite. Make it instantiate hash-test-template.h once to test one of the algorithms, maybe SHA3-256. Then add test cases (that is, additional KUnit test cases in the same KUnit test suite) that cover the code specific to the other variants, including the XOFs.
- Eric
David Howells
maintainerQuoted 1 line · Eric Biggers
This should be based on libcrypto-next.
This?
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git libcrypto-next
Quoted 2 lines
This should be split into three patches: (1) the
arch/s390/changes, (2) adding the library functions themselves, and (3) adding the tests.
Sure. But I'll wait a bit to see if there are any other comments first.
Quoted 3 lines
We'll also need to integrate the existing arch-optimized SHA-3 code, and reimplement the SHA-3
crypto_shashalgorithms on top of the library. Let me know whether you're planning to do that to. If not, I can do it.
I don't really have time at the moment. Nor am I particularly familiar with the optimised asm instructions for this on any arch.
Quoted 2 lines
In kerneldoc comments, please make it clear that lengths are measured in bytes,
Sure. The hash algos do love talking in terms of bits. Possibly because bytes aren't necessarily 8 bits in size on all arches?
Quoted 1 line
and that the functions can be called in any context.
"Context" as in?
Quoted 4 lines
The testing situation looks odd. This patch adds six KUnit test suites:
one for each of the SHA-3 algorithms. But they only include thehash-test-template.htest cases, and they don't test the unique behavior of SHAKE. The KUnit tests need to fully test the library.
Yes, I'm aware of that. The hash-test-template template is rather rigid and not always correct in its assertions (for instance requiring the final function to have zeroed the context - I had to modify my API to work around the testsuite).
Quoted 10 lines
I see you also have a test in
sha3_mod_init(), which doesn't make sense. The tests should be in the KUnit test suite(s). If you intended for thesha3_mod_init()test to be a FIPS pre-operational self-test, then (1) it would first need to be confirmed with the people doing FIPS certifications that a FIPS pre-operational self-test is actually necessary here, (2) it would need to be fixed to actually fulfill the requirements for that type of test such as panicing the kernel on failure, and (3) it would need to come in its own patch with its own explanation. But, unless you are sure you actually need the FIPS test, just omit it out for now and focus on the real tests.
I disagree. It should have at least a single self-test. If we fail to load any modules because the hash is broken on a particular CPU, it would be useful to have a note in dmesg. Loading kunit test modules becomes tricky in such a case.
Quoted 7 lines
I also think that splitting the SHA-3 tests into six KUnit test suites is awkward. I know I did something similar for SHA-2, but it made more sense for SHA-2 because (1) there are only four SHA-2 variants, (2) SHA-256 and SHA-512 don't share any code, and (3) there wasn't anything more to add on top of
hash-test-template.h. In contrast, SHA-3 has six variants, which all share most of their code, and there will need to be SHA-3 specific tests (for the XOFs).
Yes, but I believe you wanted me to use hash-test-template. The problem is that it hard-encodes by macroisation of the #include's file various parameters including the hash size.
Quoted 5 lines
I think what I'd recommend is creating a single
sha3_kunittest suite. Make it instantiatehash-test-template.honce to test one of the algorithms, maybe SHA3-256. Then add test cases (that is, additional KUnit test cases in the same KUnit test suite) that cover the code specific to the other variants, including the XOFs.
I could do that. It would at least exercise the common engine.
Possibly all 5 different block sizes employed (128, 224, 256, 384 and 512) need to be so tested - but that only affects how much of the state array is modified directly sha3_update() and how much can be extracted from by sha3_squeeze(). The actual keccak mixing function doesn't care.
Other than that, the only differences between the algorithms are the padding char and how much digest is extracted by default.
On top of that, you can have a variety of different usage sequences: different sequences of updating and squeezing with different amounts of data added and extracted. I wonder if a small sampling is sufficient or whether we need some loopy thing that tries to exhaustively test a portion of the sample space.
David
Am Freitag, 19. September 2025, 21:48:00 Mitteleuropäische Sommerzeit schrieb David Howells:
Hi David,
Quoted 1 line
Quoted 1 line
I see you also have a test in
sha3_mod_init(), which doesn't make
sense.
Quoted 1 line
Quoted 1 line
The tests should be in the KUnit test suite(s). If you intended for
the
Quoted 1 line
Quoted 1 line
sha3_mod_init()test to be a FIPS pre-operational self-test, then (1)
it
Quoted 9 lines
Quoted 7 lines
would first need to be confirmed with the people doing FIPS certifications that a FIPS pre-operational self-test is actually necessary here, (2) it would need to be fixed to actually fulfill the requirements for that type of test such as panicing the kernel on failure, and (3) it would need to come in its own patch with its own explanation. But, unless you are sure you actually need the FIPS test, just omit it out for now and focus on the real tests.
I disagree. It should have at least a single self-test. If we fail to
load
Quoted 2 lines
any modules because the hash is broken on a particular CPU, it would be useful to have a note in dmesg. Loading kunit test modules becomes
tricky
Quoted 1 line
in such a case.
Just for clarifications of the FIPS requirements: One test of any of the SHA3/SHAKE algorithms during startup is sufficient for *one* Keccak implementation. FIPS wants the actual Keccak sponge being tested, it does not care for the miniscule differences between the different SHA/SHAKE definitions.
Yet, if we have multiple Keccak sponge implementations, then each needs its own self test.
Ciao
Stephan
Eric Biggers
maintainerQuoted 19 lines · Stephan Mueller
Am Freitag, 19. September 2025, 21:48:00 Mitteleuropäische Sommerzeit schrieb David Howells:
Hi David,
Quoted 1 line
Quoted 1 line
I see you also have a test in
sha3_mod_init(), which doesn't makesense.
Quoted 1 line
Quoted 1 line
The tests should be in the KUnit test suite(s). If you intended for
the
Quoted 1 line
Quoted 1 line
sha3_mod_init()test to be a FIPS pre-operational self-test, then (1)it
Quoted 8 lines
Quoted 7 lines
would first need to be confirmed with the people doing FIPS certifications that a FIPS pre-operational self-test is actually necessary here, (2) it would need to be fixed to actually fulfill the requirements for that type of test such as panicing the kernel on failure, and (3) it would need to come in its own patch with its own explanation. But, unless you are sure you actually need the FIPS test, just omit it out for now and focus on the real tests.
Quoted 6 lines
Quoted 1 line
I disagree. It should have at least a single self-test. If we fail to
load
Quoted 2 lines
any modules because the hash is broken on a particular CPU, it would be useful to have a note in dmesg. Loading kunit test modules becomes
tricky
Quoted 1 line
in such a case.
Well, testing is supposed to be done before the kernel is released, not on every end user's computer.
If you insist, at least keep it lightweight and make sure it doesn't detract from the real tests.
Quoted 5 lines
Just for clarifications of the FIPS requirements: One test of any of the SHA3/SHAKE algorithms during startup is sufficient for *one* Keccak implementation. FIPS wants the actual Keccak sponge being tested, it does not care for the miniscule differences between the different SHA/SHAKE definitions.
Yes. But I'm still a bit puzzled why there suddenly seems to be interest in a FIPS pre-operational self-test for SHA-3 specifically. lib/ has had SHA-1 for two decades without a FIPS pre-operational self-test. If someone actually needs this, surely they would also need it, and have already needed it, for other algorithms?
Quoted 2 lines
Yet, if we have multiple Keccak sponge implementations, then each needs its own self test.
While lib/crypto/ often has multiple implementations of the algorithms, only one implementation is used on a given system.
- Eric
Am Freitag, 19. September 2025, 22:47:49 Mitteleuropäische Sommerzeit schrieb Eric Biggers:
Hi Eric,
Quoted 5 lines
Yes. But I'm still a bit puzzled why there suddenly seems to be interest in a FIPS pre-operational self-test for SHA-3 specifically.
lib/has had SHA-1 for two decades without a FIPS pre-operational self-test. If someone actually needs this, surely they would also need it, and have already needed it, for other algorithms?
I just answered on the FIPS requirements as I interpreted your question in this regard. I am not saying it needs to be added here.
I am currently a bit confused between crypto vs lib/crypto with its FIPS vs non-FIPS support. That, perhaps, contributed to my answer.
Ciao
Stephan
Eric Biggers
maintainerQuoted 7 lines · David Howells
Quoted 1 line · Eric Biggers
This should be based on libcrypto-next.
This?
https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git libcrypto-next
Yes.
Quoted 3 lines
Quoted 1 line
and that the functions can be called in any context.
"Context" as in?
See the "Function context" section ofDocumentation/doc-guide/kernel-doc.rst
Quoted 6 lines
Quoted 4 lines
The testing situation looks odd. This patch adds six KUnit test suites:
one for each of the SHA-3 algorithms. But they only include thehash-test-template.htest cases, and they don't test the unique behavior of SHAKE. The KUnit tests need to fully test the library.Yes, I'm aware of that. The hash-test-template template is rather rigid
hash-test-template.h is designed for traditional hash functions. If you'd like to extend it to support XOFs, that's one option. But I think just keeping the XOF testing in sha3_kunit.c would make sense for now.
Quoted 3 lines
and not always correct in its assertions (for instance requiring the final function to have zeroed the context - I had to modify my API to work around the testsuite).
But that's the correct behavior. Callers may be hashing sensitize data, so *_final() zeroizes the context.
The "multiple squeezes" use case should use different functions.
Quoted 11 lines
Quoted 7 lines
I also think that splitting the SHA-3 tests into six KUnit test suites is awkward. I know I did something similar for SHA-2, but it made more sense for SHA-2 because (1) there are only four SHA-2 variants, (2) SHA-256 and SHA-512 don't share any code, and (3) there wasn't anything more to add on top of
hash-test-template.h. In contrast, SHA-3 has six variants, which all share most of their code, and there will need to be SHA-3 specific tests (for the XOFs).Yes, but I believe you wanted me to use hash-test-template. The problem is that it hard-encodes by macroisation of the #include's file various parameters including the hash size.
Did you miss my response at
https://lore.kernel.org/linux-crypto/20250917192829.GA8743@quark/ ?
- Eric
David Howells
maintainerQuoted 6 lines · Eric Biggers
Quoted 3 lines
Quoted 1 line
and that the functions can be called in any context.
"Context" as in?
See the "Function context" section of
Documentation/doc-guide/kernel-doc.rst
Btw, in include/crypto/sha1.h:
/**
* hmac_sha1_update() - Update an HMAC-SHA1 context with message data
* @ctx: the HMAC context to update; must have been initialized
* @data: the message data
* @data_len: the data length in bytes
*
* This can be called any number of times.
*
* Context: Any context.
*/
static inline void hmac_sha1_update(struct hmac_sha1_ctx *ctx,
const u8 *data, size_t data_len)
{
sha1_update(&ctx->sha_ctx, data, data_len);
}for example, your specification of "Context: Any context." is probably not correct if FPU/Vector registers are used by optimised assembly as part of the function. See:
void kernel_fpu_begin_mask(unsigned int kfpu_mask)
{
if (!irqs_disabled())
fpregs_lock();
WARN_ON_FPU(!irq_fpu_usable());
/* Toggle kernel_fpu_allowed to false: */
WARN_ON_FPU(!this_cpu_read(kernel_fpu_allowed));
this_cpu_write(kernel_fpu_allowed, false);
if (!(current->flags & (PF_KTHREAD | PF_USER_WORKER)) &&
!test_thread_flag(TIF_NEED_FPU_LOAD)) {
set_thread_flag(TIF_NEED_FPU_LOAD);
save_fpregs_to_fpstate(x86_task_fpu(current));
}
__cpu_invalidate_fpregs_state();
/* Put sane initial values into the control registers. */
if (likely(kfpu_mask & KFPU_MXCSR) && boot_cpu_has(X86_FEATURE_XMM))
ldmxcsr(MXCSR_DEFAULT);
if (unlikely(kfpu_mask & KFPU_387) && boot_cpu_has(X86_FEATURE_FPU))
asm volatile ("fninit");
}If you try and access the function in IRQ mode, for example, you'll get a warning, and if IRQs are not disabled, it will disable BH/preemption.
You also can't use it from inside something else that uses FPU registers.
I suggest something like:
* Context: Arch-dependent: May use the FPU/Vector unit registers.
David.
Eric Biggers
maintainerQuoted 57 lines · David Howells
Quoted 6 lines · Eric Biggers
Quoted 3 lines
Quoted 1 line
and that the functions can be called in any context.
"Context" as in?
See the "Function context" section of
Documentation/doc-guide/kernel-doc.rstBtw, in
include/crypto/sha1.h:/**
*hmac_sha1_update()- Update an HMAC-SHA1 context with message data
* @ctx: the HMAC context to update; must have been initialized
* @data: the message data
* @data_len: the data length in bytes
*
* This can be called any number of times.
*
* Context: Any context.
*/
static inline void hmac_sha1_update(structhmac_sha1_ctx*ctx,
const u8 *data,size_tdata_len){ sha1_update(&ctx->sha_ctx, data, data_len); }for example, your specification of "Context: Any context." is probably not correct if FPU/Vector registers are used by optimised assembly as part of the function. See:
void
kernel_fpu_begin_mask(unsigned int kfpu_mask){ if (!irqs_disabled()) fpregs_lock(); WARN_ON_FPU(!irq_fpu_usable()); /* Toggle kernel_fpu_allowed to false: */ WARN_ON_FPU(!this_cpu_read(kernel_fpu_allowed)); this_cpu_write(kernel_fpu_allowed, false); if (!(current->flags & (PF_KTHREAD | PF_USER_WORKER)) && !test_thread_flag(TIF_NEED_FPU_LOAD)) { set_thread_flag(TIF_NEED_FPU_LOAD); save_fpregs_to_fpstate(x86_task_fpu(current)); } __cpu_invalidate_fpregs_state(); /* Put sane initial values into the control registers. */ if (likely(kfpu_mask & KFPU_MXCSR) && boot_cpu_has(X86_FEATURE_XMM)) ldmxcsr(MXCSR_DEFAULT); if (unlikely(kfpu_mask & KFPU_387) && boot_cpu_has(X86_FEATURE_FPU)) asm volatile ("fninit"); }
Quoted 8 lines
If you try and access the function in IRQ mode, for example, you'll get a warning, and if IRQs are not disabled, it will disable BH/preemption.
You also can't use it from inside something else that uses FPU registers.
I suggest something like:
* Context: Arch-dependent: May use the FPU/Vector unit registers.
Kernel-mode FPU is used only when irq_fpu_usable().
The tests verify that the functions do work in IRQ context.
- Eric
kernel test robot
build botkernel test robot noticed the following build errors:
Hi David,
kernel test robot noticed the following build errors:
[auto build test ERROR on ebiggers/libcrypto-fixes]
[also build test ERROR on herbert-cryptodev-2.6/master herbert-crypto-2.6/master linus/master v6.17-rc6 next-20250919] [cannot apply to ebiggers/libcrypto-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/David-Howells/lib-crypto-Add-SHA3-224-SHA3-256-SHA3-384-SHA-512-SHAKE128-SHAKE256/20250920-003544 base: https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux.git libcrypto-fixes patch link: https://lore.kernel.org/r/3936580.1758299519%40warthog.procyon.org.uk patch subject: [PATCH v2] lib/crypto: Add SHA3-224, SHA3-256, SHA3-384, SHA-512, SHAKE128, SHAKE256 config: sparc-defconfig (https://download.01.org/0day-ci/archive/20250920/[email protected]/config) compiler: sparc-linux-gcc (GCC) 15.1.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
sparc-linux-ld: lib/crypto/sha3.o: in function `sha3_keccakf_generic':
>> sha3.c:(.text+0x964): undefined reference to `__bswapdi2' >> sparc-linux-ld: sha3.c:(.text+0x9c0): undefined reference to `__bswapdi2'
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Eric Biggers
maintainerSome more comments:
Quoted 5 lines · David Howells
lib/crypto/sha3.c new file +5
-0+#if 0 +static void sha3_dump_state (const struct sha3_ctx *ctx,const char *prefix) +{ +const __be64 *p = (const __be64 *)ctx->state.st; +
lib/crypto/sha3.c line 34
Quoted 10 lines
lib/crypto/sha3.c new file +10
-0+for (int i =0 ; i <6 ; i++) { +pr_info ("SHA3 %s %016llx %016llx %016llx %016llx\n" , + prefix, +be64_to_cpu (p[0 ]),be64_to_cpu (p[1 ]), +be64_to_cpu (p[2 ]),be64_to_cpu (p[3 ])); + p +=4 ; + } + pr_info("SHA3 %s %016llx\n", prefix, be64_to_cpu(p[0])); +} +#endif
Unused code should not be added.
Quoted 8 lines
+/*
+ * On some 32-bit architectures (h8300), GCC ends up using
+ * over 1 KB of stack if we inline the round calculation into the loop
+ * inkeccakf(). On the other hand, on 64-bit architectures with plenty
+ * of [64-bit wide] general purpose registers, not inlining it severely
+ * hurts performance. So let's use 64-bitness as a heuristic to decide
+ * whether to inline or not.
+ */
The above comment doesn't match the code. "keccakf()" should be sha3_keccakf_generic().
lib/crypto/sha3.c line 65
Quoted 3 lines
lib/crypto/sha3.c new file +3
-0+/* update the state with given number of rounds */ + +static SHA3_INLINE void keccakf_round(u64 st[25])
The above comment doesn't match the code. keccakf_round() does only one round.
Also, the parameter should be strongly typed: 'struct sha3_state *' Likewise in all the other functions that take the raw u64 array.
lib/crypto/sha3.c line 174
Quoted 7 lines
lib/crypto/sha3.c new file +7
-0+static void sha3_be_swap_state (u64 st[25 ]) +{ +#ifdef __BIG_ENDIAN +for (int i =0 ; i <25 ; i++) + st[i] =__builtin_bswap64 (st[i]); +#endif +}
As noted by the kernel test robot, this doesn't compile on all architectures. How about:
for (size_t i = 0; i < ARRAY_SIZE(state->st); i++) cpu_to_le64s(&state->st[i]);
There also should be a comment somewhere that describes what's going on with the state endianness. I.e., on big endian CPUs we keep the state words as little endian to make XOR'ing data into them easier, and we swap them to native endianness temporarily when doing the permutation.
Quoted 4 lines
lib/crypto/sha3.c new file +4
-0+/* +* Perform the mixing step. +*/ +static void sha3_keccakf_generic (u64 st[25 ])
lib/crypto/sha3.c line 189
Quoted 10 lines
lib/crypto/sha3.c new file +10
-0+{ +int round; + +sha3_be_swap_state (st); + +for (round =0 ; round <KECCAK_ROUNDS ; round++) { +keccakf_round (st); +/* Iota */ + st[0 ] ^= keccakf_rndc[round]; + }
In the spec, "Iota" is part of the round. Having it be separate from keccakf_round() in the code is confusing.
Quoted 5 lines
+/**
+ *sha3_update()- Update a SHA3 context of any type with message data
+ * @ctx: the context to update; must have been initialized
+ * @data: the message data
+ * @len: the data length in bytes
Quoted 8 lines
+ *
+ * This can be called any number of times to perform the "keccak sponge
+ * absorbing" phase.
+ *
+ * It may be called again after @sha3_final() has been called to add more data
+ * to the hash. Note, however, that finalising a hash modifies the state, so
+ * {update,final,update} is not equivalent to {update,update}.
+ */
Would you mind putting all the kerneldoc comments for the public API in the .h file, similar to sha1.h and sha2.h? Then they'll all be in a consistent place, and they won't have to be moved around each time we change functions to inline functions or vice versa.
Quoted 11 lines
lib/crypto/sha3.c new file +11
-0+void sha3_update (struct sha3_ctx *ctx,const u8 *data,unsigned int len) +{ +unsigned int partial = ctx->partial; +unsigned int bsize = ctx->block_size; + +if (partial && partial + len >= bsize) { +sha3_absorb_xorle (ctx, data, partial, bsize - partial); + len -= bsize - partial; + data += bsize - partial; +sha3_keccakf (ctx->state.st); + ctx->partial =0 ;
lib/crypto/sha3.c line 270
Quoted 10 lines
lib/crypto/sha3.c new file +10
-0+ } + +if (len >= bsize) { +size_t nblocks = len / bsize; + +if (nblocks) { +sha3_absorb_blocks (ctx, data, nblocks); + data += nblocks * bsize; + len -= nblocks * bsize; + }
The 'if (nblocks)' check above is unnecessary, since 'len >= bsize' implies there is at least one block.
Quoted 9 lines
+/**
+ *sha3_squeeze()- Finalize a SHA3 digest of any type and extract the digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: Where to write the resulting message digest
+ * @out_size: The amount of digest to extract to @out
+ *
+ * Finish the computation of a SHA3 message digest of any type and perform the
+ * "Keccak sponge squeezing" phase. @out_size amount of digest is written to
+ * @out buffer.
Quoted 8 lines
+ *
+ * This may be called multiple times to extract continuations of the digest.
+ * Note that, for example, two consecutive 16-byte squeezes laid end-to-end
+ * will yield the same as one 32-byte squeeze.
+ *
+ * The state will have the end marker added again if any new updates have
+ * happened since the last time it was squeezed.
+ */
First, this patch's proposed API is error-prone due to the weak typing that allows mixing steps of different algorithms together. For example, users could initialize a sha3_ctx with sha3_256_init() and then squeeze an arbitrary amount from it, incorrectly treating it as a XOF. It would be worth considering separating the APIs for the different algorithms that are part of SHA-3, similar to what I did with SHA-224 and SHA-256. (They would of course still share code internally, just like SHA-2.)
Second, the support for update() + squeeze() + update() + squeeze() seems to be trying to achieve something that is not defined in the SHA-3 spec. Could you elaborate on what it is meant to be doing, and why it's here? According to the spec, the XOFs SHAKE128 and SHAKE256 actually just take a single message as their input.
One might think the above sequence would compute SHAKE*() on the first message as well as SHAKE*() of the two messages concatenated together. But that's not what you've made it do, as you've made it apply padding after *both* messages. So, then one might think that it's meant to be a XOF on the sequence of messages, built on top of SHAKE*(). But it's not that either, since the way that you've proposed to format the sequence of messages into a longer message isn't injective. So, I can't figure out why you're supporting this usage and what it is meant to be doing.
lib/crypto/sha3.c line 301
Quoted 4 lines
lib/crypto/sha3.c new file +4
-0+void sha3_squeeze (struct sha3_ctx *ctx,u8 *out,size_t out_size) +{ +unsigned int squeeze_offset = ctx->squeeze_offset; +unsigned int digest_size = out_size;
The digest_size variable is unnecessary, and it's also misleading since this function is used both for the digests and the XOFs. Just use the already-existing 'out_size' variable.
Quoted 16 lines
lib/crypto/sha3.c new file +16
-0+unsigned int bsize = ctx->block_size; +u8 *p = (u8 *)ctx->state.st, end_marker =0x80 ; + +if (!ctx->end_marked) { +sha3_absorb_xorle (ctx, &ctx->padding, ctx->partial,1 ); +sha3_absorb_xorle (ctx, &end_marker, bsize -1 ,1 ); + ctx->end_marked = true; + } + +for (;;) { +if (squeeze_offset ==0 ) { +sha3_keccakf (ctx->state.st); + } + +unsigned int part =umin (digest_size, bsize - squeeze_offset); +
lib/crypto/sha3.c line 327
Quoted 10 lines
lib/crypto/sha3.c new file +10
-0+if (part >0 ) { +memcpy (out, p + squeeze_offset, part); + digest_size -= part; + out += part; + squeeze_offset += part; + } +if (!digest_size) +break ; +if (squeeze_offset >= bsize) + squeeze_offset -= bsize;
The last two lines could just set squeeze_offset to 0 unconditionally, given that the block must have been filled.
lib/crypto/sha3.c line 344
Quoted 11 lines
lib/crypto/sha3.c new file +11
-0+/** +* sha3_init() - Initialize a SHA3 context for a new message +* @ctx: the context to initialize +* +* Initialize a SHA3 context for any size of SHA-3 digest. +*/ +void sha3_init (struct sha3_ctx *ctx) +{ +memset (ctx,0 ,sizeof (*ctx)); +} +EXPORT_SYMBOL_GPL (sha3_init);
Here's another place where the proposed API invites misuse. If someone calls sha3_init() followed by sha3_update(), which would be a very intuitive thing to do, there will be an infinite loop, because the block_size wouldn't have been initialized. sha3_init() is actually an internal implementation detail of the real initialization functions.
I recommend just removing it, and making the real initialization functions use designated initializers to assign to the whole sha3_ctx.
Quoted 7 lines
+/**
+ *shake128()- Convenience wrapper to digest a simple buffer as SHAKE128
+ * @in: The data to be digested
+ * @in_len: The amount of data to be digested
+ * @out: The buffer into which the digest will be stored
+ * @out_len: The size of the digest desired (variable length)
+ */
Shouldn't the term "digest" be avoided when talking about the output from a XOF, to avoid confusion with traditional hash functions?
Quoted 99 lines
lib/crypto/sha3.c new file +99
-0+/* +* Do a quick test using SHAKE256 and a 200 byte digest. +*/ +static const u8 sha3_sample1[]__initconst = +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" ; +static const u8 sha3_sample2[]__initconst = +"hello\n" ; +static const u8 sha3_sample_shake256_200_step1[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0xab ,0x06 ,0xd4 ,0xf9 ,0x8b ,0xfd ,0xb2 ,0xc4 ,0xfe ,0xf1 ,0xcc ,0xe2 , +0x40 ,0x45 ,0xdd ,0x15 ,0xcb ,0xdd ,0x02 ,0x8d ,0xb7 ,0x9f ,0x1e ,0x67 , +0xd6 ,0x7f ,0x98 ,0x5e ,0x1b ,0x19 ,0xf8 ,0x01 ,0x43 ,0x82 ,0xcb ,0xd8 , +0x5d ,0x21 ,0x64 ,0xa8 ,0x80 ,0xc9 ,0x22 ,0xe5 ,0x07 ,0xaf ,0xe2 ,0x5d , +0xcd ,0xc6 ,0x23 ,0x36 ,0x2b ,0xc7 ,0xc7 ,0x7d ,0x09 ,0x9d ,0x68 ,0x05 , +0xe4 ,0x62 ,0x63 ,0x1b ,0x67 ,0xbc ,0xf8 ,0x95 ,0x07 ,0xd2 ,0xe4 ,0xd0 , +0xba ,0xa2 ,0x67 ,0xf5 ,0xe3 ,0x15 ,0xbc ,0x85 ,0xa1 ,0x50 ,0xd6 ,0x6f , +0x6f ,0xd4 ,0x54 ,0x4c ,0x3f ,0x4f ,0xe5 ,0x1f ,0xb7 ,0x00 ,0x27 ,0xfc , +0x15 ,0x33 ,0xc2 ,0xf9 ,0xb3 ,0x4b ,0x9e ,0x81 ,0xe5 ,0x96 ,0xbe ,0x05 , +0x6c ,0xac ,0xf9 ,0x9f ,0x65 ,0x36 ,0xbb ,0x11 ,0x47 ,0x6d ,0xf6 ,0x8f , +0x9f ,0xa2 ,0x77 ,0x37 ,0x3b ,0x18 ,0x77 ,0xcf ,0x65 ,0xc5 ,0xa1 ,0x7e , +0x2c ,0x0e ,0x71 ,0xf0 ,0x4d ,0x18 ,0x67 ,0xb9 ,0xc4 ,0x8c ,0x64 ,0x3b , +0x4b ,0x45 ,0xea ,0x16 ,0xb2 ,0x4a ,0xc5 ,0xf5 ,0x85 ,0xdc ,0xd2 ,0xd9 , +0x13 ,0x77 ,0xb3 ,0x19 ,0xd9 ,0x8c ,0x9f ,0x28 ,0xe7 ,0x64 ,0x91 ,0x0f , +0x6f ,0x32 ,0xbf ,0xa8 ,0xa8 ,0xa3 ,0xff ,0x99 ,0x0e ,0x0b ,0x62 ,0x50 , +0xf8 ,0x3a ,0xc2 ,0xf5 ,0x98 ,0x21 ,0xeb ,0x9d ,0xe8 ,0x45 ,0xf4 ,0x46 , +0x1e ,0x8b ,0xbd ,0x10 ,0x59 ,0x2c ,0x87 ,0xe2 , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; +static const u8 sha3_sample_shake256_200_step2[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x5b ,0x43 ,0x98 ,0x3a ,0x4a ,0x5f ,0xa9 ,0x14 ,0xa5 ,0x98 ,0x26 ,0xe2 , +0xed ,0x45 ,0x6a ,0x1d ,0x61 ,0x24 ,0xf5 ,0x0c ,0xef ,0xda ,0xc2 ,0x8a , +0x30 ,0x0e ,0x03 ,0xe5 ,0x67 ,0xdd ,0x7e ,0x9f ,0xa0 ,0xa4 ,0x07 ,0x63 , +0xdc ,0x6b ,0x7e ,0xbd ,0xd7 ,0x7d ,0x7a ,0x6d ,0x55 ,0x03 ,0x02 ,0x18 , +0x12 ,0x5d ,0xf9 ,0x21 ,0xc8 ,0x78 ,0x69 ,0x7c ,0x64 ,0x39 ,0xfd ,0xf4 , +0xd6 ,0x06 ,0xe6 ,0xd8 ,0x6f ,0xaa ,0x04 ,0x5b ,0x40 ,0xf3 ,0x96 ,0xb2 , +0xb5 ,0xd0 ,0xb5 ,0x43 ,0x50 ,0x9c ,0x08 ,0xd6 ,0x54 ,0x8e ,0x8c ,0x85 , +0xc2 ,0x34 ,0xce ,0x0c ,0x24 ,0x31 ,0x6f ,0x49 ,0xec ,0x3d ,0x13 ,0x1f , +0x36 ,0x0a ,0x14 ,0xa6 ,0x5d ,0x51 ,0x9a ,0x90 ,0x1f ,0xf5 ,0x1f ,0x61 , +0xb7 ,0x65 ,0x64 ,0x2a ,0x00 ,0x07 ,0xe4 ,0x56 ,0x80 ,0x5c ,0xfa ,0x03 , +0xc4 ,0x97 ,0xc1 ,0x09 ,0x35 ,0xa2 ,0x55 ,0x72 ,0x28 ,0xe5 ,0xb6 ,0xef , +0x8e ,0xf4 ,0xc2 ,0x82 ,0x22 ,0xc7 ,0x23 ,0xac ,0xcb ,0xc1 ,0x03 ,0x52 , +0x46 ,0x9c ,0x17 ,0xe0 ,0xa3 ,0x1b ,0x59 ,0x9f ,0x01 ,0xef ,0x5b ,0x46 , +0xb2 ,0x4b ,0x98 ,0x6b ,0x32 ,0x52 ,0xe3 ,0x29 ,0x36 ,0x8f ,0x66 ,0x98 , +0x5f ,0x6a ,0xa2 ,0xf4 ,0x68 ,0x13 ,0x5c ,0x94 ,0xe4 ,0x22 ,0xb6 ,0x83 , +0xa0 ,0xd7 ,0xa3 ,0xda ,0xa4 ,0x84 ,0x0c ,0xf6 ,0xa2 ,0xa4 ,0x0e ,0x08 , +0x6d ,0x2b ,0xd2 ,0x31 ,0x77 ,0x36 ,0xae ,0x53 , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; + +static const u8 sha3_sample_shake256_200_step3[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x61 ,0x0a ,0x5e ,0xdf ,0xf1 ,0x29 ,0xae ,0x82 ,0xab ,0x57 ,0xa8 ,0x1b , +0x4e ,0x7c ,0xb9 ,0x14 ,0x4a ,0x32 ,0x7d ,0x82 ,0xdc ,0xc2 ,0x48 ,0x1c , +0xf4 ,0xa4 ,0x18 ,0xd5 ,0x97 ,0x35 ,0x9a ,0x25 ,0x0f ,0x5f ,0x1b ,0x04 , +0xf1 ,0x09 ,0x2a ,0xe8 ,0xb6 ,0xa7 ,0xe1 ,0x90 ,0xb6 ,0x4d ,0x96 ,0xf1 , +0x7d ,0x4d ,0xb0 ,0x4f ,0x44 ,0xaf ,0x16 ,0x4e ,0x63 ,0xce ,0x46 ,0x4c , +0x76 ,0x18 ,0xbe ,0x5f ,0xf4 ,0x35 ,0xef ,0x1f ,0xb1 ,0x97 ,0x94 ,0x70 , +0x96 ,0x2f ,0xa2 ,0x1b ,0xd6 ,0x02 ,0x51 ,0x88 ,0x33 ,0x2b ,0x54 ,0xb9 , +0x44 ,0xb4 ,0xab ,0x6f ,0xeb ,0xfc ,0xe5 ,0xee ,0xe3 ,0x77 ,0x91 ,0xed , +0x3a ,0x4e ,0x60 ,0x00 ,0x44 ,0xd1 ,0xc7 ,0x4a ,0x54 ,0x77 ,0x71 ,0x95 , +0x53 ,0x88 ,0x6b ,0x1e ,0x0f ,0xfd ,0x62 ,0x02 ,0xa7 ,0x8e ,0x05 ,0x6d , +0x21 ,0x8f ,0x97 ,0x20 ,0xa0 ,0xd7 ,0xcf ,0xd8 ,0x54 ,0xec ,0x50 ,0x72 , +0x07 ,0xb8 ,0x9c ,0x76 ,0xdb ,0x12 ,0x00 ,0xd2 ,0x2e ,0x93 ,0xee ,0xb9 , +0x6a ,0x28 ,0x5a ,0x46 ,0x87 ,0x90 ,0xd5 ,0xd6 ,0x1d ,0x14 ,0x0e ,0x16 , +0xf1 ,0x2c ,0xed ,0x7f ,0x28 ,0x34 ,0x8c ,0x2b ,0x96 ,0x03 ,0x80 ,0x80 , +0x9f ,0xc8 ,0xf4 ,0x2c ,0x53 ,0xe0 ,0x4b ,0x7b ,0xf4 ,0x19 ,0x8a ,0xc5 , +0xb3 ,0x21 ,0x17 ,0xce ,0xdb ,0xbf ,0xb7 ,0x6b ,0x9a ,0xb5 ,0x19 ,0x89 , +0x4c ,0x54 ,0x28 ,0x32 ,0xe6 ,0x85 ,0xfa ,0x8f , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; +static const u8 sha3_sample_shake256_200_step4[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x83 ,0x99 ,0xc6 ,0xda ,0x75 ,0x79 ,0x8e ,0x47 ,0x06 ,0xad ,0x19 ,0xcb , +0x47 ,0x61 ,0x25 ,0x6a ,0x8c ,0xa4 ,0x7e ,0x74 ,0xba ,0xbd ,0xda ,0xb6 , +0x3b ,0x08 ,0x48 ,0x0f ,0x55 ,0xd5 ,0x85 ,0x78 ,0x5b ,0xd4 ,0x31 ,0xcb , +0x59 ,0xff ,0x93 ,0xff ,0xf6 ,0x65 ,0x4c ,0xf7 ,0x6e ,0x4b ,0xef ,0x4d , +0x0e ,0x43 ,0x8a ,0x2b ,0xed ,0x10 ,0x26 ,0x68 ,0x12 ,0x63 ,0xed ,0x7a , +0x38 ,0x0a ,0xa5 ,0xd0 ,0x79 ,0x26 ,0x75 ,0xef ,0xce ,0xad ,0x6c ,0x12 , +0x52 ,0x33 ,0xec ,0xe8 ,0xe1 ,0x89 ,0x2f ,0x0f ,0x29 ,0xb0 ,0xf6 ,0xff , +0x54 ,0x11 ,0xb2 ,0x6b ,0x22 ,0xb3 ,0x48 ,0x01 ,0xa5 ,0xcf ,0x29 ,0xb7 , +0xaf ,0x8c ,0xec ,0x1e ,0x75 ,0x3e ,0xff ,0xfb ,0x31 ,0xb8 ,0xf6 ,0xab , +0xae ,0xac ,0xec ,0xed ,0x27 ,0x0b ,0x79 ,0x10 ,0x4f ,0x87 ,0xe8 ,0x43 , +0x28 ,0x94 ,0x09 ,0xca ,0x48 ,0x63 ,0x65 ,0x61 ,0x86 ,0x83 ,0x33 ,0x30 , +0x02 ,0x6d ,0xf4 ,0xef ,0x3c ,0x1a ,0x47 ,0x8a ,0x25 ,0x90 ,0x31 ,0x39 , +0x95 ,0x1d ,0x6f ,0x11 ,0x5c ,0x0c ,0x72 ,0xe6 ,0x1b ,0xe1 ,0x60 ,0x45 , +0x79 ,0x89 ,0x39 ,0x48 ,0x31 ,0x4c ,0xc4 ,0xd1 ,0x08 ,0x12 ,0xf3 ,0x5f , +0x84 ,0x8c ,0x86 ,0xba ,0xe5 ,0xf1 ,0x24 ,0x61 ,0x2f ,0xef ,0x17 ,0x16 , +0x4a ,0x29 ,0xc0 ,0xc6 ,0x38 ,0x47 ,0x3a ,0x11 ,0xc5 ,0x7d ,0x62 ,0x85 , +0x9b ,0x18 ,0x92 ,0x4c ,0x12 ,0x92 ,0x9c ,0x0b , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; + +static void __init sha3_check_digest (const u8 *digest,const u8 *sample_digest, +int digest_len) +{
lib/crypto/sha3.c line 554
Quoted 10 lines
lib/crypto/sha3.c new file +10
-0+if (memcmp (digest, sample_digest, digest_len) !=0 ) { +pr_err ("SHAKE256(200) failed\n" ); +for (int i =0 ; i < digest_len;) { +int part =min (digest_len - i,32 ); + +pr_err ("%*phN\n" , part, digest + i); + i += part; + } + } +}
If a test is included at all here, it should be simpler, e.g. just compute the SHA3-256 of some short hardcoded message. That will be sufficient to test the implementation of the Keccak permutation. The real tests will be in the KUnit suite, as discussed previously.
- Eric
David Howells
maintainerQuoted 6 lines · Eric Biggers
lib/crypto/sha3.c line 65
Quoted 3 lines
lib/crypto/sha3.c new file +3
-0+/* update the state with given number of rounds */ + +static SHA3_INLINE void keccakf_round(u64 st[25])The above comment doesn't match the code.
keccakf_round()does only one round.
This is exactly as in crypto/sha3_generic.c. You may want to fix that also.
Quoted 11 lines
lib/crypto/sha3.c line 174
Quoted 7 lines
lib/crypto/sha3.c new file +7
-0+static void sha3_be_swap_state (u64 st[25 ]) +{ +#ifdef __BIG_ENDIAN +for (int i =0 ; i <25 ; i++) + st[i] =__builtin_bswap64 (st[i]); +#endif +}As noted by the kernel test robot, this doesn't compile on all architectures. How about:
Quoted 2 lines
for (size_t i = 0; i < ARRAY_SIZE(state->st); i++) cpu_to_le64s(&state->st[i]);
Um. Maybe. If I'm going to do that I should probably do:
for (size_t i = 0; i < ARRAY_SIZE(state->st); i++)
le64_to_cpus(&state->st[i]);
on the way in and:
for (size_t i = 0; i < ARRAY_SIZE(state->st); i++)
cpu_to_le64s(&state->st[i]);
on the way out. You never know if it's PDP-11 Linux, right? ;-)
Quoted 9 lines
Quoted 9 lines
lib/crypto/sha3.c new file +9
-0+/* +* Perform the mixing step. +*/ +static void sha3_keccakf_generic (u64 st[25 ]) +{ +int round; + +sha3_be_swap_state (st); +
Quoted 8 lines
lib/crypto/sha3.c line 189
Quoted 5 lines
lib/crypto/sha3.c new file +5
-0+for (round =0 ; round <KECCAK_ROUNDS ; round++) { +keccakf_round (st); +/* Iota */ + st[0 ] ^= keccakf_rndc[round]; + }In the spec, "Iota" is part of the round. Having it be separate from
keccakf_round()in the code is confusing.
It's as crypto/sha3_generic.c.
Quoted 4 lines
Would you mind putting all the kerneldoc comments for the public API in the .h file, similar to
sha1.handsha2.h? Then they'll all be in a consistent place, and they won't have to be moved around each time we change functions to inline functions or vice versa.
Yes. I know include/crypto/ does it, but that really makes the header files suck as the doc-to-code ratio is too heavily in the doc's favour.
Actually, this should be done properly and I'll add an API doc to Documention/crypto/.
Quoted 11 lines
lib/crypto/sha3.c line 270
Quoted 8 lines
lib/crypto/sha3.c new file +8
-0+if (len >= bsize) { +size_t nblocks = len / bsize; + +if (nblocks) { +sha3_absorb_blocks (ctx, data, nblocks); + data += nblocks * bsize; + len -= nblocks * bsize; + }The 'if (nblocks)' check above is unnecessary, since 'len >= bsize' implies there is at least one block.
Fair point.
Quoted 18 lines
Quoted 9 lines
+/**
+ *sha3_squeeze()- Finalize a SHA3 digest of any type and extract the digest
+ * @ctx: the context to finalize; must have been initialized
+ * @out: Where to write the resulting message digest
+ * @out_size: The amount of digest to extract to @out
+ *
+ * Finish the computation of a SHA3 message digest of any type and perform the
+ * "Keccak sponge squeezing" phase. @out_size amount of digest is written to
+ * @out buffer.Quoted 8 lines
+ *
+ * This may be called multiple times to extract continuations of the digest.
+ * Note that, for example, two consecutive 16-byte squeezes laid end-to-end
+ * will yield the same as one 32-byte squeeze.
+ *
+ * The state will have the end marker added again if any new updates have
+ * happened since the last time it was squeezed.
+ */
Quoted 7 lines
First, this patch's proposed API is error-prone due to the weak typing that allows mixing steps of different algorithms together. For example, users could initialize a
sha3_ctxwithsha3_256_init()and then squeeze an arbitrary amount from it, incorrectly treating it as a XOF. It would be worth considering separating the APIs for the different algorithms that are part of SHA-3, similar to what I did with SHA-224 and SHA-256. (They would of course still share code internally, just like SHA-2.)
I disagree. You're adding excessive complexity. I would be more inclined to agree if the thing might crash because you used the wrong thing. That said, sha3_final() uses the digestsize set by sha3_whatever_init(), which could cause a problem, so I think it's better on balance to just add an extra parameter to sha3_final() to say how much digest you want and ditch ctx->digest_size.
Quoted 6 lines
Second, the support for
update()+squeeze()+update()+squeeze()seems to be trying to achieve something that is not defined in the SHA-3 spec. Could you elaborate on what it is meant to be doing, and why it's here? According to the spec, the XOFs SHAKE128 and SHAKE256 actually just take a single message as their input.
Quoted 8 lines
One might think the above sequence would compute SHAKE*() on the first message as well as SHAKE*() of the two messages concatenated together. But that's not what you've made it do, as you've made it apply padding after *both* messages. So, then one might think that it's meant to be a XOF on the sequence of messages, built on top of SHAKE*(). But it's not that either, since the way that you've proposed to format the sequence of messages into a longer message isn't injective. So, I can't figure out why you're supporting this usage and what it is meant to be doing.
I can't speak to that except to say that ML-DSA does exactly this as far as I can tell.
Quoted 8 lines
lib/crypto/sha3.c line 301
Quoted 4 lines
lib/crypto/sha3.c new file +4
-0+void sha3_squeeze (struct sha3_ctx *ctx,u8 *out,size_t out_size) +{ +unsigned int squeeze_offset = ctx->squeeze_offset; +unsigned int digest_size = out_size;The
digest_sizevariable is unnecessary, and it's also misleading since this function is used both for the digests and the XOFs. Just use the already-existing 'out_size' variable.
Fair point. I was setting digest_size to ctx->cigest_size if out_size was 0 and then returning it, but I'm no longer doing that.
Quoted 21 lines
Quoted 21 lines
lib/crypto/sha3.c new file +21
-0+unsigned int bsize = ctx->block_size; +u8 *p = (u8 *)ctx->state.st, end_marker =0x80 ; + +if (!ctx->end_marked) { +sha3_absorb_xorle (ctx, &ctx->padding, ctx->partial,1 ); +sha3_absorb_xorle (ctx, &end_marker, bsize -1 ,1 ); + ctx->end_marked = true; + } + +for (;;) { +if (squeeze_offset ==0 ) { +sha3_keccakf (ctx->state.st); + } + +unsigned int part =umin (digest_size, bsize - squeeze_offset); + +if (part >0 ) { +memcpy (out, p + squeeze_offset, part); + digest_size -= part; + out += part; + squeeze_offset += part;
Quoted 8 lines
lib/crypto/sha3.c line 327
Quoted 5 lines
lib/crypto/sha3.c new file +5
-0+ } +if (!digest_size) +break ; +if (squeeze_offset >= bsize) + squeeze_offset -= bsize;The last two lines could just set
squeeze_offsetto 0 unconditionally, given that the block must have been filled.
Also a good point. I'm sure I tried that at some point, but it didn't work. I can't see why that should be the case though, and it should be an error if squeeze_offset > bsize.
Quoted 18 lines
lib/crypto/sha3.c line 344
Quoted 11 lines
lib/crypto/sha3.c new file +11
-0+/** +* sha3_init() - Initialize a SHA3 context for a new message +* @ctx: the context to initialize +* +* Initialize a SHA3 context for any size of SHA-3 digest. +*/ +void sha3_init (struct sha3_ctx *ctx) +{ +memset (ctx,0 ,sizeof (*ctx)); +} +EXPORT_SYMBOL_GPL (sha3_init);Here's another place where the proposed API invites misuse. If someone calls
sha3_init()followed bysha3_update(), which would be a very intuitive thing to do, there will be an infinite loop, because theblock_sizewouldn't have been initialized.sha3_init()is actually an internal implementation detail of the real initialization functions.
Quoted 2 lines
I recommend just removing it, and making the real initialization functions use designated initializers to assign to the whole
sha3_ctx.
Okay, that's reasonable.
Quoted 10 lines
Quoted 7 lines
+/**
+ *shake128()- Convenience wrapper to digest a simple buffer as SHAKE128
+ * @in: The data to be digested
+ * @in_len: The amount of data to be digested
+ * @out: The buffer into which the digest will be stored
+ * @out_len: The size of the digest desired (variable length)
+ */Shouldn't the term "digest" be avoided when talking about the output from a XOF, to avoid confusion with traditional hash functions?
No idea. There's not any real programmatic difference except for the padding byte. I can change it to "output" if it makes you happy.
Quoted 106 lines
Quoted 106 lines
lib/crypto/sha3.c new file +106
-0+/* +* Do a quick test using SHAKE256 and a 200 byte digest. +*/ +static const u8 sha3_sample1[]__initconst = +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" +"The quick red fox jumped over the lazy brown dog!\n" ; +static const u8 sha3_sample2[]__initconst = +"hello\n" ; +static const u8 sha3_sample_shake256_200_step1[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0xab ,0x06 ,0xd4 ,0xf9 ,0x8b ,0xfd ,0xb2 ,0xc4 ,0xfe ,0xf1 ,0xcc ,0xe2 , +0x40 ,0x45 ,0xdd ,0x15 ,0xcb ,0xdd ,0x02 ,0x8d ,0xb7 ,0x9f ,0x1e ,0x67 , +0xd6 ,0x7f ,0x98 ,0x5e ,0x1b ,0x19 ,0xf8 ,0x01 ,0x43 ,0x82 ,0xcb ,0xd8 , +0x5d ,0x21 ,0x64 ,0xa8 ,0x80 ,0xc9 ,0x22 ,0xe5 ,0x07 ,0xaf ,0xe2 ,0x5d , +0xcd ,0xc6 ,0x23 ,0x36 ,0x2b ,0xc7 ,0xc7 ,0x7d ,0x09 ,0x9d ,0x68 ,0x05 , +0xe4 ,0x62 ,0x63 ,0x1b ,0x67 ,0xbc ,0xf8 ,0x95 ,0x07 ,0xd2 ,0xe4 ,0xd0 , +0xba ,0xa2 ,0x67 ,0xf5 ,0xe3 ,0x15 ,0xbc ,0x85 ,0xa1 ,0x50 ,0xd6 ,0x6f , +0x6f ,0xd4 ,0x54 ,0x4c ,0x3f ,0x4f ,0xe5 ,0x1f ,0xb7 ,0x00 ,0x27 ,0xfc , +0x15 ,0x33 ,0xc2 ,0xf9 ,0xb3 ,0x4b ,0x9e ,0x81 ,0xe5 ,0x96 ,0xbe ,0x05 , +0x6c ,0xac ,0xf9 ,0x9f ,0x65 ,0x36 ,0xbb ,0x11 ,0x47 ,0x6d ,0xf6 ,0x8f , +0x9f ,0xa2 ,0x77 ,0x37 ,0x3b ,0x18 ,0x77 ,0xcf ,0x65 ,0xc5 ,0xa1 ,0x7e , +0x2c ,0x0e ,0x71 ,0xf0 ,0x4d ,0x18 ,0x67 ,0xb9 ,0xc4 ,0x8c ,0x64 ,0x3b , +0x4b ,0x45 ,0xea ,0x16 ,0xb2 ,0x4a ,0xc5 ,0xf5 ,0x85 ,0xdc ,0xd2 ,0xd9 , +0x13 ,0x77 ,0xb3 ,0x19 ,0xd9 ,0x8c ,0x9f ,0x28 ,0xe7 ,0x64 ,0x91 ,0x0f , +0x6f ,0x32 ,0xbf ,0xa8 ,0xa8 ,0xa3 ,0xff ,0x99 ,0x0e ,0x0b ,0x62 ,0x50 , +0xf8 ,0x3a ,0xc2 ,0xf5 ,0x98 ,0x21 ,0xeb ,0x9d ,0xe8 ,0x45 ,0xf4 ,0x46 , +0x1e ,0x8b ,0xbd ,0x10 ,0x59 ,0x2c ,0x87 ,0xe2 , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; +static const u8 sha3_sample_shake256_200_step2[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x5b ,0x43 ,0x98 ,0x3a ,0x4a ,0x5f ,0xa9 ,0x14 ,0xa5 ,0x98 ,0x26 ,0xe2 , +0xed ,0x45 ,0x6a ,0x1d ,0x61 ,0x24 ,0xf5 ,0x0c ,0xef ,0xda ,0xc2 ,0x8a , +0x30 ,0x0e ,0x03 ,0xe5 ,0x67 ,0xdd ,0x7e ,0x9f ,0xa0 ,0xa4 ,0x07 ,0x63 , +0xdc ,0x6b ,0x7e ,0xbd ,0xd7 ,0x7d ,0x7a ,0x6d ,0x55 ,0x03 ,0x02 ,0x18 , +0x12 ,0x5d ,0xf9 ,0x21 ,0xc8 ,0x78 ,0x69 ,0x7c ,0x64 ,0x39 ,0xfd ,0xf4 , +0xd6 ,0x06 ,0xe6 ,0xd8 ,0x6f ,0xaa ,0x04 ,0x5b ,0x40 ,0xf3 ,0x96 ,0xb2 , +0xb5 ,0xd0 ,0xb5 ,0x43 ,0x50 ,0x9c ,0x08 ,0xd6 ,0x54 ,0x8e ,0x8c ,0x85 , +0xc2 ,0x34 ,0xce ,0x0c ,0x24 ,0x31 ,0x6f ,0x49 ,0xec ,0x3d ,0x13 ,0x1f , +0x36 ,0x0a ,0x14 ,0xa6 ,0x5d ,0x51 ,0x9a ,0x90 ,0x1f ,0xf5 ,0x1f ,0x61 , +0xb7 ,0x65 ,0x64 ,0x2a ,0x00 ,0x07 ,0xe4 ,0x56 ,0x80 ,0x5c ,0xfa ,0x03 , +0xc4 ,0x97 ,0xc1 ,0x09 ,0x35 ,0xa2 ,0x55 ,0x72 ,0x28 ,0xe5 ,0xb6 ,0xef , +0x8e ,0xf4 ,0xc2 ,0x82 ,0x22 ,0xc7 ,0x23 ,0xac ,0xcb ,0xc1 ,0x03 ,0x52 , +0x46 ,0x9c ,0x17 ,0xe0 ,0xa3 ,0x1b ,0x59 ,0x9f ,0x01 ,0xef ,0x5b ,0x46 , +0xb2 ,0x4b ,0x98 ,0x6b ,0x32 ,0x52 ,0xe3 ,0x29 ,0x36 ,0x8f ,0x66 ,0x98 , +0x5f ,0x6a ,0xa2 ,0xf4 ,0x68 ,0x13 ,0x5c ,0x94 ,0xe4 ,0x22 ,0xb6 ,0x83 , +0xa0 ,0xd7 ,0xa3 ,0xda ,0xa4 ,0x84 ,0x0c ,0xf6 ,0xa2 ,0xa4 ,0x0e ,0x08 , +0x6d ,0x2b ,0xd2 ,0x31 ,0x77 ,0x36 ,0xae ,0x53 , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; + +static const u8 sha3_sample_shake256_200_step3[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x61 ,0x0a ,0x5e ,0xdf ,0xf1 ,0x29 ,0xae ,0x82 ,0xab ,0x57 ,0xa8 ,0x1b , +0x4e ,0x7c ,0xb9 ,0x14 ,0x4a ,0x32 ,0x7d ,0x82 ,0xdc ,0xc2 ,0x48 ,0x1c , +0xf4 ,0xa4 ,0x18 ,0xd5 ,0x97 ,0x35 ,0x9a ,0x25 ,0x0f ,0x5f ,0x1b ,0x04 , +0xf1 ,0x09 ,0x2a ,0xe8 ,0xb6 ,0xa7 ,0xe1 ,0x90 ,0xb6 ,0x4d ,0x96 ,0xf1 , +0x7d ,0x4d ,0xb0 ,0x4f ,0x44 ,0xaf ,0x16 ,0x4e ,0x63 ,0xce ,0x46 ,0x4c , +0x76 ,0x18 ,0xbe ,0x5f ,0xf4 ,0x35 ,0xef ,0x1f ,0xb1 ,0x97 ,0x94 ,0x70 , +0x96 ,0x2f ,0xa2 ,0x1b ,0xd6 ,0x02 ,0x51 ,0x88 ,0x33 ,0x2b ,0x54 ,0xb9 , +0x44 ,0xb4 ,0xab ,0x6f ,0xeb ,0xfc ,0xe5 ,0xee ,0xe3 ,0x77 ,0x91 ,0xed , +0x3a ,0x4e ,0x60 ,0x00 ,0x44 ,0xd1 ,0xc7 ,0x4a ,0x54 ,0x77 ,0x71 ,0x95 , +0x53 ,0x88 ,0x6b ,0x1e ,0x0f ,0xfd ,0x62 ,0x02 ,0xa7 ,0x8e ,0x05 ,0x6d , +0x21 ,0x8f ,0x97 ,0x20 ,0xa0 ,0xd7 ,0xcf ,0xd8 ,0x54 ,0xec ,0x50 ,0x72 , +0x07 ,0xb8 ,0x9c ,0x76 ,0xdb ,0x12 ,0x00 ,0xd2 ,0x2e ,0x93 ,0xee ,0xb9 , +0x6a ,0x28 ,0x5a ,0x46 ,0x87 ,0x90 ,0xd5 ,0xd6 ,0x1d ,0x14 ,0x0e ,0x16 , +0xf1 ,0x2c ,0xed ,0x7f ,0x28 ,0x34 ,0x8c ,0x2b ,0x96 ,0x03 ,0x80 ,0x80 , +0x9f ,0xc8 ,0xf4 ,0x2c ,0x53 ,0xe0 ,0x4b ,0x7b ,0xf4 ,0x19 ,0x8a ,0xc5 , +0xb3 ,0x21 ,0x17 ,0xce ,0xdb ,0xbf ,0xb7 ,0x6b ,0x9a ,0xb5 ,0x19 ,0x89 , +0x4c ,0x54 ,0x28 ,0x32 ,0xe6 ,0x85 ,0xfa ,0x8f , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; +static const u8 sha3_sample_shake256_200_step4[]__initconst = { +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-before guard */ +0x83 ,0x99 ,0xc6 ,0xda ,0x75 ,0x79 ,0x8e ,0x47 ,0x06 ,0xad ,0x19 ,0xcb , +0x47 ,0x61 ,0x25 ,0x6a ,0x8c ,0xa4 ,0x7e ,0x74 ,0xba ,0xbd ,0xda ,0xb6 , +0x3b ,0x08 ,0x48 ,0x0f ,0x55 ,0xd5 ,0x85 ,0x78 ,0x5b ,0xd4 ,0x31 ,0xcb , +0x59 ,0xff ,0x93 ,0xff ,0xf6 ,0x65 ,0x4c ,0xf7 ,0x6e ,0x4b ,0xef ,0x4d , +0x0e ,0x43 ,0x8a ,0x2b ,0xed ,0x10 ,0x26 ,0x68 ,0x12 ,0x63 ,0xed ,0x7a , +0x38 ,0x0a ,0xa5 ,0xd0 ,0x79 ,0x26 ,0x75 ,0xef ,0xce ,0xad ,0x6c ,0x12 , +0x52 ,0x33 ,0xec ,0xe8 ,0xe1 ,0x89 ,0x2f ,0x0f ,0x29 ,0xb0 ,0xf6 ,0xff , +0x54 ,0x11 ,0xb2 ,0x6b ,0x22 ,0xb3 ,0x48 ,0x01 ,0xa5 ,0xcf ,0x29 ,0xb7 , +0xaf ,0x8c ,0xec ,0x1e ,0x75 ,0x3e ,0xff ,0xfb ,0x31 ,0xb8 ,0xf6 ,0xab , +0xae ,0xac ,0xec ,0xed ,0x27 ,0x0b ,0x79 ,0x10 ,0x4f ,0x87 ,0xe8 ,0x43 , +0x28 ,0x94 ,0x09 ,0xca ,0x48 ,0x63 ,0x65 ,0x61 ,0x86 ,0x83 ,0x33 ,0x30 , +0x02 ,0x6d ,0xf4 ,0xef ,0x3c ,0x1a ,0x47 ,0x8a ,0x25 ,0x90 ,0x31 ,0x39 , +0x95 ,0x1d ,0x6f ,0x11 ,0x5c ,0x0c ,0x72 ,0xe6 ,0x1b ,0xe1 ,0x60 ,0x45 , +0x79 ,0x89 ,0x39 ,0x48 ,0x31 ,0x4c ,0xc4 ,0xd1 ,0x08 ,0x12 ,0xf3 ,0x5f , +0x84 ,0x8c ,0x86 ,0xba ,0xe5 ,0xf1 ,0x24 ,0x61 ,0x2f ,0xef ,0x17 ,0x16 , +0x4a ,0x29 ,0xc0 ,0xc6 ,0x38 ,0x47 ,0x3a ,0x11 ,0xc5 ,0x7d ,0x62 ,0x85 , +0x9b ,0x18 ,0x92 ,0x4c ,0x12 ,0x92 ,0x9c ,0x0b , +0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,/* Write-after guard */ +}; + +static void __init sha3_check_digest (const u8 *digest,const u8 *sample_digest, +int digest_len) +{ +if (memcmp (digest, sample_digest, digest_len) !=0 ) { +pr_err ("SHAKE256(200) failed\n" ); +for (int i =0 ; i < digest_len;) { +int part =min (digest_len - i,32 ); + +pr_err ("%*phN\n" , part, digest + i); + i += part;
Quoted 8 lines
include/crypto/sha3.h struct sha3_state { · line 91
Quoted 3 lines
include/crypto/sha3.h +3
-0+ } + } +}If a test is included at all here, it should be simpler, e.g. just compute the SHA3-256 of some short hardcoded message. That will be sufficient to test the implementation of the Keccak permutation. The real tests will be in the KUnit suite, as discussed previously.
No. Please stick with the one I have. It exercises the extended features used by ML-DSA that will be used for PQ module signing.
David
Eric Biggers
maintainerQuoted 10 lines · David Howells
Quoted 6 lines · Eric Biggers
lib/crypto/sha3.c line 65
Quoted 3 lines
lib/crypto/sha3.c new file +3
-0+/* update the state with given number of rounds */ + +static SHA3_INLINE void keccakf_round(u64 st[25])The above comment doesn't match the code.
keccakf_round()does only one round.This is exactly as in
crypto/sha3_generic.c. You may want to fix that also.
First, as I've mentioned, the SHA-3 crypto_shash algorithms will eventually need to be reimplemented on top of the library. We shouldn't have two competing SHA-3 implementations in the kernel. So sha3_generic.c will get replaced soon anyway.
Second, this patch already makes changes to the SHA-3 code and doesn't simply copy it. Cleaning up some additional small things like incorrect comments would make sense, similar to how you've also cleaned up the endianness handling. If you'd like for it to instead be a simple copy followed by a separate cleanup patch, that could be okay too, but the submission would need to be organized that way.
Quoted 10 lines
Quoted 4 lines
Would you mind putting all the kerneldoc comments for the public API in the .h file, similar to
sha1.handsha2.h? Then they'll all be in a consistent place, and they won't have to be moved around each time we change functions to inline functions or vice versa.Yes. I know
include/crypto/does it, but that really makes the header files suck as the doc-to-code ratio is too heavily in the doc's favour.Actually, this should be done properly and I'll add an API doc to Documention/crypto/.
These are just functions. The first priority should be kerneldoc, as that's where most people will look. Documention/crypto/ maybe could contain a high-level overview of the various hash function APIs (not just SHA-3), but it should not be duplicative of the kerneldoc and it doesn't eliminate the need to write high-quality kerneldoc.
Quoted 8 lines
Quoted 7 lines
First, this patch's proposed API is error-prone due to the weak typing that allows mixing steps of different algorithms together. For example, users could initialize a
sha3_ctxwithsha3_256_init()and then squeeze an arbitrary amount from it, incorrectly treating it as a XOF. It would be worth considering separating the APIs for the different algorithms that are part of SHA-3, similar to what I did with SHA-224 and SHA-256. (They would of course still share code internally, just like SHA-2.)
Quoted 6 lines
I disagree. You're adding excessive complexity. I would be more inclined to agree if the thing might crash because you used the wrong thing. That said,
sha3_final()uses the digestsize set bysha3_whatever_init(), which could cause a problem, so I think it's better on balance to just add an extra parameter tosha3_final()to say how much digest you want and ditchctx->digest_size.
A cryptographic flaw can be as bad or worse than a crash. We need to design APIs that are easy to use correctly and hard to use incorrectly.
Quoted 15 lines
Quoted 6 lines
Second, the support for
update()+squeeze()+update()+squeeze()seems to be trying to achieve something that is not defined in the SHA-3 spec. Could you elaborate on what it is meant to be doing, and why it's here? According to the spec, the XOFs SHAKE128 and SHAKE256 actually just take a single message as their input.Quoted 8 lines
One might think the above sequence would compute SHAKE*() on the first message as well as SHAKE*() of the two messages concatenated together. But that's not what you've made it do, as you've made it apply padding after *both* messages. So, then one might think that it's meant to be a XOF on the sequence of messages, built on top of SHAKE*(). But it's not that either, since the way that you've proposed to format the sequence of messages into a longer message isn't injective. So, I can't figure out why you're supporting this usage and what it is meant to be doing.
Quoted 2 lines
I can't speak to that except to say that ML-DSA does exactly this as far as I can tell.
Citation needed.
Quoted 5 lines
Quoted 2 lines
Shouldn't the term "digest" be avoided when talking about the output from a XOF, to avoid confusion with traditional hash functions?
No idea. There's not any real programmatic difference except for the padding byte. I can change it to "output" if it makes you happy.
How about reviewing the specs and seeing what terms are used? It's not about what makes me happy, but rather about using standard terminology.
Quoted 7 lines
Quoted 4 lines
If a test is included at all here, it should be simpler, e.g. just compute the SHA3-256 of some short hardcoded message. That will be sufficient to test the implementation of the Keccak permutation. The real tests will be in the KUnit suite, as discussed previously.
No. Please stick with the one I have. It exercises the extended features used by ML-DSA that will be used for PQ module signing.
That should go in the KUnit test suite.
- Eric
David Howells
maintainerQuoted 2 lines · Eric Biggers
Also, the parameter should be strongly typed: 'struct
sha3_state*' Likewise in all the other functions that take the raw u64 array.
Those function may be directly substituted by calls to assembly code - so u64[] is probably more appropriate.
Quoted 8 lines
lib/crypto/sha3.c line 189
Quoted 5 lines
lib/crypto/sha3.c new file +5
-0+for (round =0 ; round <KECCAK_ROUNDS ; round++) { +keccakf_round (st); +/* Iota */ + st[0 ] ^= keccakf_rndc[round]; + }In the spec, "Iota" is part of the round. Having it be separate from
keccakf_round()in the code is confusing.
I assume that pertains to the comment about inlining in some way. This is as is in sha3_generic.c. I can move it into the round function if you like, but can you tell me what the effect will be?
Quoted 5 lines
Second, the support for
update()+squeeze()+update()+squeeze()seems to be trying to achieve something that is not defined in the SHA-3 spec. Could you elaborate on what it is meant to be doing, and why it's here? According to the spec, the XOFs SHAKE128 and SHAKE256 actually just take a single message as their input.
Turns out I was misunderstanding what I was looking at whilst trying to adapt Leancrypto's dilithium code. Whilst it does squeeze a context several times, it doesn't update it after finalising it without reinitialising it.
David
Eric Biggers
maintainerQuoted 7 lines · David Howells
Quoted 2 lines · Eric Biggers
Also, the parameter should be strongly typed: 'struct
sha3_state*' Likewise in all the other functions that take the raw u64 array.Those function may be directly substituted by calls to assembly code - so u64[] is probably more appropriate.
We've been using strongly-typed state even for assembly functions. See the SHA-1, SHA-2, and ChaCha code.
Quoted 9 lines
Quoted 8 lines
lib/crypto/sha3.c line 189
Quoted 5 lines
lib/crypto/sha3.c new file +5
-0+for (round =0 ; round <KECCAK_ROUNDS ; round++) { +keccakf_round (st); +/* Iota */ + st[0 ] ^= keccakf_rndc[round]; + }In the spec, "Iota" is part of the round. Having it be separate from
keccakf_round()in the code is confusing.
Quoted 3 lines
I assume that pertains to the comment about inlining in some way. This is as is in
sha3_generic.c. I can move it into the round function if you like, but can you tell me what the effect will be?
The effect will be that the code will align more closely with how the algorithm is described in the SHA-3 spec and other publications.
Quoted 9 lines
Quoted 5 lines
Second, the support for
update()+squeeze()+update()+squeeze()seems to be trying to achieve something that is not defined in the SHA-3 spec. Could you elaborate on what it is meant to be doing, and why it's here? According to the spec, the XOFs SHAKE128 and SHAKE256 actually just take a single message as their input.Turns out I was misunderstanding what I was looking at whilst trying to adapt Leancrypto's dilithium code. Whilst it does squeeze a context several times, it doesn't update it after finalising it without reinitialising it.
Yes, that's what I thought.
- Eric
David Howells
maintainerQuoted 6 lines · Eric Biggers
Quoted 3 lines
I assume that pertains to the comment about inlining in some way. This is as is in
sha3_generic.c. I can move it into the round function if you like, but can you tell me what the effect will be?The effect will be that the code will align more closely with how the algorithm is described in the SHA-3 spec and other publications.
I meant on the code produced and the stack consumed. It may align with other code, but if it runs off of the end of the stack then alignment is irrelevant.
David
David Howells
maintainerQuoted 11 lines · David Howells
Quoted 6 lines · Eric Biggers
Quoted 3 lines
I assume that pertains to the comment about inlining in some way. This is as is in
sha3_generic.c. I can move it into the round function if you like, but can you tell me what the effect will be?The effect will be that the code will align more closely with how the algorithm is described in the SHA-3 spec and other publications.
I meant on the code produced and the stack consumed. It may align with other code, but if it runs off of the end of the stack then alignment is irrelevant.
See commit 4767b9ad7d762876a5865a06465e13e139a01b6b
"crypto: sha3-generic - deal with oversize stack frames"
For some reason (maybe Ard can comment on it), he left the Iota function out of the keccakf_round() function.
David
Ard Biesheuvel
maintainerQuoted 15 lines · David Howells
Quoted 11 lines · David Howells
Quoted 6 lines · Eric Biggers
Quoted 3 lines
I assume that pertains to the comment about inlining in some way. This is as is in
sha3_generic.c. I can move it into the round function if you like, but can you tell me what the effect will be?The effect will be that the code will align more closely with how the algorithm is described in the SHA-3 spec and other publications.
I meant on the code produced and the stack consumed. It may align with other code, but if it runs off of the end of the stack then alignment is irrelevant.
Quoted 7 lines
See commit
4767b9ad7d762876a5865a06465e13e139a01b6b"crypto: sha3-generic - deal with oversize stack frames"
For some reason (maybe Ard can comment on it), he left the Iota function out of the
keccakf_round()function.
The Iota function is the only transformation that does not operate purely on the state array, so that would have required passing an additional u64 into keccakf_round(). But I agree it is slightly cleaner, although arguably, it should be a separate change.