Skip to contentSkip to conversation

crypto: rsa - restrict plaintext/ciphertext values more in FIPS mode

PATCH Superseded Joachim Vandersmissen 2 participants 4 messages Jan 21, 2024 – Jan 26, 2024

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 crypto: rsa - restrict plaintext/ciphertext values more in FIPS mode

SP 800-56Br2, Section 7.1.1 [1] specifies that:
1. If m does not satisfy 1 < m < (n – 1), output an indication that m is out of range, and exit without further processing.

Similarly, Section 7.1.2 of the same standard specifies that:
1. If the ciphertext c does not satisfy 1 < c < (n – 1), output an indication that the ciphertext is out of range, and exit without further processing.

[1] https://doi.org/10.6028/NIST.SP.800-56Br2

---
 crypto/rsa.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
crypto/rsa.c +29 -0
index b9cd11fb7d36..b5c67e6f8774 100644
@@ -24,12 +24,36 @@ struct rsa_mpi_key {
 	MPI qinv;
 };
 
+static int rsa_check_payload_fips(MPI x, MPI n)
+{
+	MPI n1;
+
+	if (mpi_cmp_ui(x, 1) <= 0)
+		return -EINVAL;
+
+	n1 = mpi_alloc(0);
+	if (!n1)
+		return -ENOMEM;
+
+	if (mpi_sub_ui(n1, n, 1) || mpi_cmp(x, n1) >= 0) {
+		mpi_free(n1);
+		return -EINVAL;
+	}
+
+	mpi_free(n1);
+	return 0;
+}
+
 /*
  * RSAEP function [RFC3447 sec 5.1.1]
  * c = m^e mod n;
  */
 static int _rsa_enc(const struct rsa_mpi_key *key, MPI c, MPI m)
 {
+	/* For FIPS, SP 800-56Br2, Section 7.1.1 requires 1 < m < n - 1 */
+	if (fips_enabled && rsa_check_payload_fips(m, key->n))
+		return -EINVAL;
+
 	/* (1) Validate 0 <= m < n */
 	if (mpi_cmp_ui(m, 0) < 0 || mpi_cmp(m, key->n) >= 0)
 		return -EINVAL;
@@ -50,6 +74,11 @@ static int _rsa_dec_crt(const struct rsa_mpi_key *key, MPI m_or_m1_or_h, MPI c)
 	MPI m2, m12_or_qh;
 	int ret = -ENOMEM;
 
+	/* For FIPS, SP 800-56Br2, Section 7.1.2 requires 1 < c < n - 1 */
+	if (fips_enabled && rsa_check_payload_fips(c, key->n))
+		return -EINVAL;
+
+
 	/* (1) Validate 0 <= c < n */
 	if (mpi_cmp_ui(c, 0) < 0 || mpi_cmp(c, key->n) >= 0)
 		return -EINVAL;
Signature · 1 line
2.43.0
maintainer
#1 lore raw

crypto/rsa.c struct rsa_mpi_key { · line 59

Quoted 10 lines · Joachim Vandersmissen

static int _rsa_enc(const struct rsa_mpi_key *key, MPI c, MPI m) {

crypto/rsa.c +4 -0
+	/* For FIPS, SP 800-56Br2, Section 7.1.1 requires 1 < m < n - 1 */
+	if (fips_enabled && rsa_check_payload_fips(m, key->n))
+		return -EINVAL;
+
 	/* (1) Validate 0 <= m < n */
 	if (mpi_cmp_ui(m, 0) < 0 || mpi_cmp(m, key->n) >= 0)
 		return -EINVAL;

I think this check makes sense in general, so why not simply replace the second check above with the new check?

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#2 lore raw

Hi Herbert,

Quoted 12 lines · Herbert Xu

crypto/rsa.c struct rsa_mpi_key { · line 59

Quoted 9 lines · Joachim Vandersmissen

static int _rsa_enc(const struct rsa_mpi_key *key, MPI c, MPI m) {

crypto/rsa.c +4 -0
+	/* For FIPS, SP 800-56Br2, Section 7.1.1 requires 1 < m < n - 1 */
+	if (fips_enabled && rsa_check_payload_fips(m, key->n))
+		return -EINVAL;
+
  	/* (1) Validate 0 <= m < n */
  	if (mpi_cmp_ui(m, 0) < 0 || mpi_cmp(m, key->n) >= 0)
  		return -EINVAL;

I think this check makes sense in general, so why not simply replace the second check above with the new check?

Yes, mathematically speaking the values 1 and n - 1 aren't suitable for RSA (they will always be fixed points). I simply didn't want to introduce a breaking change. If you think a breaking change is acceptable, I can update the patch to replace the RFC3447 check with the stricter check.

Quoted 2 lines

Thanks,

maintainer
#3 lore raw
Quoted 5 lines · Joachim Vandersmissen

Yes, mathematically speaking the values 1 and n - 1 aren't suitable for RSA (they will always be fixed points). I simply didn't want to introduce a breaking change. If you think a breaking change is acceptable, I can update the patch to replace the RFC3447 check with the stricter check.

Please do. We can always change it later if someone complains.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt