tls: fix race between tx work scheduling and socket close
PATCH 1/1 tls: fix race between tx work scheduling and socket close
From: Jakub Kicinski <[email protected]>
[ Upstream commit e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb ]
Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls complete(). Reorder scheduling the work before calling complete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.
--- net/tls/tls_sw.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
net/tls/tls_sw.c +6 -10
@@ -449,7 +449,6 @@ static void tls_encrypt_done(crypto_completion_data_t *data, int err)struct scatterlist *sge;struct sk_msg *msg_en;struct tls_rec *rec; -bool ready = false;struct sock *sk; rec = container_of(aead_req, struct tls_rec, aead_req);
@@ -486,19 +485,16 @@ static void tls_encrypt_done(crypto_completion_data_t *data, int err)/* If received record is at head of tx_list, schedule tx */ first_rec =list_first_entry (&ctx->tx_list, struct tls_rec, list); -if (rec == first_rec) - ready = true; +if (rec == first_rec) { +/* Schedule the transmission */ +if (!test_and_set_bit (BIT_TX_SCHEDULED , + &ctx->tx_bitmask)) +schedule_delayed_work (&ctx->tx_work.work,1 ); + } }if (atomic_dec_and_test (&ctx->encrypt_pending))complete (&ctx->async_wait.completion); - -if (!ready) -return ; - -/* Schedule the transmission */ -if (!test_and_set_bit (BIT_TX_SCHEDULED , &ctx->tx_bitmask)) -schedule_delayed_work (&ctx->tx_work.work,1 ); }static int tls_encrypt_async_wait (struct tls_sw_context_tx *ctx)
Signature · 1 line
2.44.0.278.ge034bb2e1d-goog
Quoted 10 lines · On Thu, 7 Mar 2024 15:59:29 +0000 Lee Jones wrote:
From: Jakub Kicinski <[email protected]>
[ Upstream commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb]Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.
Quoted 10 lines
Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>Signed-off-by Sasha Levin <[email protected]>(cherry picked from commit 6db22d6c7a6dc914b12c0469b94eb639b6a8a146)[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>
LG, thank you!
The 5.15 / 5.10 / 5.4 fixes won't be effective, tho. I don't see commit aec7961916f3f9e88766 in the other LTS branches. Without that (it's still correct but) it doesn't fix the problem, because we still touch the context after releasing the reference (unlocking the spin lock).
Quoted 22 lines · Jakub Kicinski
Quoted 10 lines · On Thu, 7 Mar 2024 15:59:29 +0000 Lee Jones wrote:
From: Jakub Kicinski <[email protected]>
[ Upstream commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb]Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.Quoted 10 lines
Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>Signed-off-by Sasha Levin <[email protected]>(cherry picked from commit 6db22d6c7a6dc914b12c0469b94eb639b6a8a146)[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>
Quoted 7 lines
LG, thank you!
The 5.15 / 5.10 / 5.4 fixes won't be effective, tho. I don't see commit
aec7961916f3f9e88766in the other LTS branches. Without that (it's still correct but) it doesn't fix the problem, because we still touch the context after releasing the reference (unlocking the spin lock).
No problem.
Should I accompany aec7961916f3 with this fix into the aforementioned branches then? Would that then be effective?
Signature · 1 line
Lee Jones [李琼斯]
Quoted 10 lines · On Thu, 7 Mar 2024 17:09:59 +0000 Lee Jones wrote:
Quoted 5 lines
The 5.15 / 5.10 / 5.4 fixes won't be effective, tho. I don't see commit
aec7961916f3f9e88766in the other LTS branches. Without that (it's still correct but) it doesn't fix the problem, because we still touch the context after releasing the reference (unlocking the spin lock).No problem.
Should I accompany
aec7961916f3with this fix into the aforementioned branches then? Would that then be effective?
Yes, (and c57ca512f3b68d, that's all the pre-req, I think.) Tho, it may be a more tedious backport.
FWIW tools/testing/selftests/net/tls.c should be relatively solid.
PATCH 1/1 tls: fix race between tx work scheduling and socket close
From: Jakub Kicinski <[email protected]>
commit e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb upstream.
Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls complete(). Reorder scheduling the work before calling complete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.
[Harshit: bp to 5.15.y, minor conflict resolutin due to missing commit:8ae187386420 ("tls: Only use data field in crypto completion function") in 5.15.y]
---
This is a fix for CVE-2024-26585, minor conflict resolution involved Ran the tls self tests:
ok 183 tls.13_chacha.shutdown_reuse
# PASSED: 183 / 183 tests passed.
# Totals: pass:183 fail:0 xfail:0 xpass:0 skip:0 error:0
net/tls/tls_sw.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
net/tls/tls_sw.c +6 -10
@@ -468,7 +468,6 @@ static void tls_encrypt_done(struct crypto_async_request *req, int err)struct scatterlist *sge;struct sk_msg *msg_en;struct tls_rec *rec; -bool ready = false;if (err == -EINPROGRESS )/* see the comment in tls_decrypt_done() */ return ;
@@ -502,19 +501,16 @@ static void tls_encrypt_done(struct crypto_async_request *req, int err)/* If received record is at head of tx_list, schedule tx */ first_rec =list_first_entry (&ctx->tx_list, struct tls_rec, list); -if (rec == first_rec) - ready = true; +if (rec == first_rec) { +/* Schedule the transmission */ +if (!test_and_set_bit (BIT_TX_SCHEDULED , + &ctx->tx_bitmask)) +schedule_delayed_work (&ctx->tx_work.work,1 ); + } }if (atomic_dec_and_test (&ctx->encrypt_pending))complete (&ctx->async_wait.completion); - -if (!ready) -return ; - -/* Schedule the transmission */ -if (!test_and_set_bit (BIT_TX_SCHEDULED , &ctx->tx_bitmask)) -schedule_delayed_work (&ctx->tx_work.work,1 ); }static int tls_encrypt_async_wait (struct tls_sw_context_tx *ctx)
Signature · 1 line
2.45.2
Greg Kroah-Hartman
maintainerQuoted 27 lines · Harshit Mogalapalli
From: Jakub Kicinski <[email protected]>
commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdbupstream.Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>Signed-off-by Greg Kroah-Hartman <[email protected]>[Harshit: bp to 5.15.y, minor conflict resolutin due to missing commit:
8ae187386420("tls: Only use data field in crypto completion function") in 5.15.y]Signed-off-by Harshit Mogalapalli <[email protected]>---
This is a fix for CVE-2024-26585, minor conflict resolution involved Ran the tls self tests:
Quoted 6 lines
ok 183 tls.13_chacha.shutdown_reuse
# PASSED: 183 / 183 tests passed.
# Totals: pass:183 fail:0 xfail:0 xpass:0 skip:0 error:0net/tls/tls_sw.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
Now queued up, thanks.
greg k-h
Hi Jakub,
Quoted 22 lines · Jakub Kicinski
Quoted 10 lines · On Thu, 7 Mar 2024 15:59:29 +0000 Lee Jones wrote:
From: Jakub Kicinski <[email protected]>
[ Upstream commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb]Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.Quoted 10 lines
Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>Signed-off-by Sasha Levin <[email protected]>(cherry picked from commit 6db22d6c7a6dc914b12c0469b94eb639b6a8a146)[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>
Quoted 8 lines
LG, thank you!
The 5.15 / 5.10 / 5.4 fixes won't be effective, tho. I don't see commit
aec7961916f3f9e88766in the other LTS branches. Without that (it's still correct but) it doesn't fix the problem, because we still touch the context after releasing the reference (unlocking the spin lock).
commit: aec7961916f3 ("tls: fix race between async notify and socket close") is backported to 5.15.y as commit f17d21ea7391 ("tls: fix race between async notify and socket close") in v5.15.160
So I think we should now backport this fix e01e3934a1b2 ("tls: fix race between tx work scheduling and socket close") to 5.15.y ( this also fixes CVE-2024-26585)
Will send the backport to stable kernel list.
Thanks,
Harshit
Greg KH
maintainerQuoted 20 lines · Lee Jones
From: Jakub Kicinski <[email protected]>
[ Upstream commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb]Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>Signed-off-by Sasha Levin <[email protected]>(cherry picked from commit 6db22d6c7a6dc914b12c0469b94eb639b6a8a146)[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>
Quoted 4 lines
--- net/tls/tls_sw.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
Now qeueued up, but only this version, the older ones I've dropped from my review queue based on the review from Jakub. If they are still needed, can you provide backported versions?
thanks,
greg k-h
Quoted 26 lines · Greg KH
Quoted 20 lines · Lee Jones
From: Jakub Kicinski <[email protected]>
[ Upstream commit
e01e3934a1b2d122919f73bc6ddbe1cdafc4bbdb]Similarly to previous commit, the submitting thread (recvmsg/sendmsg) may exit as soon as the async crypto handler calls
complete(). Reorder scheduling the work before callingcomplete(). This seems more logical in the first place, as it's
the inverse order of what the submitting thread will do.Reported-by valis <[email protected]>Fixes a42055e8d2c3 ("net/tls: Add support for async encryption of records for performance")Signed-off-by Jakub Kicinski <[email protected]>Reviewed-by Simon Horman <[email protected]>Reviewed-by Sabrina Dubroca <[email protected]>Signed-off-by David S. Miller <[email protected]>Signed-off-by Sasha Levin <[email protected]>(cherry picked from commit 6db22d6c7a6dc914b12c0469b94eb639b6a8a146)[Lee: Fixed merge-conflict in Stable branches linux-6.1.y and older]Signed-off-by Lee Jones <[email protected]>Quoted 4 lines
--- net/tls/tls_sw.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-)
Quoted 3 lines
Now qeueued up, but only this version, the older ones I've dropped from my review queue based on the review from Jakub. If they are still needed, can you provide backported versions?
Thanks.
Full disclosure, I have no plans to backport the remainder given Jakub's comments.
Signature · 1 line
Lee Jones [李琼斯]