Skip to contentSkip to conversation

net/tls: fixes for NVMe-over-TLS

PATCH Superseded Hannes Reinecke 3 participants 9 messages Jun 9, 2023 – Jun 12, 2023

Subsystems (5)

v2 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#4 lore raw

PATCH v2 2/4 net/tls: handle MSG_EOR for tls_device TX flow

In reply to Hannes Reinecke

tls_push_data() MSG_MORE / MSG_SENDPAGE_NOTLAST, but bails out on MSG_EOR.
But seeing that MSG_EOR is basically the opposite of MSG_MORE / MSG_SENDPAGE_NOTLAST this patch adds handling MSG_EOR by treating it as the negation of MSG_MORE.

---
 net/tls/tls_device.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
net/tls/tls_device.c +7 -1
index a7cc4f9faac2..9603a3c9ec24 100644
@@ -449,7 +449,7 @@ static int tls_push_data(struct sock *sk,
 	long timeo;
 
 	if (flags &
-	    ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST))
+	    ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_EOR))
 		return -EOPNOTSUPP;
 
 	if (unlikely(sk->sk_err))
@@ -529,6 +529,10 @@ static int tls_push_data(struct sock *sk,
 				more = true;
 				break;
 			}
+			if (flags & MSG_EOR) {
+				more = false;
+				break;
+			}
 
 			done = true;
 		}
@@ -603,6 +607,8 @@ int tls_device_sendpage(struct sock *sk, struct page *page,
 
 	if (flags & MSG_SENDPAGE_NOTLAST)
 		flags |= MSG_MORE;
+	if (flags & MSG_EOR)
+		flags &= ~(MSG_MORE | MSG_SENDPAGE_NOTLAST);
 
 	mutex_lock(&tls_ctx->tx_lock);
 	lock_sock(sk);
Signature · 1 line
2.35.3
#7 lore raw

PATCH v2 4/4 selftests/net/tls: add test for MSG_EOR

In reply to Hannes Reinecke

As the recent patch is modifying the behaviour for TLS re MSG_EOR handling we should be having a test for it.

---
 tools/testing/selftests/net/tls.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/testing/selftests/net/tls.c +11 -0
index e699548d4247..c1cd1446c7e2 100644
@@ -476,6 +476,17 @@ TEST_F(tls, msg_more_unsent)
 	EXPECT_EQ(recv(self->cfd, buf, send_len, MSG_DONTWAIT), -1);
 }
 
+TEST_F(tls, msg_eor)
+{
+	char const *test_str = "test_read";
+	int send_len = 10;
+	char buf[10];
+
+	EXPECT_EQ(send(self->fd, test_str, send_len, MSG_EOR), send_len);
+	EXPECT_EQ(recv(self->cfd, buf, send_len, MSG_WAITALL), send_len);
+	EXPECT_EQ(memcmp(buf, test_str, send_len), 0);
+}
+
 TEST_F(tls, sendmsg_single)
 {
 	struct msghdr msg;
Signature · 1 line
2.35.3
#0 lore raw

v2 cover net/tls: fixes for NVMe-over-TLS

Hi all,

here are some small fixes to get NVMe-over-TLS up and running. The first two are just minor modifications to have MSG_EOR handled for TLS, but the third implements the ->read_sock() callback for tls_sw and I guess could do with some reviews.

As usual, comments and reviews are welcome.

Changes to the original submission:
- Add a testcase for MSG_EOR handling

Hannes Reinecke (4):
net/tls: handle MSG_EOR for tls_sw TX flow
net/tls: handle MSG_EOR for tls_device TX flow
net/tls: implement ->read_sock()
selftests/net/tls: add test for MSG_EOR

 net/tls/tls.h                     |  2 +
 net/tls/tls_device.c              |  8 ++-
 net/tls/tls_main.c                |  2 +
 net/tls/tls_sw.c                  | 82 +++++++++++++++++++++++++++++--
 tools/testing/selftests/net/tls.c | 11 +++++
 5 files changed, 101 insertions(+), 4 deletions(-)
Signature · 1 line
2.35.3
#1 lore raw

PATCH v2 1/4 net/tls: handle MSG_EOR for tls_sw TX flow

tls_sw_sendmsg() / tls_do_sw_sendpage() already handles MSG_MORE / MSG_SENDPAGE_NOTLAST, but bails out on MSG_EOR. But seeing that MSG_EOR is basically the opposite of MSG_MORE / MSG_SENDPAGE_NOTLAST this patch adds handling MSG_EOR by treating it as the negation of MSG_MORE.

---
 net/tls/tls_sw.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
net/tls/tls_sw.c +8 -3
index 635b8bf6b937..be8e0459d403 100644
@@ -953,9 +953,12 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
 	int pending;
 
 	if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
-			       MSG_CMSG_COMPAT))
+			       MSG_EOR | MSG_CMSG_COMPAT))
 		return -EOPNOTSUPP;
 
+	if (msg->msg_flags & MSG_EOR)
+		eor = true;
+
 	ret = mutex_lock_interruptible(&tls_ctx->tx_lock);
 	if (ret)
 		return ret;
@@ -1173,6 +1176,8 @@ static int tls_sw_do_sendpage(struct sock *sk, struct page *page,
 	bool eor;
 
 	eor = !(flags & MSG_SENDPAGE_NOTLAST);
+	if (flags & MSG_EOR)
+		eor = true;
 	sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk);
 
 	/* Call the sk_stream functions to manage the sndbuf mem. */
@@ -1274,7 +1279,7 @@ static int tls_sw_do_sendpage(struct sock *sk, struct page *page,
 int tls_sw_sendpage_locked(struct sock *sk, struct page *page,
 			   int offset, size_t size, int flags)
 {
-	if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
+	if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_EOR |
 		      MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY |
 		      MSG_NO_SHARED_FRAGS))
 		return -EOPNOTSUPP;
@@ -1288,7 +1293,7 @@ int tls_sw_sendpage(struct sock *sk, struct page *page,
 	struct tls_context *tls_ctx = tls_get_ctx(sk);
 	int ret;
 
-	if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
+	if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_EOR |
 		      MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY))
 		return -EOPNOTSUPP;
 
Signature · 1 line
2.35.3
#5 lore raw

PATCH v2 3/4 net/tls: implement ->read_sock()

In reply to Hannes Reinecke

Implement ->read_sock() function for use with nvme-tcp.

---
 net/tls/tls.h      |  2 ++
 net/tls/tls_main.c |  2 ++
 net/tls/tls_sw.c   | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+)
net/tls/tls.h +2 -0
index 804c3880d028..a5bf3a9ce142 100644
@@ -113,6 +113,8 @@ bool tls_sw_sock_is_readable(struct sock *sk);
 ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
 			   struct pipe_inode_info *pipe,
 			   size_t len, unsigned int flags);
+int tls_sw_read_sock(struct sock *sk, read_descriptor_t *desc,
+		     sk_read_actor_t read_actor);
 
 int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
 int tls_device_sendpage(struct sock *sk, struct page *page,
net/tls/tls_main.c +2 -0
index f2e7302a4d96..767297a029b9 100644
@@ -922,9 +922,11 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG]
 
 	ops[TLS_BASE][TLS_SW  ] = ops[TLS_BASE][TLS_BASE];
 	ops[TLS_BASE][TLS_SW  ].splice_read	= tls_sw_splice_read;
+	ops[TLS_BASE][TLS_SW  ].read_sock	= tls_sw_read_sock;
 
 	ops[TLS_SW  ][TLS_SW  ] = ops[TLS_SW  ][TLS_BASE];
 	ops[TLS_SW  ][TLS_SW  ].splice_read	= tls_sw_splice_read;
+	ops[TLS_SW  ][TLS_SW  ].read_sock	= tls_sw_read_sock;
 
 #ifdef CONFIG_TLS_DEVICE
 	ops[TLS_HW  ][TLS_BASE] = ops[TLS_BASE][TLS_BASE];
net/tls/tls_sw.c +71 -0

77 lines not on this page · raw message

Signature · 1 line
2.35.3
maintainer
#2 lore raw

Re: [PATCH 1/4] net/tls: handle MSG_EOR for tls_sw TX flow

In reply to Hannes Reinecke

Hi Hannes,

Quoted 18 lines · Hannes Reinecke

tls_sw_sendmsg() / tls_do_sw_sendpage() already handles MSG_MORE / MSG_SENDPAGE_NOTLAST, but bails out on MSG_EOR. But seeing that MSG_EOR is basically the opposite of MSG_MORE / MSG_SENDPAGE_NOTLAST this patch adds handling MSG_EOR by treating it as the negation of MSG_MORE.

---
 net/tls/tls_sw.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
net/tls/tls_sw.c +0 -0
index 635b8bf6b937..be8e0459d403 100644

net/tls/tls_sw.c int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) · line 960

Quoted 9 lines
net/tls/tls_sw.c +3 -1
@@ -953,9 +953,12 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
 	int pending;
 
 	if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
-			       MSG_CMSG_COMPAT))
+			       MSG_EOR | MSG_CMSG_COMPAT))
 		return -EOPNOTSUPP;
 
+	if (msg->msg_flags & MSG_EOR)
+		eor = true;

Is MSG_EOR supposed to be incompatible with MSG_MORE, or is it supposed to cancel it? (ie: MSG_MORE | MSG_EOR is invalid, or MSG_MORE | MSG_EOR behaves like MSG_EOR) The current code already behaves as if _EOR was passed as long as MSG_MORE isn't passed, so _EOR is only needed to cancel out _MORE (or in your case, because NVMe-over-TLS sets it).

If _EOR and _MORE (or MSG_SENDPAGE_NOTLAST below) are supposed to be incompatible, we should return an error when they're both set. If we accept both flags being set at the same time, I think we should document the expected behavior ("_EOR overrides _MORE/_NOTLAST") and add specific selftests to avoid regressions.

Signature · 1 line
Sabrina
#3 lore raw

Re: [PATCH 1/4] net/tls: handle MSG_EOR for tls_sw TX flow

Quoted 28 lines · Sabrina Dubroca

Hi Hannes,

Quoted 25 lines · Hannes Reinecke

tls_sw_sendmsg() / tls_do_sw_sendpage() already handles MSG_MORE / MSG_SENDPAGE_NOTLAST, but bails out on MSG_EOR. But seeing that MSG_EOR is basically the opposite of MSG_MORE / MSG_SENDPAGE_NOTLAST this patch adds handling MSG_EOR by treating it as the negation of MSG_MORE.

---
  net/tls/tls_sw.c | 11 ++++++++---
  1 file changed, 8 insertions(+), 3 deletions(-)
net/tls/tls_sw.c +1 -1
index 635b8bf6b937..be8e0459d403 100644
@@ -953,9 +953,12 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
  	int pending;
 
  	if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL |
-			       MSG_CMSG_COMPAT))
+			       MSG_EOR | MSG_CMSG_COMPAT))
  		return -EOPNOTSUPP;
 
Quoted 10 lines

net/tls/tls_sw.c int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) · line 960

Quoted 2 lines
net/tls/tls_sw.c +2 -0
+	if (msg->msg_flags & MSG_EOR)
+		eor = true;

Is MSG_EOR supposed to be incompatible with MSG_MORE, or is it supposed to cancel it? (ie: MSG_MORE | MSG_EOR is invalid, or MSG_MORE | MSG_EOR behaves like MSG_EOR) The current code already behaves as if _EOR was passed as long as MSG_MORE isn't passed, so _EOR is only needed to cancel out _MORE (or in your case, because NVMe-over-TLS sets it).

I guess the correct answer here is: Implementation defined. Personally I would veer towards the first option (ie MSG_MORE is incompatible with MSG_EOR), but then it depends on the implementation to check for that. And most don't.

Quoted 6 lines

If _EOR and _MORE (or MSG_SENDPAGE_NOTLAST below) are supposed to be incompatible, we should return an error when they're both set. If we accept both flags being set at the same time, I think we should document the expected behavior ("_EOR overrides _MORE/_NOTLAST") and add specific selftests to avoid regressions.

Hmm. Yes, guess you are correct. I'll be adding a check to make MSG_MORE incompatible with MSG_EOR.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke                Kernel Storage Architect
[email protected]                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman