net/tls: fixes for NVMe-over-TLS
Subsystems (5)
v9 cover net/tls: fixes for NVMe-over-TLS
Hi all,
here are some small fixes to get NVMe-over-TLS up and running. The first set are just minor modifications to have MSG_EOR handled for TLS, but the second set implements the ->read_sock() callback for tls_sw.
The ->read_sock() callbacks return -EIO when encountering any TLS Alert message, but as that's the default behaviour anyway I guess we can get away with it.
As usual, comments and reviews are welcome.
Changes to the original submission:
- Add a testcase for MSG_EOR handling
Changes to v2:
- Bail out on conflicting message flags
- Rework flag handling
Changes to v3:
- Return -EINVAL on conflicting flags
- Rebase on top of net-next
Changes to v4:
- Add tlx_rx_reader_lock() to read_sock
- Add MSG_EOR handling to tls_sw_readpages()
Changes to v5:
- Rebase to latest upstream
- Split tls_rx_reader_lock() as suggested by Sagi
Changes to v6:
- Fixup tls_strp_read_copyin() to avoid infinite recursion in tls_read_sock()
- Rework tls_read_sock() to read all available data
Changes to v7:
- Include reviews from Jakub
Changes to v8:
- Use tls_read_flush_backlog()
Hannes Reinecke (6):
net/tls: handle MSG_EOR for tls_sw TX flow net/tls: handle MSG_EOR for tls_device TX flow selftests/net/tls: add test for MSG_EOR net/tls: Use tcp_read_sock() instead of ops->read_sock() net/tls: split tls_rx_reader_lock
net/tls: implement ->read_sock()
net/tls/tls.h | 2 + net/tls/tls_device.c | 6 +- net/tls/tls_main.c | 2 + net/tls/tls_strp.c | 3 +- net/tls/tls_sw.c | 142 ++++++++++++++++++++++++++---- tools/testing/selftests/net/tls.c | 11 +++ 6 files changed, 146 insertions(+), 20 deletions(-)
Signature · 1 line
2.35.3
PATCH v9 1/6 net/tls: handle MSG_EOR for tls_sw TX flow
tls_sw_sendmsg() already handles MSG_MORE, but bails out on MSG_EOR.
Seeing that MSG_EOR is basically the opposite of MSG_MORE this patch adds handling MSG_EOR by treating it as the negation of MSG_MORE.
And erroring out if MSG_EOR is specified with MSG_MORE.
--- net/tls/tls_sw.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
net/tls/tls_sw.c +4 -1
@@ -984,6 +984,9 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg,int ret =0 ;int pending; +if (!eor && (msg->msg_flags &MSG_EOR )) +return -EINVAL ; +if (unlikely (msg->msg_controllen)) { ret =tls_process_cmsg (sk, msg, &record_type);if (ret) {
@@ -1193,7 +1196,7 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)int ret;if (msg->msg_flags & ~(MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL | -MSG_CMSG_COMPAT |MSG_SPLICE_PAGES | +MSG_CMSG_COMPAT |MSG_SPLICE_PAGES |MSG_EOR |MSG_SENDPAGE_NOPOLICY ))return -EOPNOTSUPP ;
Signature · 1 line
2.35.3
PATCH v9 2/6 net/tls: handle MSG_EOR for tls_device TX flow
In reply to Hannes Reinecketls_push_data() MSG_MORE, but bails out on MSG_EOR. Seeing that MSG_EOR is basically the opposite of MSG_MORE this patch adds handling MSG_EOR by treating it as the absence of MSG_MORE.
Consequently we should return an error when both are set.
--- net/tls/tls_device.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
net/tls/tls_device.c +5 -1
@@ -441,9 +441,13 @@ static int tls_push_data(struct sock *sk,long timeo;if (flags & - ~(MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL |MSG_SPLICE_PAGES )) + ~(MSG_MORE |MSG_DONTWAIT |MSG_NOSIGNAL | +MSG_SPLICE_PAGES |MSG_EOR ))return -EOPNOTSUPP ; +if ((flags & (MSG_MORE |MSG_EOR )) == (MSG_MORE |MSG_EOR )) +return -EINVAL ; +if (unlikely (sk->sk_err))return -sk->sk_err;
Signature · 1 line
2.35.3
PATCH v9 3/6 selftests/net/tls: add test for MSG_EOR
In reply to Hannes ReineckeAs the recent patch is modifying the behaviour for TLS re MSG_EOR handling we should be having a test for it.
--- tools/testing/selftests/net/tls.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/testing/selftests/net/tls.c +11 -0
@@ -486,6 +486,17 @@ TEST_F(tls, msg_more_unsent)EXPECT_EQ (recv (self->cfd, buf, send_len,MSG_DONTWAIT ),-1 ); } +TEST_F(tls, msg_eor) +{ +char const *test_str ="test_read" ; +int send_len =10 ; +char buf[10 ]; + +EXPECT_EQ (send (self->fd, test_str, send_len,MSG_EOR ), send_len); +EXPECT_EQ (recv (self->cfd, buf, send_len,MSG_WAITALL ), send_len); +EXPECT_EQ (memcmp (buf, test_str, send_len),0 ); +} + TEST_F(tls, sendmsg_single) {struct msghdr msg;
Signature · 1 line
2.35.3
PATCH v9 4/6 net/tls: Use tcp_read_sock() instead of ops->read_sock()
In reply to Hannes ReineckeTLS resets the protocol operations, so the read_sock() callback might be changed, too.
In this case using sock->ops->readsock() in tls_strp_read_copyin() will enter an infinite recursion if the read_sock() callback is calling tls_rx_rec_wait() which will call into sock->ops->readsock() via tls_strp_read_copyin().
But as tls_strp_read_copyin() is supposed to produce data from the consumed socket and that socket is always a TCP socket we can call tcp_read_sock() directly without having to deal with callbacks.
--- net/tls/tls_strp.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
net/tls/tls_strp.c +1 -2
@@ -369,7 +369,6 @@ static int tls_strp_copyin(read_descriptor_t *desc, struct sk_buff *in_skb,static int tls_strp_read_copyin (struct tls_strparser *strp) { -struct socket *sock = strp->sk->sk_socket;read_descriptor_t desc; desc.arg.data = strp;
@@ -377,7 +376,7 @@ static int tls_strp_read_copyin(struct tls_strparser *strp) desc.count =1 ;/* give more than one skb per call */ /* sk should be locked here, so okay to do read_sock */ - sock->ops->read_sock (strp->sk, &desc, tls_strp_copyin); +tcp_read_sock (strp->sk, &desc, tls_strp_copyin);return desc.error; }
Signature · 1 line
2.35.3
PATCH v9 5/6 net/tls: split tls_rx_reader_lock
In reply to Hannes ReineckeSplit tls_rx_reader_{lock,unlock} into an 'acquire/release' and the actual locking part.
With that we can use the tls_rx_reader_lock in situations where the socket is already locked.
--- net/tls/tls_sw.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-)
net/tls/tls_sw.c +22 -16
@@ -1848,13 +1848,10 @@ tls_read_flush_backlog(struct sock *sk, struct tls_prot_info *prot,return sk_flush_backlog (sk); } -static int tls_rx_reader_lock (struct sock *sk,struct tls_sw_context_rx *ctx, -bool nonblock) +static int tls_rx_reader_acquire (struct sock *sk,struct tls_sw_context_rx *ctx, +bool nonblock) {long timeo; -int err; - -lock_sock (sk); timeo =sock_rcvtimeo (sk, nonblock);
@@ -1868,26 +1865,30 @@ static int tls_rx_reader_lock(struct sock *sk, struct tls_sw_context_rx *ctx, !READ_ONCE(ctx->reader_present), &wait);remove_wait_queue (&ctx->wq, &wait); -if (timeo <=0 ) { - err = -EAGAIN ; - goto err_unlock; - } -if (signal_pending (current)) { - err =sock_intr_errno (timeo); - goto err_unlock; - } +if (timeo <=0 ) +return -EAGAIN ; +if (signal_pending (current)) +return sock_intr_errno (timeo); }WRITE_ONCE (ctx->reader_present,1 );return 0 ; +} -err_unlock: -release_sock (sk); +static int tls_rx_reader_lock (struct sock *sk,struct tls_sw_context_rx *ctx, +bool nonblock) +{ +int err; + +lock_sock (sk); + err =tls_rx_reader_acquire (sk, ctx, nonblock); +if (err) +release_sock (sk);return err; } -static void tls_rx_reader_unlock (struct sock *sk,struct tls_sw_context_rx *ctx) +static void tls_rx_reader_release (struct sock *sk,struct tls_sw_context_rx *ctx) {if (unlikely (ctx->reader_contended)) {if (wq_has_sleeper (&ctx->wq))
@@ -1899,6 +1900,11 @@ static void tls_rx_reader_unlock(struct sock *sk, struct tls_sw_context_rx *ctx) }WRITE_ONCE (ctx->reader_present,0 ); +} + +static void tls_rx_reader_unlock (struct sock *sk,struct tls_sw_context_rx *ctx) +{ +tls_rx_reader_release (sk, ctx);release_sock (sk); }
Signature · 1 line
2.35.3
PATCH v9 6/6 net/tls: implement ->read_sock()
In reply to Hannes ReineckeImplement ->read_sock() function for use with nvme-tcp.
--- net/tls/tls.h | 2 + net/tls/tls_main.c | 2 + net/tls/tls_sw.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 103 insertions(+)
net/tls/tls.h +2 -0
@@ -110,6 +110,8 @@ bool tls_sw_sock_is_readable(struct sock *sk);ssize_t tls_sw_splice_read (struct socket *sock,loff_t *ppos,struct pipe_inode_info *pipe,size_t len,unsigned int flags); +int tls_sw_read_sock (struct sock *sk,read_descriptor_t *desc, +sk_read_actor_t read_actor);int tls_device_sendmsg (struct sock *sk,struct msghdr *msg,size_t size);void tls_device_splice_eof (struct socket *sock);
net/tls/tls_main.c +2 -0
@@ -962,10 +962,12 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG] ops[TLS_BASE ][TLS_SW ] = ops[TLS_BASE ][TLS_BASE ]; ops[TLS_BASE ][TLS_SW ].splice_read = tls_sw_splice_read; ops[TLS_BASE ][TLS_SW ].poll = tls_sk_poll; + ops[TLS_BASE ][TLS_SW ].read_sock = tls_sw_read_sock; ops[TLS_SW ][TLS_SW ] = ops[TLS_SW ][TLS_BASE ]; ops[TLS_SW ][TLS_SW ].splice_read = tls_sw_splice_read; ops[TLS_SW ][TLS_SW ].poll = tls_sk_poll; + ops[TLS_SW ][TLS_SW ].read_sock = tls_sw_read_sock;#ifdef CONFIG_TLS_DEVICE ops[TLS_HW ][TLS_BASE ] = ops[TLS_BASE ][TLS_BASE ];
net/tls/tls_sw.c +99 -0
105 lines not on this page · raw message
Signature · 1 line
2.35.3
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:
Quoted 11 lines · On Wed, 26 Jul 2023 21:15:50 +0200 you wrote:
Hi all,
here are some small fixes to get NVMe-over-TLS up and running. The first set are just minor modifications to have
MSG_EORhandled for TLS, but the second set implements the ->read_sock() callback fortls_sw.
The ->read_sock() callbacks return-EIOwhen encountering any TLS Alert message, but as that's the default behaviour anyway I guess we can get away with it.[...]
Here is the summary with links:
- [1/6] net/tls: handle MSG_EOR for tls_sw TX flow
https://git.kernel.org/netdev/net-next/c/e22e358bbeb3 - [2/6] net/tls: handle MSG_EOR for tls_device TX flow https://git.kernel.org/netdev/net-next/c/c004b0e00c94 - [3/6] selftests/net/tls: add test for MSG_EOR https://git.kernel.org/netdev/net-next/c/8790c6a4f54d - [4/6] net/tls: Use tcp_read_sock() instead of ops->read_sock() https://git.kernel.org/netdev/net-next/c/11863c6d440d - [5/6] net/tls: split tls_rx_reader_lock https://git.kernel.org/netdev/net-next/c/f9ae3204fb45 - [6/6] net/tls: implement ->read_sock() https://git.kernel.org/netdev/net-next/c/662fbcec32f4
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html