Skip to contentSkip to conversation

get msk->snd_nxt with READ_ONCE

PATCH In review Geliang Tang 3 participants 8 messages Jul 10, 2024 – Jul 15, 2024

Subsystems (5)
lore
Share & tools
lore

Geliang Tang

author reviewer
#0 lore raw

cover get msk->snd_nxt with READ_ONCE

From: Geliang Tang <[email protected]>

Get msk->snd_nxt with READ_ONCE() in get_subflow() of the burst schedulers.

Geliang Tang (3):
mptcp: add mptcp_snd_nxt helper
Squash to "bpf: Export more bpf_burst related functions" Squash to "selftests/bpf: Add bpf_burst scheduler"

 net/mptcp/bpf.c                                     | 1 +
 net/mptcp/protocol.c                                | 7 ++++++-
 net/mptcp/protocol.h                                | 1 +
 tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c | 3 ++-
 4 files changed, 10 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0
author reviewer
#1 lore raw

PATCH 1/3 mptcp: add mptcp_snd_nxt helper

From: Geliang Tang <[email protected]>

It's better to get msk->snd_nxt with READ_ONCE() in get_subflow() of the burst scheduler.

Similar to mptcp_wnd_end() helper, this patch defines a new helper mptcp_snd_nxt(), and use it in mptcp_subflow_get_send().

It will be used in BPF burst scheduler too, so export it in protocol.h.

---
 net/mptcp/protocol.c | 7 ++++++-
 net/mptcp/protocol.h | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
net/mptcp/protocol.c +6 -1
index ac94225489f8..6f662e7d6ff9 100644
@@ -55,6 +55,11 @@ u64 mptcp_wnd_end(const struct mptcp_sock *msk)
 	return READ_ONCE(msk->wnd_end);
 }
 
+u64 mptcp_snd_nxt(const struct mptcp_sock *msk)
+{
+	return READ_ONCE(msk->snd_nxt);
+}
+
 static const struct proto_ops *mptcp_fallback_tcp_ops(const struct sock *sk)
 {
 #if IS_ENABLED(CONFIG_MPTCP_IPV6)
@@ -1465,7 +1470,7 @@ struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 	if (!ssk || !sk_stream_memory_free(ssk))
 		return NULL;
 
-	burst = min_t(int, MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
+	burst = min_t(int, MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - mptcp_snd_nxt(msk));
 	wmem = READ_ONCE(ssk->sk_wmem_queued);
 	if (!burst)
 		return ssk;
net/mptcp/protocol.h +1 -0
index 19d60b6d5b45..26b010c88773 100644
@@ -706,6 +706,7 @@ void mptcp_subflow_reset(struct sock *ssk);
 void mptcp_subflow_queue_clean(struct sock *sk, struct sock *ssk);
 void mptcp_sock_graft(struct sock *sk, struct socket *parent);
 u64 mptcp_wnd_end(const struct mptcp_sock *msk);
+u64 mptcp_snd_nxt(const struct mptcp_sock *msk);
 void mptcp_set_timeout(struct sock *sk);
 bool bpf_mptcp_subflow_queues_empty(struct sock *sk);
 struct mptcp_subflow_context *
Signature · 1 line
2.43.0
maintainer
#2 lore raw

Re: [PATCH mptcp-next 1/3] mptcp: add mptcp_snd_nxt helper

Hi Geliang,

Thank you for this patch!

Quoted 4 lines · Geliang Tang

From: Geliang Tang <[email protected]>

It's better to get msk->snd_nxt with READ_ONCE() in get_subflow() of the burst scheduler.

Please always explain the reason → "It's better": OK, but why is it better? :)

Can we currently have a data race with the in-kernel scheduler? Is this field being read and written locklessly?

If yes, this should be explained in the commit message, target -net, and have a proper 'Fixes' tag.

Quoted 4 lines

Similar to mptcp_wnd_end() helper, this patch defines a new helper mptcp_snd_nxt(), and use it in mptcp_subflow_get_send().

It will be used in BPF burst scheduler too, so export it in protocol.h.

(if this patch is for -net, you can remove it).

Cheers,
Matt

Signature · 1 line
Sponsored by the NGI0 Core fund.
author reviewer
#3 lore raw

Re: [PATCH mptcp-next 1/3] mptcp: add mptcp_snd_nxt helper

Hi Matt,

Thanks for your review.

Quoted 11 lines · Matthieu Baerts

Hi Geliang,

Thank you for this patch!

Quoted 5 lines · Geliang Tang

From: Geliang Tang <[email protected]>

It's better to get msk->snd_nxt with READ_ONCE() in get_subflow() of
the burst scheduler.

Quoted 6 lines

Please always explain the reason → "It's better": OK, but why is it better? :)

Can we currently have a data race with the in-kernel scheduler? Is this
field being read and written locklessly?

No. I haven't actually encountered this data race, so this patch can only be considered as a cleanup, not a fixes. This patch was written when I debugged a data race issue a long time ago. It has been in my tree for a long time. I just discovered it recently.

Alternatively, we can simply drop this set.

Thanks,
-Geliang

Quoted 15 lines

If yes, this should be explained in the commit message, target -net, and
have a proper 'Fixes' tag.

Quoted 5 lines

Similar to mptcp_wnd_end() helper, this patch defines a new helper mptcp_snd_nxt(), and use it in mptcp_subflow_get_send().

It will be used in BPF burst scheduler too, so export it in protocol.h.

(if this patch is for -net, you can remove it).

Cheers,
Matt

author reviewer
#4 lore raw

PATCH 2/3 Squash to "bpf: Export more bpf_burst related functions"

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Allow mptcp_snd_nxt() helper to be accessed in BPF.

---
 net/mptcp/bpf.c | 1 +
 1 file changed, 1 insertion(+)
net/mptcp/bpf.c +1 -0
index 9672a70c24b0..9c72f080e7ee 100644
@@ -229,6 +229,7 @@ BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos)
 BTF_ID_FLAGS(func, mptcp_subflow_active)
 BTF_ID_FLAGS(func, mptcp_set_timeout)
 BTF_ID_FLAGS(func, mptcp_wnd_end)
+BTF_ID_FLAGS(func, mptcp_snd_nxt)
 BTF_ID_FLAGS(func, tcp_stream_memory_free)
 BTF_ID_FLAGS(func, bpf_mptcp_subflow_queues_empty)
 BTF_ID_FLAGS(func, mptcp_pm_subflow_chk_stale)
Signature · 1 line
2.43.0
author reviewer
#5 lore raw

Re: [PATCH mptcp-next 2/3] Squash to "bpf: Export more bpf_burst related functions"

Quoted 5 lines · Geliang Tang

From: Geliang Tang <[email protected]>

Allow mptcp_snd_nxt() helper to be accessed in BPF.

Sorry I forgot to update the email address, luckily this is just a squash-to patch. Please help me to update it as my new email address [email protected] when squashing it if this patch is acceptable.

Thanks,
-Geliang

Quoted 16 lines

---
 net/mptcp/bpf.c | 1 +

 1 file changed, 1 insertion(+)
net/mptcp/bpf.c +1 -0
index 9672a70c24b0..9c72f080e7ee 100644
@@ -229,6 +229,7 @@ BTF_ID_FLAGS(func, bpf_mptcp_subflow_ctx_by_pos)
  BTF_ID_FLAGS(func, mptcp_subflow_active)
  BTF_ID_FLAGS(func, mptcp_set_timeout)
  BTF_ID_FLAGS(func, mptcp_wnd_end)
+BTF_ID_FLAGS(func, mptcp_snd_nxt)
  BTF_ID_FLAGS(func, tcp_stream_memory_free)
  BTF_ID_FLAGS(func, bpf_mptcp_subflow_queues_empty)
  BTF_ID_FLAGS(func, mptcp_pm_subflow_chk_stale)
author reviewer
#6 lore raw

PATCH 3/3 Squash to "selftests/bpf: Add bpf_burst scheduler"

In reply to Geliang Tang

From: Geliang Tang <[email protected]>

Use mptcp_snd_nxt in bpf_burst.

---
 tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/progs/mptcp_bpf_burst.c +2 -1
index 0026587a94df..a8506cddcefe 100644
@@ -19,6 +19,7 @@ struct bpf_subflow_send_info {
 extern bool mptcp_subflow_active(struct mptcp_subflow_context *subflow) __ksym;
 extern void mptcp_set_timeout(struct sock *sk) __ksym;
 extern __u64 mptcp_wnd_end(const struct mptcp_sock *msk) __ksym;
+extern __u64 mptcp_snd_nxt(const struct mptcp_sock *msk) __ksym;
 extern bool tcp_stream_memory_free(const struct sock *sk, int wake) __ksym;
 extern bool bpf_mptcp_subflow_queues_empty(struct sock *sk) __ksym;
 extern void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk) __ksym;
@@ -121,7 +122,7 @@ static int bpf_burst_get_send(struct mptcp_sock *msk,
 	if (!ssk || !sk_stream_memory_free(ssk))
 		return -1;
 
-	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
+	burst = min(MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - mptcp_snd_nxt(msk));
 	wmem = ssk->sk_wmem_queued;
 	if (!burst)
 		goto out;
Signature · 1 line
2.43.0
#7 lore raw
In reply to Geliang Tang

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Success! ✅
- KVM Validation: btf (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/9875728508

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/f8f3a07b51e8
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=870130

If there are some issues, you can reproduce them using the same environment as the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

https://github.com/multipath-tcp/mptcp-upstream-virtme-docker

Please note that despite all the efforts that have been already done to have a stable tests suite when executed on a public CI like here, it is possible some reported issues are not due to your modifications. Still, do not hesitate to help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)