Skip to contentSkip to conversation

selftests/bpf: Add timer lockup selftest

PATCH Applied Kumar Kartikeya Dwivedi 3 participants 3 messages Jul 11, 2024

Subsystems (3)
lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

PATCH v2 1/1 selftests/bpf: Add timer lockup selftest

Add a selftest that tries to trigger a situation where two timer callbacks are attempting to cancel each other's timer. By running them continuously, we hit a condition where both run in parallel and cancel each other. Without the fix in the previous patch, this would cause a lockup as hrtimer_cancel on either side will wait for forward progress from the callback.

Ensure that this situation leads to a EDEADLK error.

---
Changelog:
* Add missing timer_lockup__destroy.
* Fix inline declarations (Alexei).
* Shorten pthread_create line (Alexei).
* Fix type of map value parameter to time cb (Alexei).
* Add a counter to skip test if race does not reproduce (Alexei).

---
 .../selftests/bpf/prog_tests/timer_lockup.c   | 86 ++++++++++++++++++
 .../selftests/bpf/progs/timer_lockup.c        | 87 +++++++++++++++++++
 2 files changed, 173 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/timer_lockup.c
 create mode 100644 tools/testing/selftests/bpf/progs/timer_lockup.c
tools/testing/selftests/bpf/prog_tests/timer_lockup.c new file +86 -0
new file mode 100644
index 000000000000..2894fdb7942c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+#define _GNU_SOURCE
+#include <sched.h>
+#include <test_progs.h>
+#include <pthread.h>
+#include <network_helpers.h>
+#include "timer_lockup.skel.h"
+
+static long cpu;
+static int *timer1_err;
+static int *timer2_err;
+static bool skip;
+
+volatile int k = 0;
+static void *timer_lockup_thread(void *arg)
+{
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.data_in = &pkt_v4,
+		.data_size_in = sizeof(pkt_v4),
+		.repeat = 1000,
+	);
+	int prog_fd = *(int *)arg;
+	cpu_set_t cpuset;
+	int i;
+
+	CPU_ZERO(&cpuset);
+	CPU_SET(__sync_fetch_and_add(&cpu, 1), &cpuset);
+	ASSERT_OK(pthread_setaffinity_np(pthread_self(), sizeof(cpuset), &cpuset), "cpu affinity");
+
+	for (i = 0; !READ_ONCE(*timer1_err) && !READ_ONCE(*timer2_err); i++) {
+		bpf_prog_test_run_opts(prog_fd, &opts);
+		/* Skip the test if we can't reproduce the race in a reasonable
+		 * amount of time.
+		 */
+		if (i > 50) {
+			WRITE_ONCE(skip, true);
+			break;
+		}
+	}
+
+	return NULL;
+}
+
+void test_timer_lockup(void)
+{
+	int timer1_prog, timer2_prog;
+	struct timer_lockup *skel;
+	pthread_t thrds[2];
+	void *ret;
+
+	skel = timer_lockup__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "timer_lockup__open_and_load"))
+		return;
+
+	timer1_prog = bpf_program__fd(skel->progs.timer1_prog);
+	timer2_prog = bpf_program__fd(skel->progs.timer2_prog);
+
+	timer1_err = &skel->bss->timer1_err;
+	timer2_err = &skel->bss->timer2_err;
+
+	if (!ASSERT_OK(pthread_create(&thrds[0], NULL, timer_lockup_thread, &timer1_prog),
+		       "pthread_create thread1"))
+		goto out;
+	if (!ASSERT_OK(pthread_create(&thrds[1], NULL, timer_lockup_thread, &timer2_prog),
+		       "pthread_create thread2")) {
+		pthread_exit(&thrds[0]);
+		goto out;
+	}
+
+	pthread_join(thrds[1], &ret);
+	pthread_join(thrds[0], &ret);
+
+	if (skip) {
+		test__skip();
+		goto out;
+	}
+
+	if (*timer1_err != -EDEADLK && *timer1_err != 0)
+		ASSERT_FAIL("timer1_err bad value");
+	if (*timer2_err != -EDEADLK && *timer2_err != 0)
+		ASSERT_FAIL("timer2_err bad value");
+
+out:
+	timer_lockup__destroy(skel);
+	return;
+}
tools/testing/selftests/bpf/progs/timer_lockup.c new file +87 -0

87 lines not on this page · raw message

--
2.43.0

#1 lore raw
This patch was applied to bpf/bpf.git (master) by Daniel Borkmann <[email protected]>:

Hello:

This patch was applied to bpf/bpf.git (master)
by Daniel Borkmann <[email protected]>:

Quoted 8 lines · On Thu, 11 Jul 2024 05:27:09 +0000 you wrote:

Add a selftest that tries to trigger a situation where two timer callbacks are attempting to cancel each other's timer. By running them continuously, we hit a condition where both run in parallel and cancel each other. Without the fix in the previous patch, this would cause a lockup as hrtimer_cancel on either side will wait for forward progress from the callback.

[...]

Here is the summary with links:
- [bpf,v2] selftests/bpf: Add timer lockup selftest

    https://git.kernel.org/bpf/bpf/c/50bd5a0c658d

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
maintainer
#2 lore raw
In reply to Kumar Kartikeya Dwivedi
Quoted 9 lines · Kumar Kartikeya Dwivedi

Add a selftest that tries to trigger a situation where two timer callbacks are attempting to cancel each other's timer. By running them continuously, we hit a condition where both run in parallel and cancel each other. Without the fix in the previous patch, this would cause a lockup as hrtimer_cancel on either side will wait for forward progress from the callback.

Ensure that this situation leads to a EDEADLK error.

Quoted 8 lines

---
Changelog:
* Add missing timer_lockup__destroy.
* Fix inline declarations (Alexei).
* Shorten pthread_create line (Alexei).
* Fix type of map value parameter to time cb (Alexei).
* Add a counter to skip test if race does not reproduce (Alexei).

Did some small cleanups while applying, and ran several times w/o issues. Applied, thanks Kumar!