Skip to contentSkip to conversation

slub_kunit: add a test case for {kmalloc,kfree}_nolock

PATCH Superseded Harry Yoo (Oracle) 3 participants 8 messages Mar 30 – Apr 3

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Harry Yoo (Oracle)

author maintainer
#0 lore raw

cover slub_kunit: add a test case for {kmalloc,kfree}_nolock

This came up while developing kvfree_rcu_nolock(), but even without that it's worth adding a test case. Let's add it in advance.

Probably a similar test case for the page allocator is worth adding but I'm not sure where the right place would be.

While at it, also add the missing MAINTAINERS entry for slub_kunit.c.

Harry Yoo (Oracle) (2):
MAINTAINERS: add lib/tests/slub_kunit.c to SLAB ALLOCATOR section lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

 MAINTAINERS            |  1 +
 lib/tests/slub_kunit.c | 88 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)

base-commit: dbc785fb4d9cfa0115e69e614fe20b4ab90f8366

Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH 1/2 MAINTAINERS: add lib/tests/slub_kunit.c to SLAB ALLOCATOR section

The slub_kunit module has been maintained by SLAB ALLOCATOR folks, but is missing in the MAINTAINERS file. Add the missing entry.

---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index 96ea84948d76..5f5add4f72a0 100644
@@ -24354,6 +24354,7 @@ F:	Documentation/admin-guide/mm/slab.rst
 F:	Documentation/mm/slab.rst
 F:	include/linux/mempool.h
 F:	include/linux/slab.h
+F:	lib/tests/slub_kunit.c
 F:	mm/failslab.c
 F:	mm/mempool.c
 F:	mm/slab.h
Signature · 1 line
2.43.0
author maintainer
#3 lore raw

PATCH 2/2 lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

In reply to Harry Yoo (Oracle)

Testing invocation of {kmalloc,kfree}_nolock() during kmalloc() or kfree() is tricky, and it is even harder to ensure that slowpaths are properly tested. Lack of such testing has led to late discovery of the bug fixed by commit a1e244a9f177 ("mm/slab: use prandom if !allow_spin").

Add a slub_kunit test that allocates and frees objects in a tight loop while a perf event triggers NMIs on the same task, invoking {kmalloc,kfree}_nolock() from the NMI handler.

---
 lib/tests/slub_kunit.c | 88 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
lib/tests/slub_kunit.c +88 -0
index 848b682a2d70..2ed5f90a748a 100644
@@ -7,6 +7,7 @@
 #include <linux/kernel.h>
 #include <linux/rcupdate.h>
 #include <linux/delay.h>
+#include <linux/perf_event.h>
 #include "../mm/slab.h"
 
 static struct kunit_resource resource;
@@ -291,6 +292,92 @@ static void test_krealloc_redzone_zeroing(struct kunit *test)
 	kmem_cache_destroy(s);
 }
 
+#define REPEAT_TIMES 1000
+#define LOOP_SIZE 1000
+static void *objects[LOOP_SIZE];
+
+struct nmi_context {
+	struct kunit *test;
+	int callback_count;
+	int alloc_ok;
+	int alloc_fail;
+	struct perf_event *event;
+};
+
+static struct perf_event_attr nmi_hw_attr = {
+	.type = PERF_TYPE_HARDWARE,
+	.config = PERF_COUNT_HW_CPU_CYCLES,
+	.size = sizeof(struct perf_event_attr),
+	.pinned = 1,
+	.disabled = 1,
+	.freq = 1,
+	.sample_freq = 100000,
+};
+
+static void kmalloc_kfree_nolock_in_nmi(struct perf_event *event,
+					struct perf_sample_data *data,
+					struct pt_regs *regs)
+{
+	void *objp;
+	gfp_t gfp;
+	struct nmi_context *ctx = event->overflow_handler_context;
+
+	/* __GFP_ACCOUNT to test kmalloc_nolock() in alloc_slab_obj_exts() */
+	gfp = (ctx->callback_count % 2) ? 0 : __GFP_ACCOUNT;
+	objp = kmalloc_nolock(64, gfp, NUMA_NO_NODE);
+
+	if (objp)
+		ctx->alloc_ok++;
+	else
+		ctx->alloc_fail++;
+
+	kfree_nolock(objp);
+	ctx->callback_count++;
+}
+
+static void test_kmalloc_kfree_nolock(struct kunit *test)
+{
+	int i, j;
+	struct nmi_context ctx = { .test = test };
+	struct perf_event *event;
+	bool alloc_fail = false;
+
+	event = perf_event_create_kernel_counter(&nmi_hw_attr, -1, current,
+						 kmalloc_kfree_nolock_in_nmi,
+						 &ctx);
+	if (IS_ERR(event))
+		kunit_skip(test, "Failed to create perf event");
+	ctx.event = event;
+	perf_event_enable(ctx.event);
+	for (i = 0; i < REPEAT_TIMES; i++) {
+		for (j = 0; j < LOOP_SIZE; j++) {
+			gfp_t gfp = (i % 2) ? 0 : __GFP_ACCOUNT;
+
+			objects[j] = kmalloc(64, gfp);
+			if (!objects[j]) {
+				j--;
+				while (j >= 0)
+					kfree(objects[j--]);
+				alloc_fail = true;
+				goto cleanup;
+			}
+		}
+		for (j = 0; j < LOOP_SIZE; j++)
+			kfree(objects[j]);
+	}
+
+cleanup:
+	perf_event_disable(ctx.event);
+	perf_event_release_kernel(ctx.event);
+
+	kunit_info(test, "callback_count: %d, alloc_ok: %d, alloc_fail: %d\n",
+		   ctx.callback_count, ctx.alloc_ok, ctx.alloc_fail);
+
+	if (alloc_fail)
+		kunit_skip(test, "Allocation failed");
+	KUNIT_EXPECT_EQ(test, 0, slab_errors);
+}
+
 static int test_init(struct kunit *test)
 {
 	slab_errors = 0;
@@ -315,6 +402,7 @@ static struct kunit_case test_cases[] = {
 	KUNIT_CASE(test_kfree_rcu_wq_destroy),
 	KUNIT_CASE(test_leak_destroy),
 	KUNIT_CASE(test_krealloc_redzone_zeroing),
+	KUNIT_CASE_SLOW(test_kmalloc_kfree_nolock),
 	{}
 };
 
Signature · 1 line
2.43.0
reviewer Acked-by on v1 2/2
#4 lore raw

Re: [PATCH V1 2/2] lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

In reply to Harry Yoo (Oracle)
Quoted 11 lines · Harry Yoo (Oracle)

Testing invocation of {kmalloc,kfree}_nolock() during kmalloc() or kfree() is tricky, and it is even harder to ensure that slowpaths are properly tested. Lack of such testing has led to late discovery of the bug fixed by commit a1e244a9f177 ("mm/slab: use prandom if !allow_spin").

Add a slub_kunit test that allocates and frees objects in a tight loop while a perf event triggers NMIs on the same task, invoking {kmalloc,kfree}_nolock() from the NMI handler.

author maintainer
#5 lore raw

Re: [PATCH V1 2/2] lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

In reply to Harry Yoo (Oracle)
Quoted 26 lines · Harry Yoo (Oracle)

Testing invocation of {kmalloc,kfree}_nolock() during kmalloc() or kfree() is tricky, and it is even harder to ensure that slowpaths are properly tested. Lack of such testing has led to late discovery of the bug fixed by commit a1e244a9f177 ("mm/slab: use prandom if !allow_spin").

Add a slub_kunit test that allocates and frees objects in a tight loop while a perf event triggers NMIs on the same task, invoking {kmalloc,kfree}_nolock() from the NMI handler.

---
 lib/tests/slub_kunit.c | 88 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
lib/tests/slub_kunit.c +7 -0
index 848b682a2d70..2ed5f90a748a 100644
+static void test_kmalloc_kfree_nolock(struct kunit *test)
+{
+	int i, j;
+	struct nmi_context ctx = { .test = test };
+	struct perf_event *event;
+	bool alloc_fail = false;
+

lib/tests/slub_kunit.c static void test_krealloc_redzone_zeroing(struct kunit *test) · line 354

Quoted 10 lines
lib/tests/slub_kunit.c +10 -0
+	event = perf_event_create_kernel_counter(&nmi_hw_attr, -1, current,
+						 kmalloc_kfree_nolock_in_nmi,
+						 &ctx);
+	if (IS_ERR(event))
+		kunit_skip(test, "Failed to create perf event");
+	ctx.event = event;
+	perf_event_enable(ctx.event);
+	for (i = 0; i < REPEAT_TIMES; i++) {
+		for (j = 0; j < LOOP_SIZE; j++) {
+			gfp_t gfp = (i % 2) ? 0 : __GFP_ACCOUNT;

One thing sashiko pointed out [1]: '0' as gfp for kmalloc() is unnecessarily restrictive. (it's not kmalloc_nolock()!), it should be:

gfp_t gfp = (i % 2) ? GFP_KERNEL : GFP_KERNEL_ACCOUNT;

[1] https://sashiko.dev/#/patchset/20260330120517.104743-1-harry%40kernel.org

Signature · 2 lines
Cheers,
Harry / Hyeonggon
#6 lore raw

Re: [PATCH V1 2/2] lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

In reply to Harry Yoo (Oracle)
Quoted 11 lines · Harry Yoo (Oracle)

Testing invocation of {kmalloc,kfree}_nolock() during kmalloc() or kfree() is tricky, and it is even harder to ensure that slowpaths are properly tested. Lack of such testing has led to late discovery of the bug fixed by commit a1e244a9f177 ("mm/slab: use prandom if !allow_spin").

Add a slub_kunit test that allocates and frees objects in a tight loop while a perf event triggers NMIs on the same task, invoking {kmalloc,kfree}_nolock() from the NMI handler.

perf_event_create_kernel_counter() seems to only exist with CONFIG_PERF_EVENTS, there's not even some "return -EINVAL" skeleton otherwise, AFAICS. Didn't check the various other definitions, but guess we'll need to put some/all of this behind #ifdef's then?

Quoted 117 lines
---
 lib/tests/slub_kunit.c | 88 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
lib/tests/slub_kunit.c +88 -0
index 848b682a2d70..2ed5f90a748a 100644
@@ -7,6 +7,7 @@
 #include <linux/kernel.h>
 #include <linux/rcupdate.h>
 #include <linux/delay.h>
+#include <linux/perf_event.h>
 #include "../mm/slab.h"
 
 static struct kunit_resource resource;
@@ -291,6 +292,92 @@ static void test_krealloc_redzone_zeroing(struct kunit *test)
 	kmem_cache_destroy(s);
 }
 
+#define REPEAT_TIMES 1000
+#define LOOP_SIZE 1000
+static void *objects[LOOP_SIZE];
+
+struct nmi_context {
+	struct kunit *test;
+	int callback_count;
+	int alloc_ok;
+	int alloc_fail;
+	struct perf_event *event;
+};
+
+static struct perf_event_attr nmi_hw_attr = {
+	.type = PERF_TYPE_HARDWARE,
+	.config = PERF_COUNT_HW_CPU_CYCLES,
+	.size = sizeof(struct perf_event_attr),
+	.pinned = 1,
+	.disabled = 1,
+	.freq = 1,
+	.sample_freq = 100000,
+};
+
+static void kmalloc_kfree_nolock_in_nmi(struct perf_event *event,
+					struct perf_sample_data *data,
+					struct pt_regs *regs)
+{
+	void *objp;
+	gfp_t gfp;
+	struct nmi_context *ctx = event->overflow_handler_context;
+
+	/* __GFP_ACCOUNT to test kmalloc_nolock() in alloc_slab_obj_exts() */
+	gfp = (ctx->callback_count % 2) ? 0 : __GFP_ACCOUNT;
+	objp = kmalloc_nolock(64, gfp, NUMA_NO_NODE);
+
+	if (objp)
+		ctx->alloc_ok++;
+	else
+		ctx->alloc_fail++;
+
+	kfree_nolock(objp);
+	ctx->callback_count++;
+}
+
+static void test_kmalloc_kfree_nolock(struct kunit *test)
+{
+	int i, j;
+	struct nmi_context ctx = { .test = test };
+	struct perf_event *event;
+	bool alloc_fail = false;
+
+	event = perf_event_create_kernel_counter(&nmi_hw_attr, -1, current,
+						 kmalloc_kfree_nolock_in_nmi,
+						 &ctx);
+	if (IS_ERR(event))
+		kunit_skip(test, "Failed to create perf event");
+	ctx.event = event;
+	perf_event_enable(ctx.event);
+	for (i = 0; i < REPEAT_TIMES; i++) {
+		for (j = 0; j < LOOP_SIZE; j++) {
+			gfp_t gfp = (i % 2) ? 0 : __GFP_ACCOUNT;
+
+			objects[j] = kmalloc(64, gfp);
+			if (!objects[j]) {
+				j--;
+				while (j >= 0)
+					kfree(objects[j--]);
+				alloc_fail = true;
+				goto cleanup;
+			}
+		}
+		for (j = 0; j < LOOP_SIZE; j++)
+			kfree(objects[j]);
+	}
+
+cleanup:
+	perf_event_disable(ctx.event);
+	perf_event_release_kernel(ctx.event);
+
+	kunit_info(test, "callback_count: %d, alloc_ok: %d, alloc_fail: %d\n",
+		   ctx.callback_count, ctx.alloc_ok, ctx.alloc_fail);
+
+	if (alloc_fail)
+		kunit_skip(test, "Allocation failed");
+	KUNIT_EXPECT_EQ(test, 0, slab_errors);
+}
+
 static int test_init(struct kunit *test)
 {
 	slab_errors = 0;
@@ -315,6 +402,7 @@ static struct kunit_case test_cases[] = {
 	KUNIT_CASE(test_kfree_rcu_wq_destroy),
 	KUNIT_CASE(test_leak_destroy),
 	KUNIT_CASE(test_krealloc_redzone_zeroing),
+	KUNIT_CASE_SLOW(test_kmalloc_kfree_nolock),
 	{}
 };
 
author maintainer
#7 lore raw

Re: [PATCH V1 2/2] lib/tests/slub_kunit: add a test case for {kmalloc,kfree}_nolock

Quoted 7 lines · Vlastimil Babka (SUSE)
Quoted 6 lines · Harry Yoo (Oracle)

Testing invocation of {kmalloc,kfree}_nolock() during kmalloc() or kfree() is tricky, and it is even harder to ensure that slowpaths are properly tested. Lack of such testing has led to late discovery of the bug fixed by commit a1e244a9f177 ("mm/slab: use prandom if !allow_spin").

Quoted 8 lines
Quoted 5 lines

Add a slub_kunit test that allocates and frees objects in a tight loop while a perf event triggers NMIs on the same task, invoking {kmalloc,kfree}_nolock() from the NMI handler.

perf_event_create_kernel_counter() seems to only exist with CONFIG_PERF_EVENTS,

Thanks for catching that!

Quoted 2 lines

there's not even some "return -EINVAL" skeleton
otherwise, AFAICS.

Right.

Quoted 2 lines

Didn't check the various other definitions, but guess we'll need to put some/all of this behind #ifdef's then?

I'll make sure it builds w/o CONFIG_PERF_EVENTS and send v2.

Signature · 2 lines
Cheers,
Harry / Hyeonggon