Skip to contentSkip to conversation

mm/damon/tests/vaddr-kunit: don't use mas_lock for MM_MT_FLAGS-initialized maple tree

PATCH In review SeongJae Park 3 participants 17 messages Sep 4, 2024 – Sep 5, 2024

Subsystems (3)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

PATCH 1/1 mm/damon/tests/vaddr-kunit: don't use mas_lock for MM_MT_FLAGS-initialized maple tree

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
 mm/damon/tests/vaddr-kunit.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
 static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
 			ssize_t nr_vmas)
 {
-	int i, ret = -ENOMEM;
+	int i;
 	MA_STATE(mas, mt, 0, 0);
 
 	if (!nr_vmas)
 		return 0;
 
-	mas_lock(&mas);
 	for (i = 0; i < nr_vmas; i++) {
 		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
 		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
 	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
 }
 
 /*
Signature · 1 line
2.39.2
#1 lore raw
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

You can't do this, lockdep will tell you this is wrong. We need a lock and to use the lock for writes.

I'd suggest using different flags so the spinlock is used.

Quoted 43 lines
Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
 mm/damon/tests/vaddr-kunit.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
 static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
 			ssize_t nr_vmas)
 {
-	int i, ret = -ENOMEM;
+	int i;
 	MA_STATE(mas, mt, 0, 0);
 
 	if (!nr_vmas)
 		return 0;
 
-	mas_lock(&mas);
 	for (i = 0; i < nr_vmas; i++) {
 		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
 		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
 	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
 }
 
 /*
Signature · 1 line
2.39.2
author maintainer
#2 lore raw
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

Thanks,
SJ

Quoted 44 lines
Quoted 44 lines
Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
 mm/damon/tests/vaddr-kunit.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
 static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
 			ssize_t nr_vmas)
 {
-	int i, ret = -ENOMEM;
+	int i;
 	MA_STATE(mas, mt, 0, 0);
 
 	if (!nr_vmas)
 		return 0;
 
-	mas_lock(&mas);
 	for (i = 0; i < nr_vmas; i++) {
 		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
 		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
 	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
 }
 
 /*
Signature · 1 line
2.39.2
author maintainer
#3 lore raw
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

```
--- a/mm/damon/tests/vaddr-kunit.h
+++ b/mm/damon/tests/vaddr-kunit.h
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
        };

-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
        if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                kunit_skip(test, "Failed to create VMA tree");
```

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.

The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the MTREE_INIT_EXT() #define, which takes an external lock as an argument.

(from Documentation/core-api/maple_tree.rst)

I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.

Thanks,
SJ

Quoted 49 lines

Thanks,
SJ

Quoted 44 lines
Quoted 44 lines
Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
 mm/damon/tests/vaddr-kunit.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
 static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
 			ssize_t nr_vmas)
 {
-	int i, ret = -ENOMEM;
+	int i;
 	MA_STATE(mas, mt, 0, 0);
 
 	if (!nr_vmas)
 		return 0;
 
-	mas_lock(&mas);
 	for (i = 0; i < nr_vmas; i++) {
 		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
 		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
 	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
 }
 
 /*
Signature · 1 line
2.39.2
#4 lore raw
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

```

mm/damon/tests/vaddr-kunit.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                 (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
         };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
         if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                 kunit_skip(test, "Failed to create VMA tree");

```

Quoted 4 lines

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

Same here. That is why I gave up after trying MT_FLAGS_ALLOC_RANGE and "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree with MT_FLAGS_ALLOC_RANGE (but not MT_FLAGS_USE_RCU) would trigger rcu warnings.

Guenter

#5 lore raw
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

```

mm/damon/tests/vaddr-kunit.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                 (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
         };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
         if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                 kunit_skip(test, "Failed to create VMA tree");

```

Quoted 3 lines
Quoted 3 lines

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

Again, I'd use the flags "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU" because that gets you the gap tracking that may be necessary for tests in the future - it's closer to the MM_MT_FLAGS, so maybe some mm function you use depends on that.

Quoted 6 lines
Quoted 1 line

Same here. That is why I gave up after trying MT_FLAGS_ALLOC_RANGE and "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree with MT_FLAGS_ALLOC_RANGE (but not MT_FLAGS_USE_RCU) would trigger rcu warnings.

Thanks, I'll add that to my list of things to do.

Regards,
Liam

author maintainer
#6 lore raw
Quoted 48 lines · On Tue, 3 Sep 2024 22:43:40 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

```

mm/damon/tests/vaddr-kunit.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                 (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
         };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
         if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                 kunit_skip(test, "Failed to create VMA tree");

```

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

Again, I'd use the flags "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU" because that gets you the gap tracking that may be necessary for tests in the future - it's closer to the MM_MT_FLAGS, so maybe some mm function you use depends on that.

Thank you for the nice suggestion with the rationales. Just posted the v2 following it: https://lore.kernel.org/[email protected]

Quoted 9 lines
Quoted 7 lines
Quoted 2 lines

Same here. That is why I gave up after trying MT_FLAGS_ALLOC_RANGE and "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree with MT_FLAGS_ALLOC_RANGE (but not MT_FLAGS_USE_RCU) would trigger rcu warnings.

Thanks, I'll add that to my list of things to do.

Thank you. I agree that's somewhat we can visit separately.

FYI, I was also unable to reproduce rcu warnings with my v2 patch on my setup. I will also try to use Guenter's more detailed repro
(https://lore.kernel.org/[email protected]).

Thanks,
SJ

Quoted 3 lines

Regards,
Liam

#7 lore raw
In reply to SeongJae Park
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

I would recommend those two and just use the spinlock.

Quoted 84 lines

```

mm/damon/tests/vaddr-kunit.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
        };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
        if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                kunit_skip(test, "Failed to create VMA tree");

```

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.

The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the MTREE_INIT_EXT() #define, which takes an external lock as an argument.

(from Documentation/core-api/maple_tree.rst)

I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.

Thanks,
SJ

Quoted 49 lines

Thanks,
SJ

Quoted 44 lines
Quoted 44 lines
Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
 mm/damon/tests/vaddr-kunit.h | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
 static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
 			ssize_t nr_vmas)
 {
-	int i, ret = -ENOMEM;
+	int i;
 	MA_STATE(mas, mt, 0, 0);
 
 	if (!nr_vmas)
 		return 0;
 
-	mas_lock(&mas);
 	for (i = 0; i < nr_vmas; i++) {
 		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
 		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
 	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
 }
 
 /*
Signature · 1 line
2.39.2
#8 lore raw
Quoted 37 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Quoted 9 lines

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Guenter

Quoted 84 lines
Quoted 84 lines

```

mm/damon/tests/vaddr-kunit.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                 (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
         };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init(&mm.mm_mt);
         if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                 kunit_skip(test, "Failed to create VMA tree");

```

And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.

FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.

The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the MTREE_INIT_EXT() #define, which takes an external lock as an argument.

(from Documentation/core-api/maple_tree.rst)

I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.

Thanks,
SJ

Quoted 49 lines

Thanks,
SJ

Quoted 44 lines
Quoted 44 lines
Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")
---
  mm/damon/tests/vaddr-kunit.h | 10 +++-------
  1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
index 83626483f82b..c6c7e0e0ab07 100644
@@ -17,23 +17,19 @@
  static int __link_vmas(struct maple_tree *mt, struct vm_area_struct *vmas,
  			ssize_t nr_vmas)
  {
-	int i, ret = -ENOMEM;
+	int i;
  	MA_STATE(mas, mt, 0, 0);
 
  	if (!nr_vmas)
  		return 0;
 
-	mas_lock(&mas);
  	for (i = 0; i < nr_vmas; i++) {
  		mas_set_range(&mas, vmas[i].vm_start, vmas[i].vm_end - 1);
  		if (mas_store_gfp(&mas, &vmas[i], GFP_KERNEL))
-			goto failed;
+			return -ENOMEM;
  	}
 
-	ret = 0;
-failed:
-	mas_unlock(&mas);
-	return ret;
+	return 0;
  }
 
  /*
Signature · 1 line
2.39.2
#9 lore raw
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 5 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Just to be clear, you changed the init flags and kept the mas_lock() and mas_unlock() ?

Thanks,
Liam

#10 lore raw
In reply to Guenter Roeck
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 6 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

Thanks,
Liam

#11 lore raw
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 6 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Quoted 8 lines

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

I tested again, and I still see

[    6.233483] ok 4 damon
[    6.234190]     KTAP version 1
[    6.234263]     # Subtest: damon-operations
[    6.234335]     # module: vaddr
[    6.234384]     1..6
[    6.235726]
[    6.235931] =============================
[    6.236018] WARNING: suspicious RCU usage
[    6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G                 N
[    6.236398] -----------------------------
[    6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage!
[    6.236579]
[    6.236579] other info that might help us debug this:
[    6.236579]
[    6.236738]
[    6.236738] rcu_scheduler_active = 2, debug_locks = 1
[    6.237039] no locks held by kunit_try_catch/208.
[    6.237166]
[    6.237166] stack backtrace:
[    6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G                 N 6.11.0-rc6-00029-gda66250b210f-dirty #1
[    6.237629] Tainted: [N]=TEST
[    6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[    6.238065] Call Trace:
[    6.238233]  <TASK>
[    6.238547]  dump_stack_lvl+0x9e/0xe0
[    6.239473]  lockdep_rcu_suspicious+0x145/0x1b0
[    6.239621]  mas_walk+0x19f/0x1d0
[    6.239765]  mas_find+0xb5/0x150
[    6.239873]  __damon_va_three_regions+0x7e/0x130
[    6.240039]  damon_test_three_regions_in_vmas+0x1ea/0x480
[    6.240551]  ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10
[    6.240712]  kunit_try_run_case+0x93/0x190
[    6.240850]  ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10
[    6.240990]  kunit_generic_run_threadfn_adapter+0x1c/0x40
[    6.241124]  kthread+0xdd/0x110
[    6.241256]  ? __pfx_kthread+0x10/0x10
[    6.241368]  ret_from_fork+0x2f/0x50
[    6.241468]  ? __pfx_kthread+0x10/0x10
[    6.241573]  ret_from_fork_asm+0x1a/0x30
[    6.241765]  </TASK>
[    6.242180]
[    6.242270] =============================
[    6.242375] WARNING: suspicious RCU usage
[    6.242478] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G                 N
[    6.242634] -----------------------------
[    6.242734] lib/maple_tree.c:788 suspicious rcu_dereference_check() usage!
[    6.242955]
[    6.242955] other info that might help us debug this:
[    6.242955]
[    6.243098]
[    6.243098] rcu_scheduler_active = 2, debug_locks = 1
[    6.243215] no locks held by kunit_try_catch/208.
[    6.243331]
[    6.243331] stack backtrace:
[    6.243420] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G                 N 6.11.0-rc6-00029-gda66250b210f-dirty #1
[    6.243599] Tainted: [N]=TEST
[    6.243665] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[    6.243844] Call Trace:
[    6.243907]  <TASK>
[    6.243961]  dump_stack_lvl+0x9e/0xe0
[    6.244032]  lockdep_rcu_suspicious+0x145/0x1b0
[    6.244121]  mtree_range_walk+0x2b9/0x350
[    6.244211]  mas_walk+0x107/0x1d0
[    6.244278]  mas_find+0xb5/0x150
[    6.244341]  __damon_va_three_regions+0x7e/0x130
[    6.244445]  damon_test_three_regions_in_vmas+0x1ea/0x480
[    6.244770]  ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10
[    6.244865]  kunit_try_run_case+0x93/0x190
[    6.244952]  ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10
[    6.245044]  kunit_generic_run_threadfn_adapter+0x1c/0x40
[    6.245130]  kthread+0xdd/0x110
[    6.245191]  ? __pfx_kthread+0x10/0x10
[    6.245262]  ret_from_fork+0x2f/0x50
[    6.245326]  ? __pfx_kthread+0x10/0x10
[    6.245394]  ret_from_fork_asm+0x1a/0x30
[    6.245497]  </TASK>
[    6.246605]     # damon_test_three_regions_in_vmas: pass:1 fail:0 skip:0 total:1
[    6.246668]     ok 1 damon_test_three_regions_in_vmas

This is with

mm/damon/vaddr-test.h +1 -1
index 83626483f82b..a339d117150f 100644
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
                (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
        };
 
-       mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+       mt_init_flags(&mm.mm_mt, MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU);
        if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
                kunit_skip(test, "Failed to create VMA tree");
 

I'll put it all together and make it available, but that will have to wait until tomorrow.

Thanks,
Guenter

#12 lore raw
Quoted 87 lines · Guenter Roeck
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 6 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Quoted 8 lines

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

I tested again, and I still see

[    6.233483] ok 4 damon
[    6.234190]     KTAP version 1
[    6.234263]     # Subtest: damon-operations
[    6.234335]     # module: vaddr
[    6.234384]     1..6
[    6.235726]
[    6.235931] =============================
[    6.236018] WARNING: suspicious RCU usage
[    6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G                 N
[    6.236398] -----------------------------
[    6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage!
[    6.236579]
[    6.236579] other info that might help us debug this:
[    6.236579]
[    6.236738]
[    6.236738] rcu_scheduler_active = 2, debug_locks = 1
[    6.237039] no locks held by kunit_try_catch/208.
[    6.237166]
[    6.237166] stack backtrace:
[    6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G                 N 6.11.0-rc6-00029-gda66250b210f-dirty #1
[    6.237629] Tainted: [N]=TEST
Quoted 8 lines
[    6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[    6.238065] Call Trace:
[    6.238233]  <TASK>
[    6.238547]  dump_stack_lvl+0x9e/0xe0
[    6.239473]  lockdep_rcu_suspicious+0x145/0x1b0
[    6.239621]  mas_walk+0x19f/0x1d0
[    6.239765]  mas_find+0xb5/0x150
[    6.239873]  __damon_va_three_regions+0x7e/0x130

This function isn't taking the rcu read lock while iterating the tree.

Try this:

mm/damon/vaddr.c +2 -0
index b0e8b361891d..08cfd22b5249 100644
@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
         * If this is too slow, it can be optimised to examine the maple
         * tree gaps.
         */
+       rcu_read_lock();
        for_each_vma(vmi, vma) {
                unsigned long gap;
 
@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
 next:
                prev = vma;
        }
+       rcu_read_unlock();
 
        if (!sz_range(&second_gap) || !sz_range(&first_gap))
                return -EINVAL;
#13 lore raw
Quoted 115 lines · Liam R. Howlett
Quoted 87 lines · Guenter Roeck
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 6 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Quoted 8 lines

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

I tested again, and I still see

[    6.233483] ok 4 damon
[    6.234190]     KTAP version 1
[    6.234263]     # Subtest: damon-operations
[    6.234335]     # module: vaddr
[    6.234384]     1..6
[    6.235726]
[    6.235931] =============================
[    6.236018] WARNING: suspicious RCU usage
[    6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G                 N
[    6.236398] -----------------------------
[    6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage!
[    6.236579]
[    6.236579] other info that might help us debug this:
[    6.236579]
[    6.236738]
[    6.236738] rcu_scheduler_active = 2, debug_locks = 1
[    6.237039] no locks held by kunit_try_catch/208.
[    6.237166]
[    6.237166] stack backtrace:
[    6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G                 N 6.11.0-rc6-00029-gda66250b210f-dirty #1
[    6.237629] Tainted: [N]=TEST
Quoted 8 lines
[    6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[    6.238065] Call Trace:
[    6.238233]  <TASK>
[    6.238547]  dump_stack_lvl+0x9e/0xe0
[    6.239473]  lockdep_rcu_suspicious+0x145/0x1b0
[    6.239621]  mas_walk+0x19f/0x1d0
[    6.239765]  mas_find+0xb5/0x150
[    6.239873]  __damon_va_three_regions+0x7e/0x130

This function isn't taking the rcu read lock while iterating the tree.

Try this:

mm/damon/vaddr.c +1 -0
index b0e8b361891d..08cfd22b5249 100644
@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
          * If this is too slow, it can be optimised to examine the maple
          * tree gaps.
          */
+       rcu_read_lock();
         for_each_vma(vmi, vma) {
                 unsigned long gap;
 

mm/damon/vaddr.c static int __damon_va_three_regions(struct mm_struct *mm, · line 153

Quoted 7 lines
mm/damon/vaddr.c +1 -0
@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
  next:
                 prev = vma;
         }
+       rcu_read_unlock();
 
         if (!sz_range(&second_gap) || !sz_range(&first_gap))
                 return -EINVAL;

Yes, that fixes the problem for me.

Thanks,
Guenter

author maintainer
#14 lore raw
Quoted 11 lines · On Wed, 4 Sep 2024 12:56:33 -0700 Guenter Roeck <[email protected]> wrote:
Quoted 10 lines · Liam R. Howlett
Quoted 9 lines · Guenter Roeck
Quoted 8 lines · Liam R. Howlett
Quoted 7 lines · Guenter Roeck
Quoted 6 lines · Liam R. Howlett
Quoted 5 lines · SeongJae Park
Quoted 3 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 1 line · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…

* SeongJae Park <[email protected]> [240903 20:45]:

[...]

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 8 lines

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

I tested again, and I still see

[    6.233483] ok 4 damon
[    6.234190]     KTAP version 1
[    6.234263]     # Subtest: damon-operations
[    6.234335]     # module: vaddr
[    6.234384]     1..6
[    6.235726]
[    6.235931] =============================
[    6.236018] WARNING: suspicious RCU usage
[    6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G                 N
[    6.236398] -----------------------------
[    6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage!
[    6.236579]
[    6.236579] other info that might help us debug this:
[    6.236579]
[    6.236738]
[    6.236738] rcu_scheduler_active = 2, debug_locks = 1
[    6.237039] no locks held by kunit_try_catch/208.
[    6.237166]
[    6.237166] stack backtrace:
[    6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G                 N 6.11.0-rc6-00029-gda66250b210f-dirty #1
[    6.237629] Tainted: [N]=TEST
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
[    6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[    6.238065] Call Trace:
[    6.238233]  <TASK>
[    6.238547]  dump_stack_lvl+0x9e/0xe0
[    6.239473]  lockdep_rcu_suspicious+0x145/0x1b0
[    6.239621]  mas_walk+0x19f/0x1d0
[    6.239765]  mas_find+0xb5/0x150
[    6.239873]  __damon_va_three_regions+0x7e/0x130

I was able to reproduce this by further enabling PROVE_LOCKING.

Quoted 23 lines
Quoted 23 lines

This function isn't taking the rcu read lock while iterating the tree.

Try this:

mm/damon/vaddr.c +2 -0
index b0e8b361891d..08cfd22b5249 100644
@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
          * If this is too slow, it can be optimised to examine the maple
          * tree gaps.
          */
+       rcu_read_lock();
         for_each_vma(vmi, vma) {
                 unsigned long gap;
 
@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,
  next:
                 prev = vma;
         }
+       rcu_read_unlock();
 
Quoted 6 lines

mm/damon/vaddr.c static int __damon_va_three_regions(struct mm_struct *mm, · line 153

Quoted 3 lines
mm/damon/vaddr.c +0 -0
         if (!sz_range(&second_gap) || !sz_range(&first_gap))
                 return -EINVAL;

Yes, that fixes the problem for me.

Thank you for the fix, Liam. Thank you for the test, Guenter. I also confirmed this fix works on my setup.

I posted the fix as a formal patch:
https://lore.kernel.org/[email protected]

Thanks,
SJ

Quoted 3 lines

Thanks,
Guenter

#15 lore raw
In reply to Liam R. Howlett
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

Quoted 1 line

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 8 lines
Quoted 8 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

That would be the lockdep complaining, so that's good.

Quoted 3 lines

I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.

Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.

Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).

Quoted 6 lines
Quoted 2 lines

I would recommend those two and just use the spinlock.

I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.

Quoted 8 lines

I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:
CONFIG_LOCKDEP=y
CONFIG_DEBUG_SPINLOCK=y

and I don't have any issue with locking in the existing code. How do I recreate this issue?

You should find everything needed to reproduce the problem at
http://server.roeck-us.net/qemu/damon/

Please let me know if you need anything else.

Thanks,
Guenter

Tested-by
#16 lore raw
In reply to SeongJae Park
Quoted 10 lines · SeongJae Park
Quoted 8 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 7 lines · SeongJae Park

damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it

Just in case you need to resend: s/celar/clear/

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.

You can't do this, lockdep will tell you this is wrong.

Hmm, but lockdep was silence on my setup?

Quoted 1 line

We need a lock and to use the lock for writes.

Quoted 9 lines

This code is executed by a single-thread test code. Do we still need the lock?

Quoted 2 lines

I'd suggest using different flags so the spinlock is used.

The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?

Correct. I don't see those messages with your patch. From my perspective, this is

Thanks,
Guenter