mm/damon/tests/vaddr-kunit: don't use mas_lock for MM_MT_FLAGS-initialized maple tree
Subsystems (3)
SeongJae Park
maintainerPATCH 1/1 mm/damon/tests/vaddr-kunit: don't use mas_lock for MM_MT_FLAGS-initialized maple tree
damon_test_three_regions_in_vmas() initializes a maple tree with MM_MT_FLAGS. The flags contains MT_FLAGS_LOCK_EXTERN, which means mt_lock of the maple tree will not be used. And therefore the maple tree initialization code skips initialization of the mt_lock. However, __link_vmas(), which adds vmas for test to the maple tree, uses the mt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using the mt_lock as promised.
--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
mm/damon/tests/vaddr-kunit.h +3 -7
@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/*
Signature · 1 line
2.39.2
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.
You can't do this, lockdep will tell you this is wrong. We need a lock and to use the lock for writes.
I'd suggest using different flags so the spinlock is used.
Quoted 43 lines
Reported-by Guenter Roeck <[email protected]>Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")Signed-off-by SeongJae Park <[email protected]>--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)mm/damon/tests/vaddr-kunit.h +3
-7@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/* Signature · 1 line
2.39.2
SeongJae Park
maintainerQuoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.
Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping MT_FLAGS_LOCK_EXTERN from the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?
Thanks,
SJ
Quoted 44 lines
Quoted 44 lines
Reported-by Guenter Roeck <[email protected]>Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")Signed-off-by SeongJae Park <[email protected]>--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)mm/damon/tests/vaddr-kunit.h +3
-7@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/* Signature · 1 line
2.39.2
SeongJae Park
maintainerQuoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?
I was actually thinking replacing the mt_init_flags() with mt_init(), which same to mt_init_flags() with zero flag, like below.
```
--- a/mm/damon/tests/vaddr-kunit.h
+++ b/mm/damon/tests/vaddr-kunit.h
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test)
(struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
};
- mt_init_flags(&mm.mm_mt, MM_MT_FLAGS);
+ mt_init(&mm.mm_mt);
if (__link_vmas(&mm.mm_mt, vmas, ARRAY_SIZE(vmas)))
kunit_skip(test, "Failed to create VMA tree");
```And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.
The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the MTREE_INIT_EXT() #define, which takes an external lock as an argument.
(from Documentation/core-api/maple_tree.rst)
I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.
Thanks,
SJ
Quoted 49 lines
Thanks,
SJQuoted 44 lines
Quoted 44 lines
Reported-by Guenter Roeck <[email protected]>Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")Signed-off-by SeongJae Park <[email protected]>--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)mm/damon/tests/vaddr-kunit.h +3
-7@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/* Signature · 1 line
2.39.2
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.```
mm/damon/tests/vaddr-kunit.h +1
-1@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init (&mm.mm_mt);if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );```
Quoted 4 lines
And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
Same here. That is why I gave up after trying MT_FLAGS_ALLOC_RANGE and "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree with MT_FLAGS_ALLOC_RANGE (but not MT_FLAGS_USE_RCU) would trigger rcu warnings.
Guenter
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.```
mm/damon/tests/vaddr-kunit.h +1
-1@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init (&mm.mm_mt);if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );```
Quoted 3 lines
Quoted 3 lines
And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
Again, I'd use the flags "MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU" because that gets you the gap tracking that may be necessary for tests in the future - it's closer to the MM_MT_FLAGS, so maybe some mm function you use depends on that.
Quoted 6 lines
Quoted 1 line
Same here. That is why I gave up after trying
MT_FLAGS_ALLOC_RANGEand "MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree withMT_FLAGS_ALLOC_RANGE(but notMT_FLAGS_USE_RCU) would trigger rcu warnings.
Thanks, I'll add that to my list of things to do.
Regards,
Liam
SeongJae Park
maintainerQuoted 48 lines · On Tue, 3 Sep 2024 22:43:40 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 47 lines · Guenter Roeck
Quoted 46 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.```
mm/damon/tests/vaddr-kunit.h +1
-1@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init (&mm.mm_mt);if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );```
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
Again, I'd use the flags "
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU" because that gets you the gap tracking that may be necessary for tests in the future - it's closer to theMM_MT_FLAGS, so maybe some mm function you use depends on that.
Thank you for the nice suggestion with the rationales. Just posted the v2 following it: https://lore.kernel.org/[email protected]
Quoted 9 lines
Quoted 7 lines
Quoted 2 lines
Same here. That is why I gave up after trying
MT_FLAGS_ALLOC_RANGEand "MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU". After all, I really don't know what I am doing and was just playing around ... and there isn't really a good explanation why initializing the maple tree withMT_FLAGS_ALLOC_RANGE(but notMT_FLAGS_USE_RCU) would trigger rcu warnings.Thanks, I'll add that to my list of things to do.
Thank you. I agree that's somewhat we can visit separately.
FYI, I was also unable to reproduce rcu warnings with my v2 patch on my setup. I will also try to use Guenter's more detailed repro
(https://lore.kernel.org/[email protected]).
Thanks,
SJ
Quoted 3 lines
Regards,
Liam
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?
That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.
Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use MT_FLAGS_ALLOC_RANGE to use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).
I would recommend those two and just use the spinlock.
Quoted 84 lines
```
mm/damon/tests/vaddr-kunit.h +1
-1@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init (&mm.mm_mt);if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );```
And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.
The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the
MTREE_INIT_EXT()#define, which takes an external lock as an argument.(from
Documentation/core-api/maple_tree.rst)I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.
Thanks,
SJQuoted 49 lines
Thanks,
SJQuoted 44 lines
Quoted 44 lines
Reported-by Guenter Roeck <[email protected]>Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")Signed-off-by SeongJae Park <[email protected]>--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)mm/damon/tests/vaddr-kunit.h +3
-7@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/* Signature · 1 line
2.39.2
Quoted 37 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.
Quoted 9 lines
Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).I would recommend those two and just use the spinlock.
I tried that (MT_FLAGS_ALLOC_RANGE | MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.
Guenter
Quoted 84 lines
Quoted 84 lines
```
mm/damon/tests/vaddr-kunit.h +1
-1@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init (&mm.mm_mt);if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );```
And just confirmed it also convinces the reproducer. But because I'm obviously not familiar with maple tree, would like to hear some comments from Liam or others first.
FYI, I ended up writing v1 to simply remove lock usage based on my humble understanding of the documetnation.
The maple tree uses a spinlock by default, but external locks can be used for tree updates as well. To use an external lock, the tree must be initialized with the ``MT_FLAGS_LOCK_EXTERN flag``, this is usually done with the
MTREE_INIT_EXT()#define, which takes an external lock as an argument.(from
Documentation/core-api/maple_tree.rst)I was thinking the fact that the test code is executed in single thread is same to use of external lock. I will be happy to learn if I missed something.
Thanks,
SJQuoted 49 lines
Thanks,
SJQuoted 44 lines
Quoted 44 lines
Reported-by Guenter Roeck <[email protected]>Fixes d0cf3dd47f0d ("damon: convert __damon_va_three_regions to use the VMA iterator")Signed-off-by SeongJae Park <[email protected]>--- mm/damon/tests/vaddr-kunit.h | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)mm/damon/tests/vaddr-kunit.h +3
-7@@ -17,23 +17,19 @@static int __link_vmas (struct maple_tree *mt,struct vm_area_struct *vmas,ssize_t nr_vmas) { -int i, ret = -ENOMEM ; +int i;MA_STATE (mas, mt,0 ,0 );if (!nr_vmas)return 0 ; -mas_lock (&mas);for (i =0 ; i < nr_vmas; i++) {mas_set_range (&mas, vmas[i].vm_start, vmas[i].vm_end -1 );if (mas_store_gfp (&mas, &vmas[i],GFP_KERNEL )) - goto failed; +return -ENOMEM ; } - ret =0 ; -failed: -mas_unlock (&mas); -return ret; +return 0 ; }/* Signature · 1 line
2.39.2
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).
Quoted 5 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.
Just to be clear, you changed the init flags and kept the mas_lock() and mas_unlock() ?
Thanks,
Liam
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).
Quoted 6 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.
I am running ./tools/testing/kunit/kunit.py run '*damon*' --arch x86_64 --raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=y
and I don't have any issue with locking in the existing code. How do I recreate this issue?
Thanks,
Liam
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).Quoted 6 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.
Quoted 8 lines
I am running
./tools/testing/kunit/kunit.pyrun '*damon*' --archx86_64--raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=yand I don't have any issue with locking in the existing code. How do I recreate this issue?
I tested again, and I still see
[ 6.233483] ok 4 damon [ 6.234190] KTAP version 1 [ 6.234263] # Subtest: damon-operations [ 6.234335] # module: vaddr [ 6.234384] 1..6 [ 6.235726] [ 6.235931] ============================= [ 6.236018] WARNING: suspicious RCU usage [ 6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G N [ 6.236398] ----------------------------- [ 6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage! [ 6.236579] [ 6.236579] other info that might help us debug this: [ 6.236579] [ 6.236738] [ 6.236738] rcu_scheduler_active = 2, debug_locks = 1 [ 6.237039] no locks held by kunit_try_catch/208. [ 6.237166] [ 6.237166] stack backtrace: [ 6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G N 6.11.0-rc6-00029-gda66250b210f-dirty #1 [ 6.237629] Tainted: [N]=TEST [ 6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 6.238065] Call Trace: [ 6.238233] <TASK> [ 6.238547] dump_stack_lvl+0x9e/0xe0 [ 6.239473] lockdep_rcu_suspicious+0x145/0x1b0 [ 6.239621] mas_walk+0x19f/0x1d0 [ 6.239765] mas_find+0xb5/0x150 [ 6.239873] __damon_va_three_regions+0x7e/0x130 [ 6.240039] damon_test_three_regions_in_vmas+0x1ea/0x480 [ 6.240551] ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 [ 6.240712] kunit_try_run_case+0x93/0x190 [ 6.240850] ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 [ 6.240990] kunit_generic_run_threadfn_adapter+0x1c/0x40 [ 6.241124] kthread+0xdd/0x110 [ 6.241256] ? __pfx_kthread+0x10/0x10 [ 6.241368] ret_from_fork+0x2f/0x50 [ 6.241468] ? __pfx_kthread+0x10/0x10 [ 6.241573] ret_from_fork_asm+0x1a/0x30 [ 6.241765] </TASK> [ 6.242180] [ 6.242270] ============================= [ 6.242375] WARNING: suspicious RCU usage [ 6.242478] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G N [ 6.242634] ----------------------------- [ 6.242734] lib/maple_tree.c:788 suspicious rcu_dereference_check() usage! [ 6.242955] [ 6.242955] other info that might help us debug this: [ 6.242955] [ 6.243098] [ 6.243098] rcu_scheduler_active = 2, debug_locks = 1 [ 6.243215] no locks held by kunit_try_catch/208. [ 6.243331] [ 6.243331] stack backtrace: [ 6.243420] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G N 6.11.0-rc6-00029-gda66250b210f-dirty #1 [ 6.243599] Tainted: [N]=TEST [ 6.243665] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 6.243844] Call Trace: [ 6.243907] <TASK> [ 6.243961] dump_stack_lvl+0x9e/0xe0 [ 6.244032] lockdep_rcu_suspicious+0x145/0x1b0 [ 6.244121] mtree_range_walk+0x2b9/0x350 [ 6.244211] mas_walk+0x107/0x1d0 [ 6.244278] mas_find+0xb5/0x150 [ 6.244341] __damon_va_three_regions+0x7e/0x130 [ 6.244445] damon_test_three_regions_in_vmas+0x1ea/0x480 [ 6.244770] ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 [ 6.244865] kunit_try_run_case+0x93/0x190 [ 6.244952] ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 [ 6.245044] kunit_generic_run_threadfn_adapter+0x1c/0x40 [ 6.245130] kthread+0xdd/0x110 [ 6.245191] ? __pfx_kthread+0x10/0x10 [ 6.245262] ret_from_fork+0x2f/0x50 [ 6.245326] ? __pfx_kthread+0x10/0x10 [ 6.245394] ret_from_fork_asm+0x1a/0x30 [ 6.245497] </TASK> [ 6.246605] # damon_test_three_regions_in_vmas: pass:1 fail:0 skip:0 total:1 [ 6.246668] ok 1 damon_test_three_regions_in_vmas
This is with
mm/damon/vaddr-test.h +1 -1
@@ -77,7 +77,7 @@ static void damon_test_three_regions_in_vmas(struct kunit *test) (struct vm_area_struct ) {.vm_start =307 , .vm_end =330 }, }; -mt_init_flags (&mm.mm_mt,MM_MT_FLAGS ); +mt_init_flags (&mm.mm_mt,MT_FLAGS_ALLOC_RANGE |MT_FLAGS_USE_RCU );if (__link_vmas (&mm.mm_mt, vmas,ARRAY_SIZE (vmas)))kunit_skip (test,"Failed to create VMA tree" );
I'll put it all together and make it available, but that will have to wait until tomorrow.
Thanks,
Guenter
Quoted 87 lines · Guenter Roeck
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).Quoted 6 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.Quoted 8 lines
I am running
./tools/testing/kunit/kunit.pyrun '*damon*' --archx86_64--raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=yand I don't have any issue with locking in the existing code. How do I recreate this issue?
I tested again, and I still see
[ 6.233483] ok 4 damon [ 6.234190] KTAP version 1 [ 6.234263] # Subtest: damon-operations [ 6.234335] # module: vaddr [ 6.234384] 1..6 [ 6.235726] [ 6.235931] ============================= [ 6.236018] WARNING: suspicious RCU usage [ 6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G N [ 6.236398] ----------------------------- [ 6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage! [ 6.236579] [ 6.236579] other info that might help us debug this: [ 6.236579] [ 6.236738] [ 6.236738] rcu_scheduler_active = 2, debug_locks = 1 [ 6.237039] no locks held by kunit_try_catch/208. [ 6.237166] [ 6.237166] stack backtrace: [ 6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G N 6.11.0-rc6-00029-gda66250b210f-dirty #1 [ 6.237629] Tainted: [N]=TEST
Quoted 8 lines
[ 6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 6.238065] Call Trace: [ 6.238233] <TASK> [ 6.238547] dump_stack_lvl+0x9e/0xe0 [ 6.239473] lockdep_rcu_suspicious+0x145/0x1b0 [ 6.239621] mas_walk+0x19f/0x1d0 [ 6.239765] mas_find+0xb5/0x150 [ 6.239873] __damon_va_three_regions+0x7e/0x130
This function isn't taking the rcu read lock while iterating the tree.
Try this:
mm/damon/vaddr.c +2 -0
@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,* If this is too slow, it can be optimised to examine the maple * tree gaps. */ +rcu_read_lock ();for_each_vma (vmi, vma) {unsigned long gap;
@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm, next: prev = vma; } +rcu_read_unlock ();if (!sz_range (&second_gap) || !sz_range (&first_gap))return -EINVAL ;
Quoted 115 lines · Liam R. Howlett
Quoted 87 lines · Guenter Roeck
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).Quoted 6 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.Quoted 8 lines
I am running
./tools/testing/kunit/kunit.pyrun '*damon*' --archx86_64--raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=yand I don't have any issue with locking in the existing code. How do I recreate this issue?
I tested again, and I still see
[ 6.233483] ok 4 damon [ 6.234190] KTAP version 1 [ 6.234263] # Subtest: damon-operations [ 6.234335] # module: vaddr [ 6.234384] 1..6 [ 6.235726] [ 6.235931] ============================= [ 6.236018] WARNING: suspicious RCU usage [ 6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G N [ 6.236398] ----------------------------- [ 6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage! [ 6.236579] [ 6.236579] other info that might help us debug this: [ 6.236579] [ 6.236738] [ 6.236738] rcu_scheduler_active = 2, debug_locks = 1 [ 6.237039] no locks held by kunit_try_catch/208. [ 6.237166] [ 6.237166] stack backtrace: [ 6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G N 6.11.0-rc6-00029-gda66250b210f-dirty #1 [ 6.237629] Tainted: [N]=TESTQuoted 8 lines
[ 6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 6.238065] Call Trace: [ 6.238233] <TASK> [ 6.238547] dump_stack_lvl+0x9e/0xe0 [ 6.239473] lockdep_rcu_suspicious+0x145/0x1b0 [ 6.239621] mas_walk+0x19f/0x1d0 [ 6.239765] mas_find+0xb5/0x150 [ 6.239873] __damon_va_three_regions+0x7e/0x130This function isn't taking the rcu read lock while iterating the tree.
Try this:
mm/damon/vaddr.c +1
-0@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,* If this is too slow, it can be optimised to examine the maple * tree gaps. */ +rcu_read_lock ();for_each_vma (vmi, vma) {unsigned long gap;
mm/damon/vaddr.c static int __damon_va_three_regions(struct mm_struct *mm, · line 153
Quoted 7 lines
mm/damon/vaddr.c +1
-0@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm, next: prev = vma; } +rcu_read_unlock ();if (!sz_range (&second_gap) || !sz_range (&first_gap))return -EINVAL ;
Yes, that fixes the problem for me.
Thanks,
Guenter
SeongJae Park
maintainerQuoted 11 lines · On Wed, 4 Sep 2024 12:56:33 -0700 Guenter Roeck <[email protected]> wrote:
Quoted 10 lines · Liam R. Howlett
Quoted 9 lines · Guenter Roeck
Quoted 8 lines · Liam R. Howlett
Quoted 7 lines · Guenter Roeck
Quoted 6 lines · Liam R. Howlett
Quoted 5 lines · SeongJae Park
Quoted 3 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 1 line · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
* SeongJae Park <[email protected]> [240903 20:45]:
[...]
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 8 lines
I am running
./tools/testing/kunit/kunit.pyrun '*damon*' --archx86_64--raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=yand I don't have any issue with locking in the existing code. How do I recreate this issue?
I tested again, and I still see
[ 6.233483] ok 4 damon [ 6.234190] KTAP version 1 [ 6.234263] # Subtest: damon-operations [ 6.234335] # module: vaddr [ 6.234384] 1..6 [ 6.235726] [ 6.235931] ============================= [ 6.236018] WARNING: suspicious RCU usage [ 6.236280] 6.11.0-rc6-00029-gda66250b210f-dirty #1 Tainted: G N [ 6.236398] ----------------------------- [ 6.236474] lib/maple_tree.c:832 suspicious rcu_dereference_check() usage! [ 6.236579] [ 6.236579] other info that might help us debug this: [ 6.236579] [ 6.236738] [ 6.236738] rcu_scheduler_active = 2, debug_locks = 1 [ 6.237039] no locks held by kunit_try_catch/208. [ 6.237166] [ 6.237166] stack backtrace: [ 6.237385] CPU: 0 UID: 0 PID: 208 Comm: kunit_try_catch Tainted: G N 6.11.0-rc6-00029-gda66250b210f-dirty #1 [ 6.237629] Tainted: [N]=TEST
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
[ 6.237714] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [ 6.238065] Call Trace: [ 6.238233] <TASK> [ 6.238547] dump_stack_lvl+0x9e/0xe0 [ 6.239473] lockdep_rcu_suspicious+0x145/0x1b0 [ 6.239621] mas_walk+0x19f/0x1d0 [ 6.239765] mas_find+0xb5/0x150 [ 6.239873] __damon_va_three_regions+0x7e/0x130
I was able to reproduce this by further enabling PROVE_LOCKING.
Quoted 23 lines
Quoted 23 lines
This function isn't taking the rcu read lock while iterating the tree.
Try this:
mm/damon/vaddr.c +2
-0@@ -126,6 +126,7 @@ static int __damon_va_three_regions(struct mm_struct *mm,* If this is too slow, it can be optimised to examine the maple * tree gaps. */ +rcu_read_lock ();for_each_vma (vmi, vma) {unsigned long gap;@@ -146,6 +147,7 @@ static int __damon_va_three_regions(struct mm_struct *mm, next: prev = vma; } +rcu_read_unlock ();
Quoted 6 lines
mm/damon/vaddr.c static int __damon_va_three_regions(struct mm_struct *mm, · line 153
Quoted 3 lines
mm/damon/vaddr.c +0
-0if (!sz_range (&second_gap) || !sz_range (&first_gap))return -EINVAL ;Yes, that fixes the problem for me.
Thank you for the fix, Liam. Thank you for the test, Guenter. I also confirmed this fix works on my setup.
I posted the fix as a formal patch:
https://lore.kernel.org/[email protected]
Thanks,
SJ
Quoted 3 lines
Thanks,
Guenter
Quoted 53 lines · Liam R. Howlett
Quoted 45 lines · Guenter Roeck
Quoted 44 lines · Liam R. Howlett
Quoted 21 lines · SeongJae Park
Quoted 19 lines · On Tue, 3 Sep 2024 17:58:15 -0700 SeongJae Park <[email protected]> wrote:
Quoted 11 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 9 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it reports spinlock bad magic bug. Fix the issue by not using themt_lockas promised.Quoted 1 line
You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 8 lines
Quoted 8 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?That would be the lockdep complaining, so that's good.
Quoted 3 lines
I was actually thinking replacing the
mt_init_flags()withmt_init(), which same tomt_init_flags()with zero flag, like below.Yes. This will use the spinlock which should fix your issue, but it will use a different style of maple tree.
Perhaps use
MT_FLAGS_ALLOC_RANGEto use the same type of maple tree, if you ever add threading you will want the rcu flag as well (MT_FLAGS_USE_RCU).Quoted 6 lines
Quoted 2 lines
I would recommend those two and just use the spinlock.
I tried that (
MT_FLAGS_ALLOC_RANGE|MT_FLAGS_USE_RCU). it also triggers the suspicious RCU usage message.
Quoted 8 lines
I am running
./tools/testing/kunit/kunit.pyrun '*damon*' --archx86_64--raw with:CONFIG_LOCKDEP=yCONFIG_DEBUG_SPINLOCK=yand I don't have any issue with locking in the existing code. How do I recreate this issue?
You should find everything needed to reproduce the problem at
http://server.roeck-us.net/qemu/damon/
Please let me know if you need anything else.
Thanks,
Guenter
Guenter Roeck
Tested-byQuoted 10 lines · SeongJae Park
Quoted 8 lines · On Tue, 3 Sep 2024 20:48:53 -0400 "Liam R. Howlett" <[email protected]>…
Quoted 7 lines · SeongJae Park
damon_test_three_regions_in_vmas()initializes a maple tree withMM_MT_FLAGS. The flags containsMT_FLAGS_LOCK_EXTERN, which meansmt_lockof the maple tree will not be used. And therefore the maple tree initialization code skips initialization of themt_lock. However,__link_vmas(), which adds vmas for test to the maple tree, uses themt_lock. In other words, the uninitialized spinlock is used. The problem becomes celar when spinlock debugging is turned on, since it
Just in case you need to resend: s/celar/clear/
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
reports spinlock bad magic bug. Fix the issue by not using the
mt_lockas promised.You can't do this, lockdep will tell you this is wrong.
Hmm, but lockdep was silence on my setup?
Quoted 1 line
We need a lock and to use the lock for writes.
Quoted 9 lines
This code is executed by a single-thread test code. Do we still need the lock?
Quoted 2 lines
I'd suggest using different flags so the spinlock is used.
The reporter mentioned simply dropping
MT_FLAGS_LOCK_EXTERNfrom the flags causes suspicious RCU usage message. May I ask if you have a suggestion of better flags?
Correct. I don't see those messages with your patch. From my perspective, this is
Thanks,
Guenter