Skip to contentSkip to conversation

kasan: Don't call find_vm_area() in RT kernel

PATCH Superseded Waiman Long 5 participants 8 messages Feb 11, 2025 – Feb 12, 2025

Subsystems (3)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kasan: Don't call find_vm_area() in RT kernel

The following bug report appeared with a test run in a RT debug kernel.

[ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
[ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch
[ 3359.353853] preempt_count: 1, expected: 0
  :
[ 3359.353933] Call trace:
  :
[ 3359.353955]  rt_spin_lock+0x70/0x140
[ 3359.353959]  find_vmap_area+0x84/0x168
[ 3359.353963]  find_vm_area+0x1c/0x50
[ 3359.353966]  print_address_description.constprop.0+0x2a0/0x320
[ 3359.353972]  print_report+0x108/0x1f8
[ 3359.353976]  kasan_report+0x90/0xc8
[ 3359.353980]  __asan_load1+0x60/0x70

The print_address_description() is run with a raw_spinlock_t acquired and interrupt disabled. The find_vm_area() function needs to acquire a spinlock_t which becomes a sleeping lock in the RT kernel. IOW, we can't call find_vm_area() in a RT kernel. Fix this bug report by skipping the find_vm_area() call in this case and just print out the address as is.

For !RT kernel, follow the example set in commit 0cce06ba859a ("debugobjects,locking: Annotate debug_object_fill_pool() wait type violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t inside raw_spinlock_t warning.

---
 mm/kasan/report.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
mm/kasan/report.c +18 -2
index 3fe77a360f1c..e1ee687966aa 100644
@@ -398,9 +398,20 @@ static void print_address_description(void *addr, u8 tag,
 		pr_err("\n");
 	}
 
-	if (is_vmalloc_addr(addr)) {
-		struct vm_struct *va = find_vm_area(addr);
+	if (!is_vmalloc_addr(addr))
+		goto print_page;
 
+	/*
+	 * RT kernel cannot call find_vm_area() in atomic context.
+	 * For !RT kernel, prevent spinlock_t inside raw_spinlock_t warning
+	 * by raising wait-type to WAIT_SLEEP.
+	 */
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP);
+		struct vm_struct *va;
+
+		lock_map_acquire_try(&vmalloc_map);
+		va = find_vm_area(addr);
 		if (va) {
 			pr_err("The buggy address belongs to the virtual mapping at\n"
 			       " [%px, %px) created by:\n"
@@ -410,8 +421,13 @@ static void print_address_description(void *addr, u8 tag,
 
 			page = vmalloc_to_page(addr);
 		}
+		lock_map_release(&vmalloc_map);
+	} else {
+		pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n",
+			addr);
 	}
 
+print_page:
 	if (page) {
 		pr_err("The buggy address belongs to the physical page:\n");
 		dump_page(page, "kasan: bad access detected");
Signature · 1 line
2.48.1
maintainer
#1 lore raw
Quoted 23 lines · On Tue, 11 Feb 2025 11:07:50 -0500 Waiman Long <[email protected]> wrote:

The following bug report appeared with a test run in a RT debug kernel.

[ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
[ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch
[ 3359.353853] preempt_count: 1, expected: 0
  :
[ 3359.353933] Call trace:
  :
[ 3359.353955]  rt_spin_lock+0x70/0x140
[ 3359.353959]  find_vmap_area+0x84/0x168
[ 3359.353963]  find_vm_area+0x1c/0x50
[ 3359.353966]  print_address_description.constprop.0+0x2a0/0x320
[ 3359.353972]  print_report+0x108/0x1f8
[ 3359.353976]  kasan_report+0x90/0xc8
[ 3359.353980]  __asan_load1+0x60/0x70

The print_address_description() is run with a raw_spinlock_t acquired and interrupt disabled. The find_vm_area() function needs to acquire a spinlock_t which becomes a sleeping lock in the RT kernel. IOW, we can't call find_vm_area() in a RT kernel. Fix this bug report by skipping the find_vm_area() call in this case and just print out the address as is.

Quoted 5 lines

For !RT kernel, follow the example set in commit 0cce06ba859a ("debugobjects,locking: Annotate debug_object_fill_pool() wait type violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t inside raw_spinlock_t warning.

Thanks. I added it and shall await review from the KASAN developers.

I'm thinking we add

Fixes c056a364e954 ("kasan: print virtual mapping info in reports")

but c056a364e954 is 3 years old and I don't think we care about -rt in such old kernels. Thoughts?

maintainer
#2 lore raw
Quoted 7 lines · On Tue, 11 Feb 2025 14:57:30 -0800 Andrew Morton <[email protected]>…

I'm thinking we add

Fixes c056a364e954 ("kasan: print virtual mapping info in reports")

but c056a364e954 is 3 years old and I don't think we care about -rt in such old kernels. Thoughts?

We still support -rt in older kernels back to 5.4, and we merge in stable releases. If this fixes an -rt issue, please do mark it for stable.

Thanks,

-- Steve

#3 lore raw
In reply to Andrew Morton
Quoted 32 lines · Andrew Morton
Quoted 23 lines · On Tue, 11 Feb 2025 11:07:50 -0500 Waiman Long <[email protected]> wrote:

The following bug report appeared with a test run in a RT debug kernel.

[ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48
[ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch
[ 3359.353853] preempt_count: 1, expected: 0
   :
[ 3359.353933] Call trace:
   :
[ 3359.353955]  rt_spin_lock+0x70/0x140
[ 3359.353959]  find_vmap_area+0x84/0x168
[ 3359.353963]  find_vm_area+0x1c/0x50
[ 3359.353966]  print_address_description.constprop.0+0x2a0/0x320
[ 3359.353972]  print_report+0x108/0x1f8
[ 3359.353976]  kasan_report+0x90/0xc8
[ 3359.353980]  __asan_load1+0x60/0x70

The print_address_description() is run with a raw_spinlock_t acquired and interrupt disabled. The find_vm_area() function needs to acquire a spinlock_t which becomes a sleeping lock in the RT kernel. IOW, we can't call find_vm_area() in a RT kernel. Fix this bug report by skipping the find_vm_area() call in this case and just print out the address as is.

Quoted 5 lines

For !RT kernel, follow the example set in commit 0cce06ba859a ("debugobjects,locking: Annotate debug_object_fill_pool() wait type violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t inside raw_spinlock_t warning.

Thanks. I added it and shall await review from the KASAN developers.

Quoted 7 lines

I'm thinking we add

Fixes c056a364e954 ("kasan: print virtual mapping info in reports")

but c056a364e954 is 3 years old and I don't think we care about -rt in such old kernels. Thoughts?

The KASAN report_lock was changed to a raw_spinlock_t in v6.13 kernel with commit e30a0361b851 ("kasan: make report_lock a raw spinlock") to fix a similar RT problem. The report_lock is acquired before calling print_address_description(). Before commit e30a0361b851, this find_vm_area() is a secondary issue. We may consider commit e30a0361b851 isn't complete and this is a fix for that.

The DEFINE_WAIT_OVERRIDE_MAP() macro was introduced in v6.4. So this patch cannot be backported to a version earlier than that unless commit 0cce06ba859a is there.

Cheers,
Longman

maintainer
#4 lore raw
Quoted 6 lines · On Tue, 11 Feb 2025 19:16:34 -0500 Waiman Long <[email protected]> wrote:
Quoted 5 lines
Fixes c056a364e954 ("kasan: print virtual mapping info in reports")

but c056a364e954 is 3 years old and I don't think we care about -rt in such old kernels. Thoughts?

Quoted 6 lines

The KASAN report_lock was changed to a raw_spinlock_t in v6.13 kernel with commit e30a0361b851 ("kasan: make report_lock a raw spinlock") to fix a similar RT problem. The report_lock is acquired before calling print_address_description(). Before commit e30a0361b851, this find_vm_area() is a secondary issue. We may consider commit e30a0361b851 isn't complete and this is a fix for that.

Great, thanks, updated...

maintainer
#5 lore raw
In reply to Waiman Long
Quoted 14 lines · Waiman Long
mm/kasan/report.c +3 -2
index 3fe77a360f1c..e1ee687966aa 100644
@@ -398,9 +398,20 @@ static void print_address_description(void *addr, u8 tag,
                pr_err("\n");
        }
 
-       if (is_vmalloc_addr(addr)) {
-               struct vm_struct *va = find_vm_area(addr);
+       if (!is_vmalloc_addr(addr))
+               goto print_page;
 
+       /*

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 414

Quoted 10 lines
mm/kasan/report.c +10 -0
+        * RT kernel cannot call find_vm_area() in atomic context.
+        * For !RT kernel, prevent spinlock_t inside raw_spinlock_t warning
+        * by raising wait-type to WAIT_SLEEP.
+        */
+       if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
+               static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP);
+               struct vm_struct *va;
+
+               lock_map_acquire_try(&vmalloc_map);
+               va = find_vm_area(addr);

Can we hide all this logic behind some function like kasan_find_vm_area() which would return NULL for -rt?

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 422

Quoted 6 lines
mm/kasan/report.c +0 -0
                if (va) {
                        pr_err("The buggy address belongs to the virtual mapping at\n"
                               " [%px, %px) created by:\n"
@@ -410,8 +421,13 @@ static void print_address_description(void *addr, u8 tag,
 
                        page = vmalloc_to_page(addr);

Or does vmalloc_to_page() secretly take some lock somewhere so we need to guard it with this 'vmalloc_map' too?
So my suggestion above wouldn't be enough, if that's the case.

Quoted 7 lines
mm/kasan/report.c +4 -0
                }
+               lock_map_release(&vmalloc_map);
+       } else {
+               pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n",
+                       addr);
        }
 
#6 lore raw
Quoted 14 lines · Andrey Ryabinin
Quoted 13 lines · Waiman Long
mm/kasan/report.c +2 -2
index 3fe77a360f1c..e1ee687966aa 100644
@@ -398,9 +398,20 @@ static void print_address_description(void *addr, u8 tag,
                 pr_err("\n");
         }
 
-       if (is_vmalloc_addr(addr)) {
-               struct vm_struct *va = find_vm_area(addr);
+       if (!is_vmalloc_addr(addr))
+               goto print_page;
 
Quoted 13 lines

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 414

Quoted 11 lines
mm/kasan/report.c +11 -0
+       /*
+        * RT kernel cannot call find_vm_area() in atomic context.
+        * For !RT kernel, prevent spinlock_t inside raw_spinlock_t warning
+        * by raising wait-type to WAIT_SLEEP.
+        */
+       if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
+               static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP);
+               struct vm_struct *va;
+
+               lock_map_acquire_try(&vmalloc_map);
+               va = find_vm_area(addr);

Can we hide all this logic behind some function like kasan_find_vm_area() which would return NULL for -rt?

Sure. We can certainly do that.

Quoted 10 lines

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 422

Quoted 7 lines
mm/kasan/report.c +0 -0
                 if (va) {
                         pr_err("The buggy address belongs to the virtual mapping at\n"
                                " [%px, %px) created by:\n"
@@ -410,8 +421,13 @@ static void print_address_description(void *addr, u8 tag,
 
                         page = vmalloc_to_page(addr);

Or does vmalloc_to_page() secretly take some lock somewhere so we need to guard it with this 'vmalloc_map' too?
So my suggestion above wouldn't be enough, if that's the case.

AFAICS, vmalloc_to_page() doesn't seem to take any lock.  Even if it takes another spinlock, it will still be under the vmalloc_map protection until lock_map_release() is called.

Cheers,
Longman

maintainer
#7 lore raw
Quoted 36 lines · Waiman Long
Quoted 14 lines · Andrey Ryabinin
Quoted 13 lines · Waiman Long
mm/kasan/report.c +2 -2
index 3fe77a360f1c..e1ee687966aa 100644
@@ -398,9 +398,20 @@ static void print_address_description(void *addr, u8 tag,
                  pr_err("\n");
          }
 
-       if (is_vmalloc_addr(addr)) {
-               struct vm_struct *va = find_vm_area(addr);
+       if (!is_vmalloc_addr(addr))
+               goto print_page;
 
Quoted 13 lines

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 414

Quoted 11 lines
mm/kasan/report.c +11 -0
+       /*
+        * RT kernel cannot call find_vm_area() in atomic context.
+        * For !RT kernel, prevent spinlock_t inside raw_spinlock_t warning
+        * by raising wait-type to WAIT_SLEEP.
+        */
+       if (!IS_ENABLED(CONFIG_PREEMPT_RT)) {
+               static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP);
+               struct vm_struct *va;
+
+               lock_map_acquire_try(&vmalloc_map);
+               va = find_vm_area(addr);

Can we hide all this logic behind some function like kasan_find_vm_area() which would return NULL for -rt?

Sure. We can certainly do that.

Quoted 6 lines
Quoted 6 lines
mm/kasan/report.c +0 -0
                  if (va) {

                         pr_err("The buggy address belongs to the virtual mapping at\n" " [%px, %px) created by:\n"

mm/kasan/report.c +0 -0
@@ -410,8 +421,13 @@ static void print_address_description(void *addr, u8 tag,
 
Quoted 7 lines
Quoted 4 lines

mm/kasan/report.c static void print_address_description(void *addr, u8 tag, · line 422

Quoted 1 line
mm/kasan/report.c +0 -0
                          page = vmalloc_to_page(addr);

Or does vmalloc_to_page() secretly take  some lock somewhere so we need to guard it with this 'vmalloc_map' too?
So my suggestion above wouldn't be enough, if that's the case.

AFAICS, vmalloc_to_page() doesn't seem to take any lock.  Even if it takes another spinlock, it will still be under the vmalloc_map protection until lock_map_release() is called.

I meant to do something like bellow, which would leave vmalloc_to_page() out of vmalloc_map scope. That's why I raised this question.

---
 mm/kasan/report.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
mm/kasan/report.c +15 -2
index 3fe77a360f1c..f3683215f4ca 100644
@@ -370,6 +370,20 @@ static inline bool init_task_stack_addr(const void *addr)
 			sizeof(init_thread_union.stack));
 }
 
+static inline struct vm_struct *kasan_find_vm_area(void *addr)
+{
+	static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP);
+	struct vm_struct *va;
+
+	if (IS_ENABLED(CONFIG_PREEMPT_RT))
+		return NULL;
+
+	lock_map_acquire_try(&vmalloc_map);
+	va = find_vm_area(addr);
+	lock_map_release(&vmalloc_map);
+	return va;
+}
+
 static void print_address_description(void *addr, u8 tag,
 				      struct kasan_report_info *info)
 {
@@ -399,8 +413,7 @@ static void print_address_description(void *addr, u8 tag,
 	}
 
 	if (is_vmalloc_addr(addr)) {
-		struct vm_struct *va = find_vm_area(addr);
-
+		struct vm_area *va = kasan_find_vm_area(addr);
 		if (va) {
 			pr_err("The buggy address belongs to the virtual mapping at\n"
 			       " [%px, %px) created by:\n"
Signature · 1 line
2.45.3