Skip to contentSkip to conversation

c6307674ed: BUG:sleeping_function_called_from_invalid_context_at_mm/vmalloc.c

linus:master New kernel test robot 4 participants 14 messages Apr 21 – Apr 28

Subsystems (5)
lore
Share & tools
lore

kernel test robot

author build bot
#0 lore raw

Hello,

kernel test robot noticed "BUG:sleeping_function_called_from_invalid_context_at_mm/vmalloc.c" on:

commit: c6307674ed82c0c57d6e1e3408e84ac449ab8e94 ("mm: kvmalloc: add non-blocking support for vmalloc")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

[test failed on linus/master 40286d6379aacfcc053253ef78dc78b09addffda]
[test failed on linux-next/master 936c21068d7ade00325e40d82bfd2f3f29d9f659]

in testcase: kunit
version:
with following parameters:

	group: group-01

config: x86_64-rhel-9.4-kunit
compiler: gcc-14
test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (Haswell) with 16G memory

(please refer to attached dmesg/kmsg for entire log/backtrace)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]

[  149.217862][ T4054] BUG: sleeping function called from invalid context at mm/vmalloc.c:3435
[  149.226291][ T4054] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid: 4054, name: rhashtable_thra
[  149.235634][ T4054] preempt_count: 0, expected: 0
[  149.240353][ T4054] RCU nest depth: 1, expected: 0
[  149.245166][ T4054] CPU: 1 UID: 0 PID: 4054 Comm: rhashtable_thra Tainted: G S  B            N  6.18.0-rc5-00047-gc6307674ed82 #1 PREEMPT(voluntary)
[  149.245176][ T4054] Tainted: [S]=CPU_OUT_OF_SPEC, [B]=BAD_PAGE, [N]=TEST
[  149.245178][ T4054] Hardware name: Dell Inc. OptiPlex 9020/0DNKMN, BIOS A05 12/05/2013
[  149.245181][ T4054] Call Trace:
[  149.245185][ T4054]  <TASK>
[  149.245189][ T4054]  dump_stack_lvl (lib/dump_stack.c:122)
[  149.245200][ T4054]  __might_resched.cold (kernel/sched/core.c:8928)
[  149.245209][ T4054]  vfree (include/linux/kernel.h:61 mm/vmalloc.c:3435)
[  149.245215][ T4054]  rhashtable_try_insert (lib/rhashtable.c:476 lib/rhashtable.c:635)
[  149.245223][ T4054]  ? sysvec_apic_timer_interrupt (arch/x86/include/asm/hardirq.h:78 (discriminator 2) arch/x86/kernel/apic/apic.c:1052 (discriminator 2))
[  149.245229][ T4054]  ? sysvec_apic_timer_interrupt (arch/x86/include/asm/hardirq.h:78 (discriminator 2) arch/x86/kernel/apic/apic.c:1052 (discriminator 2))
[  149.245235][ T4054]  ? __pfx_rhashtable_try_insert (lib/rhashtable.c:592)
[  149.245242][ T4054]  rhashtable_insert_slow (lib/rhashtable.c:648)
[  149.245247][ T4054] insert_retry+0x257/0x630 test_rhashtable
[  149.245260][ T4054] threadfunc (lib/test_rhashtable.c:633) test_rhashtable
[  149.245269][ T4054]  ? try_to_wake_up (kernel/sched/core.c:4297)
[  149.245275][ T4054]  ? _raw_spin_lock_irqsave (arch/x86/include/asm/atomic.h:107 (discriminator 4) include/linux/atomic/atomic-arch-fallback.h:2170 (discriminator 4) include/linux/atomic/atomic-instrumented.h:1302 (discriminator 4) include/asm-generic/qspinlock.h:111 (discriminator 4) include/linux/spinlock.h:187 (discriminator 4) include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
[  149.245281][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245289][ T4054]  ? __pfx_autoremove_wake_function (kernel/sched/wait.c:402)
[  149.245295][ T4054]  ? __kthread_parkme (arch/x86/include/asm/bitops.h:202 (discriminator 1) arch/x86/include/asm/bitops.h:232 (discriminator 1) include/asm-generic/bitops/instrumented-non-atomic.h:142 (discriminator 1) kernel/kthread.c:290 (discriminator 1))
[  149.245301][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245309][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245317][ T4054]  kthread (kernel/kthread.c:463)
[  149.245322][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245326][ T4054]  ? __pfx__raw_spin_lock_irq (kernel/locking/spinlock.c:169)
[  149.245331][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245335][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245339][ T4054]  ret_from_fork (arch/x86/kernel/process.c:164)
[  149.245344][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245348][ T4054]  ret_from_fork_asm (arch/x86/entry/entry_64.S:255)
[  149.245356][ T4054]  </TASK>

The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20260421/[email protected]

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#1 lore raw

Hello, Thomas, Herbert Xu!

Quoted 78 lines

kernel test robot noticed "BUG:sleeping_function_called_from_invalid_context_at_mm/vmalloc.c" on:

commit: c6307674ed82c0c57d6e1e3408e84ac449ab8e94 ("mm: kvmalloc: add non-blocking support for vmalloc")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

[test failed on linus/master 40286d6379aacfcc053253ef78dc78b09addffda]
[test failed on linux-next/master 936c21068d7ade00325e40d82bfd2f3f29d9f659]

in testcase: kunit
version:
with following parameters:

	group: group-01

config: x86_64-rhel-9.4-kunit
compiler: gcc-14
test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (Haswell) with 16G memory

(please refer to attached dmesg/kmsg for entire log/backtrace)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]

[  149.217862][ T4054] BUG: sleeping function called from invalid context at mm/vmalloc.c:3435
[  149.226291][ T4054] in_atomic(): 0, irqs_disabled(): 0, non_block: 0, pid: 4054, name: rhashtable_thra
[  149.235634][ T4054] preempt_count: 0, expected: 0
[  149.240353][ T4054] RCU nest depth: 1, expected: 0
[  149.245166][ T4054] CPU: 1 UID: 0 PID: 4054 Comm: rhashtable_thra Tainted: G S  B            N  6.18.0-rc5-00047-gc6307674ed82 #1 PREEMPT(voluntary)
[  149.245176][ T4054] Tainted: [S]=CPU_OUT_OF_SPEC, [B]=BAD_PAGE, [N]=TEST
[  149.245178][ T4054] Hardware name: Dell Inc. OptiPlex 9020/0DNKMN, BIOS A05 12/05/2013
[  149.245181][ T4054] Call Trace:
[  149.245185][ T4054]  <TASK>
[  149.245189][ T4054]  dump_stack_lvl (lib/dump_stack.c:122)
[  149.245200][ T4054]  __might_resched.cold (kernel/sched/core.c:8928)
[  149.245209][ T4054]  vfree (include/linux/kernel.h:61 mm/vmalloc.c:3435)
[  149.245215][ T4054]  rhashtable_try_insert (lib/rhashtable.c:476 lib/rhashtable.c:635)
[  149.245223][ T4054]  ? sysvec_apic_timer_interrupt (arch/x86/include/asm/hardirq.h:78 (discriminator 2) arch/x86/kernel/apic/apic.c:1052 (discriminator 2))
[  149.245229][ T4054]  ? sysvec_apic_timer_interrupt (arch/x86/include/asm/hardirq.h:78 (discriminator 2) arch/x86/kernel/apic/apic.c:1052 (discriminator 2))
[  149.245235][ T4054]  ? __pfx_rhashtable_try_insert (lib/rhashtable.c:592)
[  149.245242][ T4054]  rhashtable_insert_slow (lib/rhashtable.c:648)
[  149.245247][ T4054] insert_retry+0x257/0x630 test_rhashtable
[  149.245260][ T4054] threadfunc (lib/test_rhashtable.c:633) test_rhashtable
[  149.245269][ T4054]  ? try_to_wake_up (kernel/sched/core.c:4297)
[  149.245275][ T4054]  ? _raw_spin_lock_irqsave (arch/x86/include/asm/atomic.h:107 (discriminator 4) include/linux/atomic/atomic-arch-fallback.h:2170 (discriminator 4) include/linux/atomic/atomic-instrumented.h:1302 (discriminator 4) include/asm-generic/qspinlock.h:111 (discriminator 4) include/linux/spinlock.h:187 (discriminator 4) include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
[  149.245281][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245289][ T4054]  ? __pfx_autoremove_wake_function (kernel/sched/wait.c:402)
[  149.245295][ T4054]  ? __kthread_parkme (arch/x86/include/asm/bitops.h:202 (discriminator 1) arch/x86/include/asm/bitops.h:232 (discriminator 1) include/asm-generic/bitops/instrumented-non-atomic.h:142 (discriminator 1) kernel/kthread.c:290 (discriminator 1))
[  149.245301][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245309][ T4054]  ? __pfx_threadfunc (lib/test_rhashtable.c:618) test_rhashtable
[  149.245317][ T4054]  kthread (kernel/kthread.c:463)
[  149.245322][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245326][ T4054]  ? __pfx__raw_spin_lock_irq (kernel/locking/spinlock.c:169)
[  149.245331][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245335][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245339][ T4054]  ret_from_fork (arch/x86/kernel/process.c:164)
[  149.245344][ T4054]  ? __pfx_kthread (kernel/kthread.c:412)
[  149.245348][ T4054]  ret_from_fork_asm (arch/x86/entry/entry_64.S:255)
[  149.245356][ T4054]  </TASK>

The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20260421/[email protected]

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

<snip>

lib/rhashtable.c +1 -1
index 6074ed5f66f3..a892766dcedd 100644
@@ -473,7 +473,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		call_rcu(&new_tbl->rcu, bucket_table_free_rcu);
 		if (err == -EEXIST)
 			err = 0;
 	} else

<snip>

Could you please check? The problem here is

rcu_read_lock()

    rhashtable_insert_rehash()
        vfree();

rcu_read_unlokc();

we can just defer freeing via call_rcu().

--
Uladzislau Rezki

maintainer
#2 lore raw
Quoted 17 lines · Uladzislau Rezki

<snip>

lib/rhashtable.c +1 -1
index 6074ed5f66f3..a892766dcedd 100644
@@ -473,7 +473,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		call_rcu(&new_tbl->rcu, bucket_table_free_rcu);
 		if (err == -EEXIST)
 			err = 0;
 	} else
<snip>
Quoted 8 lines

Could you please check? The problem here is

rcu_read_lock()

    rhashtable_insert_rehash()
        vfree();

rcu_read_unlokc();

we can just defer freeing via call_rcu().

I think we should fix that warning instead. The memory that's being freed here was just allocated in exactly the same context through kvmalloc with GFP_ATOMIC. I can't see why you're allowed to call kvmalloc but not kvfree.

That makes no sense.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#3 lore raw
Quoted 25 lines · Herbert Xu
Quoted 24 lines · Uladzislau Rezki

<snip>

lib/rhashtable.c +1 -1
index 6074ed5f66f3..a892766dcedd 100644
@@ -473,7 +473,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		call_rcu(&new_tbl->rcu, bucket_table_free_rcu);
 		if (err == -EEXIST)
 			err = 0;
 	} else
<snip>

Could you please check? The problem here is

rcu_read_lock()

    rhashtable_insert_rehash()
        vfree();

rcu_read_unlokc();

Quoted 9 lines
Quoted 1 line

we can just defer freeing via call_rcu().

I think we should fix that warning instead. The memory that's being freed here was just allocated in exactly the same context through kvmalloc with GFP_ATOMIC. I can't see why you're allowed to call kvmalloc but not kvfree.

That makes no sense.

There is vfree_atomic() implementation. Regular vfree(), historically is not supposed to be called from non-sleeping contexts:

<snip>
* Context:
* May sleep if called *not* from interrupt context.
* Must not be called in NMI context (strictly speaking, it could be
* if we have CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG, but making the calling
* conventions for vfree() arch-dependent would be a really bad idea). */
void vfree(const void *addr)

{
	struct vm_struct *vm;
	int i;

	if (unlikely(in_interrupt())) {
		vfree_atomic(addr);
		return;
	}

	BUG_ON(in_nmi());
	kmemleak_free(addr);
	might_sleep();

<snip>

since your GFP_ATOMIC context allocated chunk of memory using vmalloc() sense kvfree() uses vfree(). The simplest fix to free via RCU.

--
Uladzislau Rezki

maintainer
#4 lore raw
Quoted 3 lines · Uladzislau Rezki

since your GFP_ATOMIC context allocated chunk of memory using vmalloc() sense kvfree() uses vfree(). The simplest fix to free via RCU.

Why is kvmalloc returning vmalloc memory under GFP_ATOMIC? Is there a new GFP flag to tell it to not return vmalloc memory?

This has been working for over a decade, what changed?

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
maintainer
#5 lore raw

PATCH 1/1 rhashtable: Check for vmalloc in emergency rehash error path

Quoted 5 lines · Herbert Xu

Why is kvmalloc returning vmalloc memory under GFP_ATOMIC? Is there a new GFP flag to tell it to not return vmalloc memory?

This has been working for over a decade, what changed?

Nevermind, I found your commit to add vmalloc support for GFP_ATOMIC.

---8<---
As kvmalloc with GFP_ATOMIC can now return vmalloc memory, check for it when freeing while in the same context so that vfree isn't called in an atomic context.

Use RCU to defer the freeing if the memory came from vmalloc.

Fixes c6307674ed82 ("mm: kvmalloc: add non-blocking support for vmalloc")
lib/rhashtable.c +11 -1
index 6074ed5f66f3..c386f456e417 100644
@@ -119,6 +119,16 @@ static void bucket_table_free_rcu(struct rcu_head *head)
 	bucket_table_free(container_of(head, struct bucket_table, rcu));
 }
 
+static void bucket_table_free_atomic(struct bucket_table *tbl)
+{
+	if (is_vmalloc_addr(tbl)) {
+		call_rcu(&tbl->rcu, bucket_table_free_rcu);
+		return;
+	}
+
+	bucket_table_free(tbl);
+}
+
 static union nested_table *nested_table_alloc(struct rhashtable *ht,
 					      union nested_table __rcu **prev,
 					      bool leaf)
@@ -473,7 +483,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		bucket_table_free_atomic(new_tbl);
 		if (err == -EEXIST)
 			err = 0;
 	} else
Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#6 lore raw
In reply to Herbert Xu
Quoted 10 lines · Herbert Xu
Quoted 3 lines · Uladzislau Rezki

since your GFP_ATOMIC context allocated chunk of memory using vmalloc() sense kvfree() uses vfree(). The simplest fix to free via RCU.

Why is kvmalloc returning vmalloc memory under GFP_ATOMIC? Is there a new GFP flag to tell it to not return vmalloc memory?

This has been working for over a decade, what changed?

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Any thoughts?

--
Uladzislau Rezki

maintainer
#7 lore raw
Quoted 8 lines · Uladzislau Rezki

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Any thoughts?

Perhaps add a kvfree_atomic that just calls vfree_atomic?

For rhashtable it really makes no difference either way. But it would eliminate the unsightly call to is_vmalloc_addr in rhashtable.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#8 lore raw
Quoted 11 lines · Herbert Xu
Quoted 8 lines · Uladzislau Rezki

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Any thoughts?

Perhaps add a kvfree_atomic that just calls vfree_atomic?

kvfree()'s comment says

" * Context: Either preemptible task context or not-NMI interrupt."

so this is neither. It might be ok then to create kvfree_atomic(). Always using vfree_atomic() from kvfree() might be wasteful.

Quoted 4 lines

For rhashtable it really makes no difference either way. But it would eliminate the unsightly call to is_vmalloc_addr in rhashtable.

Thanks,

#9 lore raw
Quoted 9 lines · Vlastimil Babka
Quoted 8 lines · Herbert Xu
Quoted 7 lines · Uladzislau Rezki

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Any thoughts?

Perhaps add a kvfree_atomic that just calls vfree_atomic?

kvfree()'s comment says

" * Context: Either preemptible task context or not-NMI interrupt."

I am not sure the description clearly reflects the intended usage. To me it sounds like all contexts but excluding NMI.

For example, calling this under spin_lock() will be invalid, as the vfree() path may invoke cond_resched().

Quoted 6 lines

so this is neither. It might be ok then to create kvfree_atomic(). Always using vfree_atomic() from kvfree() might be wasteful.

Quoted 3 lines

For rhashtable it really makes no difference either way. But it would eliminate the unsightly call to is_vmalloc_addr in rhashtable.

Seems atomic version makes sense here:

<snip>

include/linux/slab.h +3 -0
index 15a60b501b95..2b5ab488e96b 100644
@@ -1234,6 +1234,9 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 extern void kvfree(const void *addr);
 DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))
 
+extern void kvfree_atomic(const void *addr);
+DEFINE_FREE(kvfree_atomic, void *, if (!IS_ERR_OR_NULL(_T)) kvfree_atomic(_T))
+
 extern void kvfree_sensitive(const void *addr, size_t len);
 
 unsigned int kmem_cache_size(struct kmem_cache *s);
mm/slub.c +16 -0
index 2b2d33cc735c..e25a0eab6ff7 100644
@@ -6802,6 +6802,22 @@ void kvfree(const void *addr)
 }
 EXPORT_SYMBOL(kvfree);
 
+/**
+ * kvfree_atomic() - Free memory.
+ * @addr: Pointer to allocated memory.
+ *
+ * Same as kvfree(), but safe to use in atomic contexts.
+ * Must not be called from NMI context.
+ */
+void kvfree_atomic(const void *addr)
+{
+	if (is_vmalloc_addr(addr))
+		vfree_atomic(addr);
+	else
+		kfree(addr);
+}
+EXPORT_SYMBOL(kvfree_atomic);
+
 /**
  * kvfree_sensitive - Free a data object containing sensitive information.
  * @addr: address of the data object to be freed.

<snip>

I can post it if no objections.

--
Uladzislau Rezki

#10 lore raw
Quoted 12 lines · Uladzislau Rezki
Quoted 11 lines · Vlastimil Babka
Quoted 10 lines · Herbert Xu
Quoted 8 lines · Uladzislau Rezki

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Any thoughts?

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Perhaps add a kvfree_atomic that just calls vfree_atomic?

kvfree()'s comment says

" * Context: Either preemptible task context or not-NMI interrupt."

I am not sure the description clearly reflects the intended usage. To me it sounds like all contexts but excluding NMI.

Agree it's not clear, my immediate reaction was the same and took me a bit to understand.

Quoted 2 lines

For example, calling this under spin_lock() will be invalid, as the vfree() path may invoke cond_resched().

Yeah, it's covered by the "preemptible" word, but not super clear. IIUC it all boils down to this in vfree():

        if (unlikely(in_interrupt())) {
                vfree_atomic(addr);
                return;
        }

Which can do the right thing in an interrupt, but obviously cannot automatically recognize the "under spin_lock()" situation.

So I don't think we can change it now with all the existing callers, but if a better wording exists, we could make the comment more clear.

Quoted 52 lines
Quoted 6 lines

so this is neither. It might be ok then to create kvfree_atomic(). Always using vfree_atomic() from kvfree() might be wasteful.

Quoted 3 lines

For rhashtable it really makes no difference either way. But it would eliminate the unsightly call to is_vmalloc_addr in rhashtable.

Seems atomic version makes sense here:

<snip>

include/linux/slab.h +3 -0
index 15a60b501b95..2b5ab488e96b 100644
@@ -1234,6 +1234,9 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 extern void kvfree(const void *addr);
 DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))
 
+extern void kvfree_atomic(const void *addr);
+DEFINE_FREE(kvfree_atomic, void *, if (!IS_ERR_OR_NULL(_T)) kvfree_atomic(_T))
+
 extern void kvfree_sensitive(const void *addr, size_t len);
 
 unsigned int kmem_cache_size(struct kmem_cache *s);
mm/slub.c +16 -0
index 2b2d33cc735c..e25a0eab6ff7 100644
@@ -6802,6 +6802,22 @@ void kvfree(const void *addr)
 }
 EXPORT_SYMBOL(kvfree);
 
+/**
+ * kvfree_atomic() - Free memory.
+ * @addr: Pointer to allocated memory.
+ *
+ * Same as kvfree(), but safe to use in atomic contexts.
+ * Must not be called from NMI context.
+ */
+void kvfree_atomic(const void *addr)
+{
+	if (is_vmalloc_addr(addr))
+		vfree_atomic(addr);
+	else
+		kfree(addr);
+}
+EXPORT_SYMBOL(kvfree_atomic);
+
 /**
  * kvfree_sensitive - Free a data object containing sensitive information.
  * @addr: address of the data object to be freed.
<snip>
Quoted 1 line

I can post it if no objections.

LGTM but also should be now used in rhashtable_try_insert() I think?

Quoted 3 lines

--
Uladzislau Rezki

#11 lore raw
Quoted 95 lines · Vlastimil Babka
Quoted 12 lines · Uladzislau Rezki
Quoted 11 lines · Vlastimil Babka
Quoted 10 lines · Herbert Xu
Quoted 8 lines · Uladzislau Rezki

We have added non-sleeping flags for vmalloc() to extend kvmalloc() functionality as folk need those.

Another option, would be: always use vfree_atomic() from the kvfree() path.

Any thoughts?

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Perhaps add a kvfree_atomic that just calls vfree_atomic?

kvfree()'s comment says

" * Context: Either preemptible task context or not-NMI interrupt."

I am not sure the description clearly reflects the intended usage. To me it sounds like all contexts but excluding NMI.

Agree it's not clear, my immediate reaction was the same and took me a bit to understand.

Quoted 2 lines

For example, calling this under spin_lock() will be invalid, as the vfree() path may invoke cond_resched().

Yeah, it's covered by the "preemptible" word, but not super clear. IIUC it all boils down to this in vfree():

        if (unlikely(in_interrupt())) {
                vfree_atomic(addr);
                return;
        }

Which can do the right thing in an interrupt, but obviously cannot automatically recognize the "under spin_lock()" situation.

So I don't think we can change it now with all the existing callers, but if a better wording exists, we could make the comment more clear.

Quoted 52 lines
Quoted 6 lines

so this is neither. It might be ok then to create kvfree_atomic(). Always using vfree_atomic() from kvfree() might be wasteful.

Quoted 3 lines

For rhashtable it really makes no difference either way. But it would eliminate the unsightly call to is_vmalloc_addr in rhashtable.

Seems atomic version makes sense here:

<snip>

include/linux/slab.h +3 -0
index 15a60b501b95..2b5ab488e96b 100644
@@ -1234,6 +1234,9 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 extern void kvfree(const void *addr);
 DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))
 
+extern void kvfree_atomic(const void *addr);
+DEFINE_FREE(kvfree_atomic, void *, if (!IS_ERR_OR_NULL(_T)) kvfree_atomic(_T))
+
 extern void kvfree_sensitive(const void *addr, size_t len);
 
 unsigned int kmem_cache_size(struct kmem_cache *s);
mm/slub.c +16 -0
index 2b2d33cc735c..e25a0eab6ff7 100644
@@ -6802,6 +6802,22 @@ void kvfree(const void *addr)
 }
 EXPORT_SYMBOL(kvfree);
 
+/**
+ * kvfree_atomic() - Free memory.
+ * @addr: Pointer to allocated memory.
+ *
+ * Same as kvfree(), but safe to use in atomic contexts.
+ * Must not be called from NMI context.
+ */
+void kvfree_atomic(const void *addr)
+{
+	if (is_vmalloc_addr(addr))
+		vfree_atomic(addr);
+	else
+		kfree(addr);
+}
+EXPORT_SYMBOL(kvfree_atomic);
+
 /**
  * kvfree_sensitive - Free a data object containing sensitive information.
  * @addr: address of the data object to be freed.
<snip>
Quoted 4 lines
Quoted 1 line

I can post it if no objections.

LGTM but also should be now used in rhashtable_try_insert() I think?

Like below:

rhashtable_try_insert()
rhashtable_insert_rehash()

    bucket_table_free_atomic(new_tbl);

the patch for rhashtable.c can look like:

<snip>

lib/rhashtable.c +9 -1
index 6074ed5f66f3..4111aab8cee4 100644
@@ -114,6 +114,14 @@ static void bucket_table_free(const struct bucket_table *tbl)
 	kvfree(tbl);
 }
 
+static void bucket_table_free_atomic(const struct bucket_table *tbl)
+{
+	if (tbl->nest)
+		nested_bucket_table_free(tbl);
+
+	kvfree_atomic(tbl);
+}
+
 static void bucket_table_free_rcu(struct rcu_head *head)
 {
 	bucket_table_free(container_of(head, struct bucket_table, rcu));
@@ -473,7 +481,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		bucket_table_free_atomic(new_tbl);
 		if (err == -EEXIST)
 			err = 0;
 	} else

<snip>

--
Uladzislau Rezki

maintainer Acked-by
#12 lore raw
Quoted 10 lines · Uladzislau Rezki

Like below:

rhashtable_try_insert()
rhashtable_insert_rehash()

    bucket_table_free_atomic(new_tbl);

the patch for rhashtable.c can look like:

<snip>

If you want to send that as a submission you can add my ack:

Alternatively, could you send both patches to linux-crypto and I can push them through my tree.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#13 lore raw
Quoted 10 lines · Herbert Xu
Quoted 9 lines · Uladzislau Rezki

Like below:

rhashtable_try_insert()
rhashtable_insert_rehash()

    bucket_table_free_atomic(new_tbl);

the patch for rhashtable.c can look like:

Quoted 9 lines
Quoted 1 line

<snip>

If you want to send that as a submission you can add my ack:

Alternatively, could you send both patches to linux-crypto and I can push them through my tree.

I will send out to the linux-crypto mailing list.

--
Uladzislau Rezki