Skip to contentSkip to conversation

Module load relocation hashtable size can become huge and unallocatable

BUG Superseded William Pierce 3 participants 7 messages Mar 30, 2025 – Apr 2, 2025 RISC-V

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

Hello, I have a kernel module that requires a huge number of relocations upon loading.

On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to kmalloc_array.

Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985, hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734

which kmalloc_array->...->___kmalloc_large_node->...-> __alloc_frozen_pages_noprof can't handle and returns "Cannot allocate memory" on insmod.

How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.

Thank you,
Will Pierce

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#1 lore raw

Using vmalloc_array/vfree on the relocation_hashtable allows the kernel module to load just fine. I can send this as a patch if the fix sounds good.

Quoted 15 lines · William Pierce

Hello, I have a kernel module that requires a huge number of relocations upon loading.

On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to kmalloc_array.

Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985, hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734

which kmalloc_array->...->___kmalloc_large_node->...-> __alloc_frozen_pages_noprof can't handle and returns "Cannot allocate memory" on insmod.

Quoted 7 lines

How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.

Thank you,
Will Pierce

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#2 lore raw
Quoted 3 lines · William Pierce

Using vmalloc_array/vfree on the relocation_hashtable allows the kernel module to load just fine. I can send this as a patch if the fix sounds good.

Hey William,

Using vmalloc sounds reasonable since there is no need to have physically contiguous memory.

However, there is also a new API named kvmalloc which uses kmalloc for small sizes and vmalloc for large allocations exceeding kmalloc capabilities, potentially leading to better performances than using vmalloc only.

Clément

Quoted 22 lines · William Pierce
Quoted 20 lines

Hello, I have a kernel module that requires a huge number of relocations upon loading.

On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to kmalloc_array.

Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985, hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734

which kmalloc_array->...->___kmalloc_large_node->...-> __alloc_frozen_pages_noprof can't handle and returns "Cannot allocate memory" on insmod.

How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.

Quoted 8 lines
Quoted 2 lines

Thank you,
Will Pierce

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#3 lore raw
Quoted 12 lines · Clément Léger
Quoted 3 lines · William Pierce

Using vmalloc_array/vfree on the relocation_hashtable allows the kernel module to load just fine. I can send this as a patch if the fix sounds good.

Hey William,

Using vmalloc sounds reasonable since there is no need to have physically contiguous memory.

Quoted 7 lines

However, there is also a new API named kvmalloc which uses kmalloc for small sizes and vmalloc for large allocations exceeding kmalloc capabilities, potentially leading to better performances than using vmalloc only.

Clément

Hi Clément,

Thank you for checking. Yes I expect kvmalloc should also work just fine.

I can send out a patch based on that, unless someone wants to just take the change. I'll do so in the next day or so.

- Will

Quoted 22 lines
Quoted 22 lines · William Pierce
Quoted 20 lines

Hello, I have a kernel module that requires a huge number of relocations upon loading.

On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to kmalloc_array.

Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985, hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734

which kmalloc_array->...->___kmalloc_large_node->...-> __alloc_frozen_pages_noprof can't handle and returns "Cannot allocate memory" on insmod.

How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.

Quoted 9 lines
Quoted 8 lines
Quoted 2 lines

Thank you,
Will Pierce

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#4 lore raw

PATCH 1/1 riscv: Use kvmalloc_array on relocation_hashtable

From: Will Pierce <[email protected]>

The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.

---
 arch/riscv/kernel/module.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/riscv/kernel/module.c +3 -2
index 47d0ebeec93c..bae3db50647c 100644
@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me,
 		kfree(bucket_iter);
 	}
 
-	kfree(*relocation_hashtable);
+	kvfree(*relocation_hashtable);
 }
 
 static int add_relocation_to_accumulate(struct module *me, int type,
@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations,
 
 	hashtable_size <<= should_double_size;
 
-	*relocation_hashtable = kmalloc_array(hashtable_size,
+	/* Number of relocations may be large, so kvmalloc it */
+	*relocation_hashtable = kvmalloc_array(hashtable_size,
 					      sizeof(**relocation_hashtable),
 					      GFP_KERNEL);
 	if (!*relocation_hashtable)
Signature · 7 lines
2.49.0


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
reviewer Reviewed-by on v1 1/1
#5 lore raw

Re: [PATCH] riscv: Use kvmalloc_array on relocation_hashtable

Hi Will,

Quoted 24 lines · On 02/04/2025 07:34, [email protected] wrote:

From: Will Pierce <[email protected]>

The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.

---
  arch/riscv/kernel/module.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
arch/riscv/kernel/module.c +1 -1
index 47d0ebeec93c..bae3db50647c 100644
@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me,
  		kfree(bucket_iter);
  	}
 
-	kfree(*relocation_hashtable);
+	kvfree(*relocation_hashtable);
  }
 
  static int add_relocation_to_accumulate(struct module *me, int type,

arch/riscv/kernel/module.c initialize_relocation_hashtable(unsigned int num_relocations, · line 759

Quoted 9 lines
arch/riscv/kernel/module.c +2 -1
@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations,
 
  	hashtable_size <<= should_double_size;
 
-	*relocation_hashtable = kmalloc_array(hashtable_size,
+	/* Number of relocations may be large, so kvmalloc it */
+	*relocation_hashtable = kvmalloc_array(hashtable_size,
  					      sizeof(**relocation_hashtable),
  					      GFP_KERNEL);
  	if (!*relocation_hashtable)

Let's add a fixes tag:

Fixes 8fd6c5142395 ("riscv: Add remaining module relocations")

Since Clément suggested this solution:

And finally, this looks good to me so:

It's a bit late for rc1 so I'll take this one for a next rc.

Thanks for the patch!

Alex

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#6 lore raw

Re: [PATCH] riscv: Use kvmalloc_array on relocation_hashtable

Hi Alexandre,

Quoted 44 lines · Alexandre Ghiti

Hi Will,

Quoted 22 lines · On 02/04/2025 07:34, [email protected] wrote:

From: Will Pierce <[email protected]>

The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.

---
  arch/riscv/kernel/module.c | 5 +++--

  1 file changed, 3 insertions(+), 2 deletions(-)
arch/riscv/kernel/module.c +1 -0
index 47d0ebeec93c..bae3db50647c 100644
@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me,
           kfree(bucket_iter);
       }
   -    kfree(*relocation_hashtable);
+    kvfree(*relocation_hashtable);
   }
     static int add_relocation_to_accumulate(struct module *me, int type,

arch/riscv/kernel/module.c initialize_relocation_hashtable(unsigned int num_relocations, · line 758

Quoted 7 lines
arch/riscv/kernel/module.c +2 -0
@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations,
         hashtable_size <<= should_double_size;
   -    *relocation_hashtable = kmalloc_array(hashtable_size,
+    /* Number of relocations may be large, so kvmalloc it */
+    *relocation_hashtable = kvmalloc_array(hashtable_size,
                             sizeof(**relocation_hashtable),
                             GFP_KERNEL);
       if (!*relocation_hashtable)

Let's add a fixes tag:

Fixes 8fd6c5142395 ("riscv: Add remaining module relocations")

Since Clément suggested this solution:

And finally, this looks good to me so:

Quoted 9 lines

It's a bit late for rc1 so I'll take this one for a next rc.

Thanks for the patch!

Alex

Thank you for checking. I will send a v2 with these additions and a minor whitespace fix right now.

- Will

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv