Module load relocation hashtable size can become huge and unallocatable
Hello, I have a kernel module that requires a huge number of relocations upon loading.
On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to kmalloc_array.
Right here I've dumped the values by recompiling the kernel:num_relocations: 1630985, hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734
which kmalloc_array->...->___kmalloc_large_node->...-> __alloc_frozen_pages_noprof can't handle and returns "Cannot allocate memory" on insmod.
How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.
Thank you,
Will Pierce
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Using vmalloc_array/vfree on the relocation_hashtable allows the kernel module to load just fine. I can send this as a patch if the fix sounds good.
Quoted 15 lines · William Pierce
Hello, I have a kernel module that requires a huge number of relocations upon loading.
On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to
kmalloc_array.Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985,hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734which kmalloc_array->...->___kmalloc_large_node->...->
__alloc_frozen_pages_noprofcan't handle and returns "Cannot allocate memory" on insmod.
Quoted 7 lines
How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.Thank you,
Will Pierce
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 3 lines · William Pierce
Using vmalloc_array/vfree on the
relocation_hashtableallows the kernel module to load just fine. I can send this as a patch if the fix sounds good.
Hey William,
Using vmalloc sounds reasonable since there is no need to have physically contiguous memory.
However, there is also a new API named kvmalloc which uses kmalloc for small sizes and vmalloc for large allocations exceeding kmalloc capabilities, potentially leading to better performances than using vmalloc only.
Clément
Quoted 22 lines · William Pierce
Quoted 20 lines
Hello, I have a kernel module that requires a huge number of relocations upon loading.
On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to
kmalloc_array.Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985,hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734which kmalloc_array->...->___kmalloc_large_node->...->
__alloc_frozen_pages_noprofcan't handle and returns "Cannot allocate memory" on insmod.How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.
Quoted 8 lines
Quoted 2 lines
Thank you,
Will Pierce_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 12 lines · Clément Léger
Quoted 3 lines · William Pierce
Using vmalloc_array/vfree on the
relocation_hashtableallows the kernel module to load just fine. I can send this as a patch if the fix sounds good.Hey William,
Using vmalloc sounds reasonable since there is no need to have physically contiguous memory.
Quoted 7 lines
However, there is also a new API named kvmalloc which uses kmalloc for small sizes and vmalloc for large allocations exceeding kmalloc capabilities, potentially leading to better performances than using vmalloc only.
Clément
Hi Clément,
Thank you for checking. Yes I expect kvmalloc should also work just fine.
I can send out a patch based on that, unless someone wants to just take the change. I'll do so in the next day or so.
- Will
Quoted 22 lines
Quoted 22 lines · William Pierce
Quoted 20 lines
Hello, I have a kernel module that requires a huge number of relocations upon loading.
On other architectures, it loads into the kernel just fine. However, on riscv, while allocating memory for its relocation hashtable, it ends up requesting 2097152 elements to
kmalloc_array.Right here I've dumped the values by recompiling the kernel:
num_relocations: 1630985,hastable_size: 2097152
https://elixir.bootlin.com/linux/v6.14-rc6/source/arch/riscv/kernel/module.c#L734which kmalloc_array->...->___kmalloc_large_node->...->
__alloc_frozen_pages_noprofcan't handle and returns "Cannot allocate memory" on insmod.How would it be recommended to fix this?
I could imagine allocating a more discontiguous data structure to track the relocations or otherwise reworking the tracking to do the relocation tracking inline.
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Thank you,
Will Pierce_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH 1/1 riscv: Use kvmalloc_array on relocation_hashtable
From: Will Pierce <[email protected]>
The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.
--- arch/riscv/kernel/module.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/riscv/kernel/module.c +3 -2
@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me,kfree (bucket_iter); } -kfree (*relocation_hashtable); +kvfree (*relocation_hashtable); } static int add_relocation_to_accumulate(struct module *me, int type,
@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations, hashtable_size <<= should_double_size; - *relocation_hashtable =kmalloc_array (hashtable_size, +/* Number of relocations may be large, so kvmalloc it */ + *relocation_hashtable =kvmalloc_array (hashtable_size,sizeof (**relocation_hashtable),GFP_KERNEL );if (!*relocation_hashtable)
Signature · 7 lines
2.49.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH] riscv: Use kvmalloc_array on relocation_hashtable
Hi Will,
Quoted 24 lines · On 02/04/2025 07:34, [email protected] wrote:
From: Will Pierce <[email protected]>
The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.
Signed-off-by Will Pierce <[email protected]>--- arch/riscv/kernel/module.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)arch/riscv/kernel/module.c +1
-1@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me,kfree (bucket_iter); } -kfree (*relocation_hashtable); +kvfree (*relocation_hashtable); } static int add_relocation_to_accumulate(struct module *me, int type,
arch/riscv/kernel/module.c initialize_relocation_hashtable(unsigned int num_relocations, · line 759
Quoted 9 lines
arch/riscv/kernel/module.c +2
-1@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations, hashtable_size <<= should_double_size; - *relocation_hashtable =kmalloc_array (hashtable_size, +/* Number of relocations may be large, so kvmalloc it */ + *relocation_hashtable =kvmalloc_array (hashtable_size,sizeof (**relocation_hashtable),GFP_KERNEL );if (!*relocation_hashtable)
Let's add a fixes tag:
Since Clément suggested this solution:
And finally, this looks good to me so:
It's a bit late for rc1 so I'll take this one for a next rc.
Thanks for the patch!
Alex
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Re: [PATCH] riscv: Use kvmalloc_array on relocation_hashtable
Hi Alexandre,
Quoted 44 lines · Alexandre Ghiti
Hi Will,
Quoted 22 lines · On 02/04/2025 07:34, [email protected] wrote:
From: Will Pierce <[email protected]>
The number of relocations may be a huge value that is unallocatable by kmalloc. Use kvmalloc instead so that it does not fail.
Signed-off-by Will Pierce <[email protected]>---
arch/riscv/kernel/module.c| 5 +++--1 file changed, 3 insertions(+), 2 deletions(-)arch/riscv/kernel/module.c +1
-0@@ -648,7 +648,7 @@ process_accumulated_relocations(struct module *me, kfree(bucket_iter); } - kfree(*relocation_hashtable); + kvfree(*relocation_hashtable); } static int add_relocation_to_accumulate(struct module *me, int type,arch/riscv/kernel/module.c initialize_relocation_hashtable(unsigned int num_relocations, · line 758
Quoted 7 lines
arch/riscv/kernel/module.c +2
-0@@ -752,7 +752,8 @@ initialize_relocation_hashtable(unsigned int num_relocations, hashtable_size <<= should_double_size; - *relocation_hashtable = kmalloc_array(hashtable_size, + /* Number of relocations may be large, so kvmalloc it */ + *relocation_hashtable = kvmalloc_array(hashtable_size, sizeof(**relocation_hashtable), GFP_KERNEL); if (!*relocation_hashtable)Let's add a fixes tag:
Fixes 8fd6c5142395 ("riscv: Add remaining module relocations")Since Clément suggested this solution:
Suggested-by Clément Léger <[email protected]>And finally, this looks good to me so:
Quoted 9 lines
Reviewed-by Alexandre Ghiti <[email protected]>It's a bit late for rc1 so I'll take this one for a next rc.
Thanks for the patch!
Alex
Thank you for checking. I will send a v2 with these additions and a minor whitespace fix right now.
- Will
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv