mm/hugetlb_cma: Fix null nodemask dereference in hugetlb_cma_alloc_frozen_folio
Subsystems (2)
PATCH v6 1/1 mm/hugetlb_cma: Fix null nodemask dereference in hugetlb_cma_alloc_frozen_folio
alloc_buddy_hugetlb_folio_with_mpol() can pass a NULL nodemask to alloc_fresh_hugetlb_folio() as a fallback to allocate from all nodes. If order is gigantic, alloc_fresh_hugetlb_folio() propagates the NULL nodemask down to hugetlb_cma_alloc_frozen_folio() via alloc_gigantic_frozen_folio().
Additionally, hugetlb_cma_alloc_frozen_folio() previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding a node_isset(nid, *nodemask) check ensures the initial preferred node allocation honors the memory policy / nodemask.
However, hugetlb_cma_alloc_frozen_folio() dereferences the nodemask in node_isset(nid, *nodemask) and for_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.
Fix this by checking if nodemask is NULL in hugetlb_cma_alloc_frozen_folio() and defaulting it to cpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.
From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with MPOL_PREFERRED_MANY on a system where CMA is only configured on a subset of NUMA nodes.
This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using mbind() with MPOL_PREFERRED_MANY, and triggers a page fault:
void *ptr = mmap(NULL, 1UL << 30, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_1GB | MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30, MPOL_PREFERRED_MANY, &nodemask,
sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */
This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650
---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpuset mems_allowed updates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)
mm/hugetlb_cma.c +20 -3
@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h>
@@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;if (!hugetlb_cma_size)return NULL ; -if (hugetlb_cma[nid]) +retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask; + }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;
@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio);
Signature · 1 line
2.55.0
Muchun Song
maintainerQuoted 97 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;
mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 10 lines
mm/hugetlb_cma.c +5
-1if (!hugetlb_cma_size)return NULL ; -if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;
I don't think we need a local_node_mask, nmask = &cpuset_current_mems_allowed is enough. nodemask_t could be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-2+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio);Signature · 1 line
2.55.0
Quoted 101 lines · Muchun Song
Quoted 101 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;if (!hugetlb_cma_size)return NULL ;
Quoted 10 lines
mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 6 lines
mm/hugetlb_cma.c +5
-1-if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;I don't think we need a
local_node_mask, nmask =&cpuset_current_mems_allowedis enough.nodemask_tcould be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.
But does not local_node_mask capture current snapshot for cpuset_current_mems_allowed after first taking a reference via read_mems_allowed_begin() which ensures that nmask has a constant value there after. Otherwise nmask could just change value by the time it actually gets used.
Quoted 35 lines
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-2+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio);Signature · 1 line
2.55.0
Quoted 114 lines · Anshuman Khandual
Quoted 101 lines · Muchun Song
Quoted 101 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;if (!hugetlb_cma_size)return NULL ;Quoted 10 lines
mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 6 lines
mm/hugetlb_cma.c +5
-1-if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;I don't think we need a
local_node_mask, nmask =&cpuset_current_mems_allowedis enough.nodemask_tcould be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.
Quoted 5 lines
But does not
local_node_maskcapture current snapshot forcpuset_current_mems_allowedafter first taking a reference viaread_mems_allowed_begin()which ensures that nmask has a constant value there after. Otherwise nmask could just change value by the time it actually gets used.
Hi,
Apologies for the poor netiquette of rushing out v7 so quickly (I just saw your message after sending v7)! I am heading out on a long vacation starting this Thursday, so I wanted to get an updated version out for review before leaving.
Regarding local_node_mask: I thought the seqcount mechanism (read_mems_allowed_begin() / read_mems_allowed_retry()) guarantees correctness here without needing a stack copy. If current->mems_allowed changes at any point while nmask is being used (even during allocation) and the allocation fails, read_mems_allowed_retry() detects the sequence change and jumps back to retry_cpuset to retry with the updated nodemask.
I've incorporated this into v7 (Sorry again - for sending it so fast):
https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
Sourav
Quoted 35 lines
Quoted 35 lines
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-3+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio); --2.55.0
Quoted 121 lines · Sourav Panda
Quoted 114 lines · Anshuman Khandual
Quoted 101 lines · Muchun Song
Quoted 101 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;if (!hugetlb_cma_size)return NULL ;Quoted 10 lines
mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 6 lines
mm/hugetlb_cma.c +5
-1-if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;I don't think we need a
local_node_mask, nmask =&cpuset_current_mems_allowedis enough.nodemask_tcould be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.Quoted 5 lines
But does not
local_node_maskcapture current snapshot forcpuset_current_mems_allowedafter first taking a reference viaread_mems_allowed_begin()which ensures that nmask has a constant value there after. Otherwise nmask could just change value by the time it actually gets used.
Quoted 6 lines
Hi,
Apologies for the poor netiquette of rushing out v7 so quickly (I just saw your message after sending v7)! I am heading out on a long vacation starting this Thursday, so I wanted to get an updated version out for review before leaving.
Not a problem.
Quoted 8 lines
Regarding
local_node_mask: I thought the seqcount mechanism (read_mems_allowed_begin()/read_mems_allowed_retry()) guarantees correctness here without needing a stack copy. Ifcurrent->mems_allowedchanges at any point while nmask is being used (even during allocation) and the allocation fails,read_mems_allowed_retry()detects the sequence change and jumps back toretry_cpusetto retry with the updated nodemask.
read_mems_allowed_begin() --> read_mems_allowed_retry() --> Retry seems to be sufficient for retrying allocation again if and when the nodemask changes while causing an allocation failure.
But is there a possibility for the loop inside for_each_node_mask() to detect a changed node mask and then successfully allocate from there ? I guess that is semantically correct as the allocation happened from task's cpuset_current_mems_allowed which might have changed afterwards.
Dropping the local variable here seems right.
Quoted 42 lines
I've incorporated this into v7 (Sorry again - for sending it so fast):
https://lore.kernel.org/linux-mm/[email protected]/Thanks,
SouravQuoted 35 lines
Quoted 35 lines
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-3+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio); --2.55.0
Muchun Song
maintainerQuoted 117 lines · Anshuman Khandual
Quoted 101 lines · Muchun Song
Quoted 101 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;if (!hugetlb_cma_size)return NULL ;Quoted 10 lines
mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 6 lines
mm/hugetlb_cma.c +5
-1-if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;I don't think we need a
local_node_mask, nmask =&cpuset_current_mems_allowedis enough.nodemask_tcould be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.
Quoted 4 lines
But does not
local_node_maskcapture current snapshot forcpuset_current_mems_allowedafter first taking a reference viaread_mems_allowed_begin()which ensures that nmask has a constant value there after. Otherwise nmask could just change value by the time it actually gets used.
However, this value can change while you are copying it, so a snapshot itself is not a consistent value. Right?
Quoted 35 lines
Quoted 35 lines
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-2+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio);Signature · 1 line
2.55.0
Quoted 117 lines · Muchun Song
Quoted 117 lines · Anshuman Khandual
Quoted 111 lines · Muchun Song
Quoted 97 lines · Sourav Panda
alloc_buddy_hugetlb_folio_with_mpol()can pass a NULL nodemask toalloc_fresh_hugetlb_folio()as a fallback to allocate from all nodes. If order is gigantic,alloc_fresh_hugetlb_folio()propagates the NULL nodemask down tohugetlb_cma_alloc_frozen_folio()viaalloc_gigantic_frozen_folio().Additionally,
hugetlb_cma_alloc_frozen_folio()previously attempted allocation on hugetlb_cma[nid] without verifying if nid is included in the caller's nodemask. Adding anode_isset(nid, *nodemask)check ensures the initial preferred node allocation honors the memory policy / nodemask.However,
hugetlb_cma_alloc_frozen_folio()dereferences the nodemask innode_isset(nid, *nodemask)andfor_each_node_mask(node, *nodemask), leading to a null pointer dereference kernel panic when nodemask is NULL.Fix this by checking if nodemask is NULL in
hugetlb_cma_alloc_frozen_folio()and defaulting it tocpuset_current_mems_allowed. Enclose the allocation attempts within the cpuset seqcount retry loop so that if the cpuset changes concurrently during allocation, the attempts are retried using the updated nodemask. This ensures that the initial node check and fallback loop safely honor the task's cpuset without violating cpuset constraints or causing NULL pointer dereferences or unexpected allocation failures.From a userspace perspective, this bug allows an unprivileged user to crash the kernel (trigger a panic) by requesting a gigantic hugepage allocation with
MPOL_PREFERRED_MANYon a system where CMA is only configured on a subset of NUMA nodes.This can be reproduced by booting a VM with two NUMA nodes, restricting CMA to Node 1 (e.g., hugetlb_cma=1:1G default_hugepagesz=1G hugepagesz=1G hugepages=0), and running a program that allocates a 1GB hugepage area without reserving, restricts allocation to Node 0 using
mbind()withMPOL_PREFERRED_MANY, and triggers a page fault:void *ptr = mmap(NULL, 1UL << 30,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB|MAP_HUGE_1GB|MAP_NORESERVE, -1, 0);
unsigned long nodemask = 1; /* Node 0 */ mbind(ptr, 1UL << 30,MPOL_PREFERRED_MANY, &nodemask,sizeof(nodemask) * 8, 0);
memset(ptr, 0, 1UL << 30); /* Trigger fault */This results in a NULL pointer dereference:
BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not-present page Oops: Oops: 0000 [#1] SMP NOPTI RIP: 0010:hugetlb_cma_alloc_frozen_folio+0x75/0x120 Call Trace: <TASK> only_alloc_fresh_hugetlb_folio.isra.0+0x2c/0x160 alloc_surplus_hugetlb_folio+0x6d/0x100 alloc_hugetlb_folio+0x3c5/0x660 hugetlb_no_page+0x3d9/0x650Fixes eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")Signed-off-by Sourav Panda <[email protected]>---
Changes in v6:
- Enclosed the CMA allocation attempts within the cpuset seqcount retry loop, retrying allocation upon cpusetmems_allowedupdates to prevent unexpected allocation failures as suggested by Muchun Song.
- v5: https://lore.kernel.org/linux-mm/[email protected]/
- v4: https://lore.kernel.org/linux-mm/[email protected]/
- v3: https://lore.kernel.org/linux-mm/[email protected]/
- v2: https://lore.kernel.org/linux-mm/[email protected]/
- v1: https://lore.kernel.org/linux-mm/[email protected]/
mm/hugetlb_cma.c| 23 ++++++++++++++++++-----1 file changed, 18 insertions(+), 5 deletions(-)mm/hugetlb_cma.c +4
-0@@ -3,6 +3,7 @@#include <linux/mm.h> #include <linux/cma.h> #include <linux/compiler.h> +#include <linux/cpuset.h> #include <linux/mm_inline.h> #include <asm/page.h> @@ -30,15 +31,27 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,int node;struct folio *folio;struct page *page =NULL ; +const nodemask_t *nmask; +nodemask_t local_node_mask; +unsigned int cpuset_mems_cookie;mm/hugetlb_cma.c struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, · line 45
Quoted 10 lines
mm/hugetlb_cma.c +5
-1if (!hugetlb_cma_size)return NULL ; -if (hugetlb_cma[nid]) + retry_cpuset: +if (!nodemask) { + cpuset_mems_cookie =read_mems_allowed_begin (); + local_node_mask = cpuset_current_mems_allowed; + nmask = &local_node_mask;I don't think we need a
local_node_mask, nmask =&cpuset_current_mems_allowedis enough.nodemask_tcould be a huge array, so it might consume a lot of memory on the stack. I don't think we need to do that unless we see a clear necessity.
Quoted 7 lines
Quoted 4 lines
But does not
local_node_maskcapture current snapshot forcpuset_current_mems_allowedafter first taking a reference viaread_mems_allowed_begin()which ensures that nmask has a constant value there after. Otherwise nmask could just change value by the time it actually gets used.However, this value can change while you are copying it, so a snapshot itself is not a consistent value. Right?
Yes it can change while copying, although it stays consistent for the for loop later but that might not be necessary after all. As mentioned earlier it seems OK drop the local variable here.
Quoted 38 lines
Quoted 36 lines
Quoted 36 lines
Otherwise, LGTM.
Thanks.
Quoted 29 lines
mm/hugetlb_cma.c +11
-2+ }else { + nmask = nodemask; + } + +if (hugetlb_cma[nid] &&node_isset (nid, *nmask)) page =cma_alloc_frozen_compound (hugetlb_cma[nid], order);if (!page && !(gfp_mask & __GFP_THISNODE)) { -for_each_node_mask (node, *nodemask) { +for_each_node_mask (node, *nmask) {if (node == nid || !hugetlb_cma[node])continue ;@@ -48,8 +61,12 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask, } } -if (!page) +if (!page) { +if (!nodemask && +unlikely (read_mems_allowed_retry (cpuset_mems_cookie))) + goto retry_cpuset;return NULL ; + } folio =page_folio (page);folio_set_hugetlb_cma (folio);Signature · 1 line
2.55.0