Fix kmalloc_node on offline nodes
Subsystems (6)
v2 cover Fix kmalloc_node on offline nodes
Changelog v1 -> v2:
- Handled comments from Vlastimil Babka and Bharata B Rao
- Changes only in patch 2 and 4.
Sachin recently reported that linux-next was no more bootable on few powerpc systems.
https://lore.kernel.org/linux-next/[email protected]/
# numactl -H available: 2 nodes (0-1) node 0 cpus: node 0 size: 0 MB node 0 free: 0 MB node 1 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 node 1 size: 35247 MB node 1 free: 30907 MB node distances: node 0 1 0: 10 40 1: 40 10 #
Sachin bisected the problem to Commit a75056fc1e7c ("mm/memcontrol.c: allocate shrinker_map on appropriate NUMA node")
The root cause analysis showed that mm/slub and powerpc/numa had some shortcomings with respect to offline nodes.
This patch series is on top of patches posted at
https://lore.kernel.org/linuxppc-dev/[email protected]/t/#u
Srikar Dronamraju (4):
mm: Check for node_online in node_present_pages mm/slub: Use mem_node to allocate a new slab
mm: Implement reset_numa_mem
powerpc/numa: Set fallback nodes for offline nodes
arch/powerpc/include/asm/topology.h | 16 ++++++++++++++++ arch/powerpc/kernel/smp.c | 1 + include/asm-generic/topology.h | 3 +++ include/linux/mmzone.h | 6 ++++-- include/linux/topology.h | 7 +++++++ mm/slub.c | 9 +++++---- 6 files changed, 36 insertions(+), 6 deletions(-)
Signature · 1 line
2.18.1
PATCH v2 1/4 mm: Check for node_online in node_present_pages
Calling a kmalloc_node on a possible node which is not yet onlined can lead to panic. Currently node_present_pages() doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.
NIP [c0000000003d55f4] ___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94] __slab_alloc+0x34/0x60
Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68
Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for node_spanned_pages() too.
--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
include/linux/mmzone.h +4 -2
@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t; -#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)#ifdef CONFIG_FLAT_NODE_MEM_MAP #define pgdat_page_nr (pgdat, pagenr) ((pgdat)->node_mem_map + (pagenr))#else
Signature · 1 line
2.18.1
Michal Hocko
reviewerRe: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
Quoted 51 lines · Srikar Dronamraju
Calling a
kmalloc_nodeon a possible node which is not yet onlined can lead to panic. Currentlynode_present_pages()doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.NIP [
c0000000003d55f4]___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94]__slab_alloc+0x34/0x60Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for
node_spanned_pages()too.Cc Andrew Morton <[email protected]>Cc Mel Gorman <[email protected]>Cc Michael Ellerman <[email protected]>Cc Sachin Sant <[email protected]>Cc Michal Hocko <[email protected]>Cc Christopher Lameter <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Srikar Dronamraju <[email protected]>Cc Bharata B Rao <[email protected]>Cc Nathan Lynch <[email protected]>Reported-by Sachin Sant <[email protected]>Tested-by Sachin Sant <[email protected]>Signed-off-by Srikar Dronamraju <[email protected]>--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)include/linux/mmzone.h +0
-0
include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 9 lines
include/linux/mmzone.h +4
-2@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t; -#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)
I believe this is a wrong approach. We really do not want to special case all the places which require NODE_DATA. Can we please go and allocate pgdat for all possible nodes?
The current state of memory less hacks subtle bugs poping up here and there just prove that we should have done that from the very begining IMHO.
Quoted 5 lines
include/linux/mmzone.h +0
-0#ifdef CONFIG_FLAT_NODE_MEM_MAP #define pgdat_page_nr (pgdat, pagenr) ((pgdat)->node_mem_map + (pagenr))#else Signature · 1 line
2.18.1
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
Quoted 55 lines · Michal Hocko
Quoted 54 lines · Srikar Dronamraju
Calling a
kmalloc_nodeon a possible node which is not yet onlined can lead to panic. Currentlynode_present_pages()doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.NIP [
c0000000003d55f4]___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94]__slab_alloc+0x34/0x60Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for
node_spanned_pages()too.Cc Andrew Morton <[email protected]>Cc Mel Gorman <[email protected]>Cc Michael Ellerman <[email protected]>Cc Sachin Sant <[email protected]>Cc Michal Hocko <[email protected]>Cc Christopher Lameter <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Srikar Dronamraju <[email protected]>Cc Bharata B Rao <[email protected]>Cc Nathan Lynch <[email protected]>Reported-by Sachin Sant <[email protected]>Tested-by Sachin Sant <[email protected]>Signed-off-by Srikar Dronamraju <[email protected]>--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)include/linux/mmzone.h +0
-0@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t;
Quoted 11 lines
include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 6 lines
include/linux/mmzone.h +4
-2-#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)I believe this is a wrong approach. We really do not want to special case all the places which require
NODE_DATA. Can we please go and allocate pgdat for all possible nodes?
I can do that but the question I had was should we make this change just for Powerpc or should the change be for other archs.
NODE_DATA initialization always seems to be in arch specific code.
The other archs that are affected seem to be mips, sh and sparc These archs seem to have making an assumption that NODE_DATA has to be local only,
For example on sparc / arch/sparc/mm/init_64.c in allocate_node_data function.
NODE_DATA(nid) = memblock_alloc_node(sizeof(struct pglist_data),
SMP_CACHE_BYTES, nid);
if (!NODE_DATA(nid)) {
prom_printf("Cannot allocate pglist_data for nid[%d]\n", nid);
prom_halt();
}NODE_DATA(nid)->node_id = nid;
So even if I make changes to allocate NODE_DATA from fallback node, I may not be able to test them.
So please let me know your thoughts around the same.
Quoted 10 lines
The current state of memory less hacks subtle bugs poping up here and there just prove that we should have done that from the very begining IMHO.
Quoted 5 lines
include/linux/mmzone.h +0
-1#ifdef CONFIG_FLAT_NODE_MEM_MAP #define pgdat_page_nr (pgdat, pagenr) ((pgdat)->node_mem_map + (pagenr))#else --2.18.1
Quoted 4 lines
--
Michal Hocko
SUSE Labs
--
Thanks and Regards
Srikar Dronamraju
Michal Hocko
reviewerRe: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
Quoted 3 lines · Srikar Dronamraju
Quoted 1 line · Michal Hocko
On Wed 18-03-20 12:58:07, Srikar Dronamraju wrote:
[...]
Quoted 12 lines
Quoted 11 lines
include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 6 lines
include/linux/mmzone.h +4
-2-#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)I believe this is a wrong approach. We really do not want to special case all the places which require
NODE_DATA. Can we please go and allocate pgdat for all possible nodes?
Quoted 2 lines
I can do that but the question I had was should we make this change just for Powerpc or should the change be for other archs.
No, we shouldn't, really. If NODE_DATA is non-null for all possible nodes then this shouldn't be really necessary and arch specific.
Quoted 5 lines
NODE_DATAinitialization always seems to be in arch specific code.The other archs that are affected seem to be mips, sh and sparc These archs seem to have making an assumption that
NODE_DATAhas to be local only,
Which is all good and fine for nodes that hold some memory. If those architectures support memory less nodes at all then I do not see any problem to have remote pgdata.
Quoted 10 lines
For example on sparc /
arch/sparc/mm/init_64.cinallocate_node_datafunction.NODE_DATA(nid) = memblock_alloc_node(sizeof(struct pglist_data), SMP_CACHE_BYTES, nid); if (!NODE_DATA(nid)) { prom_printf("Cannot allocate pglist_data for nid[%d]\n", nid); prom_halt(); }NODE_DATA(nid)->node_id = nid;
This code is not about memroy less nodes, is it? It looks more like a allocation failure panic-like handling because there is not enough memory to hold pgdat. This also strongly suggests that this platform doesn't really expect memory less nodes in the early init path.
Quoted 2 lines
So even if I make changes to allocate
NODE_DATAfrom fallback node, I may not be able to test them.
Please try to focus on the architecture you can test for. From the existing reports I have seen this looks mostly to be a problem for x86 and ppc
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
In reply to Michal HockoQuoted 66 lines · Michal Hocko
Quoted 51 lines · Srikar Dronamraju
Calling a
kmalloc_nodeon a possible node which is not yet onlined can lead to panic. Currentlynode_present_pages()doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.NIP [
c0000000003d55f4]___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94]__slab_alloc+0x34/0x60Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for
node_spanned_pages()too.Cc Andrew Morton <[email protected]>Cc Mel Gorman <[email protected]>Cc Michael Ellerman <[email protected]>Cc Sachin Sant <[email protected]>Cc Michal Hocko <[email protected]>Cc Christopher Lameter <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Srikar Dronamraju <[email protected]>Cc Bharata B Rao <[email protected]>Cc Nathan Lynch <[email protected]>Reported-by Sachin Sant <[email protected]>Tested-by Sachin Sant <[email protected]>Signed-off-by Srikar Dronamraju <[email protected]>--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)include/linux/mmzone.h +0
-0include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 9 lines
include/linux/mmzone.h +4
-2@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t; -#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)I believe this is a wrong approach. We really do not want to special case all the places which require
NODE_DATA. Can we please go and allocate pgdat for all possible nodes?
Quoted 3 lines
The current state of memory less hacks subtle bugs poping up here and there just prove that we should have done that from the very begining IMHO.
Yes. So here's an alternative proposal for fixing the current situation in SLUB, before the long-term solution of having all possible nodes provide valid pgdat with zonelists:
- fix SLUB with the hunk at the end of this mail - the point is to use NUMA_NO_NODE as fallback instead of node_to_mem_node()
- this removes all uses of node_to_mem_node (luckily it's just SLUB), kill it completely instead of trying to fix it up
- patch 1/4 is not needed with the fix
- perhaps many of your other patches are alss not needed
- once we get the long-term solution, some of the !node_online() checks can be removed
----8<----
mm/slub.c +10 -9
@@ -1511,7 +1511,7 @@ static inline struct page *alloc_slab_page(struct kmem_cache *s,struct page *page;unsigned int order =oo_order (oo); -if (node ==NUMA_NO_NODE ) +if (node ==NUMA_NO_NODE || !node_online (node)) page =alloc_pages (flags, order);else page =__alloc_pages_node (node, flags, order);
@@ -1973,8 +1973,6 @@ static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,if (node ==NUMA_NO_NODE ) searchnode =numa_mem_id (); -else if (!node_present_pages (node)) - searchnode =node_to_mem_node (node); object =get_partial_node (s,get_node (s, searchnode), c, flags);if (object || node !=NUMA_NO_NODE )
@@ -2568,12 +2566,15 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, redo:if (unlikely (!node_match (page, node))) { -int searchnode = node; - -if (node !=NUMA_NO_NODE && !node_present_pages (node)) - searchnode =node_to_mem_node (node); - -if (unlikely (!node_match (page, searchnode))) { +/* +* node_match() false implies node != NUMA_NO_NODE +* but if the node is not online and has no pages, just +* ignore the constraint +*/ +if ((!node_online (node) || !node_present_pages (node))) { + node =NUMA_NO_NODE ; + goto redo; + }else {stat (s,ALLOC_NODE_MISMATCH );deactivate_slab (s, page, c->freelist, c); goto new_slab;
Michal Hocko
reviewerRe: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
On Wed 18-03-20 12:53:32, Vlastimil Babka wrote:
[...]
Quoted 6 lines
Yes. So here's an alternative proposal for fixing the current situation in SLUB, before the long-term solution of having all possible nodes provide valid pgdat with zonelists:
- fix SLUB with the hunk at the end of this mail - the point is to use
NUMA_NO_NODEas fallback instead ofnode_to_mem_node()
I am not familiar with SLUB to review.
Quoted 2 lines
- this removes all uses of
node_to_mem_node(luckily it's just SLUB), kill it completely instead of trying to fix it up
Sounds like a good plan to me. The code shouldn't really care.
Signature · 2 lines
Michal Hocko SUSE Labs
Michael Ellerman
reviewerRe: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
In reply to Vlastimil BabkaQuoted 75 lines · Vlastimil Babka
Quoted 66 lines · Michal Hocko
Quoted 51 lines · Srikar Dronamraju
Calling a
kmalloc_nodeon a possible node which is not yet onlined can lead to panic. Currentlynode_present_pages()doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.NIP [
c0000000003d55f4]___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94]__slab_alloc+0x34/0x60Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for
node_spanned_pages()too.Cc Andrew Morton <[email protected]>Cc Mel Gorman <[email protected]>Cc Michael Ellerman <[email protected]>Cc Sachin Sant <[email protected]>Cc Michal Hocko <[email protected]>Cc Christopher Lameter <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Srikar Dronamraju <[email protected]>Cc Bharata B Rao <[email protected]>Cc Nathan Lynch <[email protected]>Reported-by Sachin Sant <[email protected]>Tested-by Sachin Sant <[email protected]>Signed-off-by Srikar Dronamraju <[email protected]>--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)include/linux/mmzone.h +0
-0include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 9 lines
include/linux/mmzone.h +4
-2@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t; -#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)I believe this is a wrong approach. We really do not want to special case all the places which require
NODE_DATA. Can we please go and allocate pgdat for all possible nodes?Quoted 3 lines
The current state of memory less hacks subtle bugs poping up here and there just prove that we should have done that from the very begining IMHO.
Yes. So here's an alternative proposal for fixing the current situation in SLUB, before the long-term solution of having all possible nodes provide valid pgdat with zonelists:
Quoted 7 lines
- fix SLUB with the hunk at the end of this mail - the point is to use
NUMA_NO_NODEas fallback instead ofnode_to_mem_node()
- this removes all uses ofnode_to_mem_node(luckily it's just SLUB), kill it completely instead of trying to fix it up
- patch 1/4 is not needed with the fix
- perhaps many of your other patches are alss not needed
- once we get the long-term solution, some of the !node_online() checks can be removed
Seems like a nice solution to me :)
mm/slub.c static inline struct page *alloc_slab_page(struct kmem_cache *s, · line 1514
Quoted 11 lines
----8<----
mm/slub.c +1
-1@@ -1511,7 +1511,7 @@ static inline struct page *alloc_slab_page(struct kmem_cache *s,struct page *page;unsigned int order =oo_order (oo); -if (node ==NUMA_NO_NODE ) +if (node ==NUMA_NO_NODE || !node_online (node))
Why don't we need the node_present_pages() check here?
Quoted 15 lines
mm/slub.c +0
-2page =alloc_pages (flags, order); else page =__alloc_pages_node (node, flags, order);@@ -1973,8 +1973,6 @@ static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,if (node ==NUMA_NO_NODE ) searchnode =numa_mem_id (); -else if (!node_present_pages (node)) - searchnode =node_to_mem_node (node); object =get_partial_node (s,get_node (s, searchnode), c, flags);if (object || node !=NUMA_NO_NODE )@@ -2568,12 +2566,15 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, redo:
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2571
Quoted 10 lines
mm/slub.c +3
-6if (unlikely (!node_match (page, node))) { -int searchnode = node; - -if (node !=NUMA_NO_NODE && !node_present_pages (node)) - searchnode =node_to_mem_node (node); - -if (unlikely (!node_match (page, searchnode))) { +/* +* node_match() false implies node != NUMA_NO_NODE +* but if the node is not online and has no pages, just
^
this should be 'or' ?
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2580
Quoted 9 lines
mm/slub.c +6
-0+* ignore the constraint +*/ +if ((!node_online (node) || !node_present_pages (node))) { + node =NUMA_NO_NODE ; + goto redo; + }else {stat (s,ALLOC_NODE_MISMATCH );deactivate_slab (s, page, c->freelist, c); goto new_slab;
cheers
Michael Ellerman
reviewerRe: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
Quoted 115 lines · Michael Ellerman
Quoted 75 lines · Vlastimil Babka
Quoted 66 lines · Michal Hocko
Quoted 51 lines · Srikar Dronamraju
Calling a
kmalloc_nodeon a possible node which is not yet onlined can lead to panic. Currentlynode_present_pages()doesn't verify the node is online before accessing the pgdat for the node. However pgdat struct may not be available resulting in a crash.NIP [
c0000000003d55f4]___slab_alloc+0x1f4/0x760
LR [c0000000003d5b94]__slab_alloc+0x34/0x60Call Trace: [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable) [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60 [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490 [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110 [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270 [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0 [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0 [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230 [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0 [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68Fix this by verifying the node is online before accessing the pgdat structure. Fix the same for
node_spanned_pages()too.Cc Andrew Morton <[email protected]>Cc Mel Gorman <[email protected]>Cc Michael Ellerman <[email protected]>Cc Sachin Sant <[email protected]>Cc Michal Hocko <[email protected]>Cc Christopher Lameter <[email protected]>Cc Joonsoo Kim <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Srikar Dronamraju <[email protected]>Cc Bharata B Rao <[email protected]>Cc Nathan Lynch <[email protected]>Reported-by Sachin Sant <[email protected]>Tested-by Sachin Sant <[email protected]>Signed-off-by Srikar Dronamraju <[email protected]>--- include/linux/mmzone.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)include/linux/mmzone.h +0
-0include/linux/mmzone.h typedef struct pglist_data { · line 762
Quoted 9 lines
include/linux/mmzone.h +4
-2@@ -756,8 +756,10 @@ typedef struct pglist_data {atomic_long_t vm_stat[NR_VM_NODE_STAT_ITEMS ]; } pg_data_t; -#define node_present_pages (nid) (NODE_DATA(nid)->node_present_pages) -#define node_spanned_pages (nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_present_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_present_pages : 0) +#define node_spanned_pages (nid) \ + (node_online(nid) ? NODE_DATA(nid)->node_spanned_pages : 0)I believe this is a wrong approach. We really do not want to special case all the places which require
NODE_DATA. Can we please go and allocate pgdat for all possible nodes?Quoted 3 lines
The current state of memory less hacks subtle bugs poping up here and there just prove that we should have done that from the very begining IMHO.
Yes. So here's an alternative proposal for fixing the current situation in SLUB, before the long-term solution of having all possible nodes provide valid pgdat with zonelists:
Quoted 7 lines
- fix SLUB with the hunk at the end of this mail - the point is to use
NUMA_NO_NODEas fallback instead ofnode_to_mem_node()
- this removes all uses ofnode_to_mem_node(luckily it's just SLUB), kill it completely instead of trying to fix it up
- patch 1/4 is not needed with the fix
- perhaps many of your other patches are alss not needed
- once we get the long-term solution, some of the !node_online() checks can be removedSeems like a nice solution to me :)
mm/slub.c static inline struct page *alloc_slab_page(struct kmem_cache *s, · line 1514
Quoted 11 lines
----8<----
mm/slub.c +1
-1@@ -1511,7 +1511,7 @@ static inline struct page *alloc_slab_page(struct kmem_cache *s,struct page *page;unsigned int order =oo_order (oo); -if (node ==NUMA_NO_NODE ) +if (node ==NUMA_NO_NODE || !node_online (node))Why don't we need the
node_present_pages()check here?Quoted 15 lines
mm/slub.c +0
-2page =alloc_pages (flags, order); else page =__alloc_pages_node (node, flags, order);@@ -1973,8 +1973,6 @@ static void *get_partial(struct kmem_cache *s, gfp_t flags, int node,if (node ==NUMA_NO_NODE ) searchnode =numa_mem_id (); -else if (!node_present_pages (node)) - searchnode =node_to_mem_node (node); object =get_partial_node (s,get_node (s, searchnode), c, flags);if (object || node !=NUMA_NO_NODE )@@ -2568,12 +2566,15 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, redo:
Quoted 12 lines
mm/slub.c static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 2571
Quoted 10 lines
mm/slub.c +3
-6if (unlikely (!node_match (page, node))) { -int searchnode = node; - -if (node !=NUMA_NO_NODE && !node_present_pages (node)) - searchnode =node_to_mem_node (node); - -if (unlikely (!node_match (page, searchnode))) { +/* +* node_match() false implies node != NUMA_NO_NODE +* but if the node is not online and has no pages, just ^
this should be 'or' ?
Sorry I see you've already fixed this in the version you posted.
cheers
Re: [PATCH v2 1/4] mm: Check for node_online in node_present_pages
In reply to Michael EllermanQuoted 1 line · Michael Ellerman
Seems like a nice solution to me
Thanks :)
Quoted 9 lines
Quoted 9 lines
----8<----
mm/slub.c +0
-0@@ -1511,7 +1511,7 @@ static inline struct page *alloc_slab_page(struct kmem_cache *s,struct page *page;unsigned int order =oo_order (oo);
Quoted 4 lines
mm/slub.c static inline struct page *alloc_slab_page(struct kmem_cache *s, · line 1514
Quoted 2 lines
mm/slub.c +1
-1-if (node ==NUMA_NO_NODE ) +if (node ==NUMA_NO_NODE || !node_online (node))Why don't we need the
node_present_pages()check here?
Page allocator is fine with a node without present pages, as long as there's a zonelist, which online nodes must have (ideally all possible nodes should have, and then we can remove this).
SLUB on the other hand doesn't allocate cache per-cpu structures for nodes without present pages (understandably) that's why the other place includes the node_present_pages() check.
Thanks
PATCH v2 2/4 mm/slub: Use mem_node to allocate a new slab
In reply to Srikar DronamrajuCurrently while allocating a slab for a offline node, we use its associated node_numa_mem to search for a partial slab. If we don't find a partial slab, we try allocating a slab from the offline node using __alloc_pages_node. However this is bound to fail.
NIP [c00000000039a300] __alloc_pages_nodemask+0x130/0x3b0 LR [c00000000039a3c4] __alloc_pages_nodemask+0x1f4/0x3b0
Call Trace: [c0000008b36837f0] [c00000000039a3b4] __alloc_pages_nodemask+0x1e4/0x3b0 (unreliable) [c0000008b3683870] [c0000000003d1ff8] new_slab+0x128/0xcf0 [c0000008b3683950] [c0000000003d6060] ___slab_alloc+0x410/0x820 [c0000008b3683a40] [c0000000003d64a4] __slab_alloc+0x34/0x60 [c0000008b3683a70] [c0000000003d78b0] __kmalloc_node+0x110/0x490 [c0000008b3683af0] [c000000000343a08] kvmalloc_node+0x58/0x110 [c0000008b3683b30] [c0000000003ffd44] mem_cgroup_css_online+0x104/0x270 [c0000008b3683b90] [c000000000234e08] online_css+0x48/0xd0 [c0000008b3683bc0] [c00000000023dedc] cgroup_apply_control_enable+0x2ec/0x4d0 [c0000008b3683ca0] [c0000000002416f8] cgroup_mkdir+0x228/0x5f0 [c0000008b3683d10] [c000000000520360] kernfs_iop_mkdir+0x90/0xf0 [c0000008b3683d50] [c00000000043e400] vfs_mkdir+0x110/0x230 [c0000008b3683da0] [c000000000441ee0] do_mkdirat+0xb0/0x1a0 [c0000008b3683e20] [c00000000000b278] system_call+0x5c/0x68
Mitigate this by allocating the new slab from the node_numa_mem.
---
Changelog v1 -> v2:
- Handled comments from Vlastimil Babka
- Now node gets set to node_numa_mem in new_slab_objects.
mm/slub.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
mm/slub.c +5 -4
@@ -2475,6 +2475,9 @@ static inline void *new_slab_objects(struct kmem_cache *s, gfp_t flags,if (freelist)return freelist; +if (node !=NUMA_NO_NODE && !node_present_pages (node)) + node =node_to_mem_node (node); + page =new_slab (s, flags, node);if (page) { c =raw_cpu_ptr (s->cpu_slab);
@@ -2569,12 +2572,10 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, redo:if (unlikely (!node_match (page, node))) { -int searchnode = node; -if (node !=NUMA_NO_NODE && !node_present_pages (node)) - searchnode =node_to_mem_node (node); + node =node_to_mem_node (node); -if (unlikely (!node_match (page, searchnode))) { +if (unlikely (!node_match (page, node))) {stat (s,ALLOC_NODE_MISMATCH );deactivate_slab (s, page, c->freelist, c); goto new_slab;
Signature · 1 line
2.18.1
PATCH v2 3/4 mm: Implement reset_numa_mem
In reply to Srikar DronamrajuFor a memoryless or offline nodes, node_numa_mem refers to a N_MEMORY fallback node. Currently kernel has an API set_numa_mem that sets node_numa_mem for memoryless node. However this API cannot be used for offline nodes. Hence all offline nodes will have their node_numa_mem set to 0. However systems can themselves have node 0 as offline i.e memoryless and cpuless at this time. In such cases, node_to_mem_node() fails to provide a N_MEMORY fallback node.
Mitigate this by having a new API that sets the default node_numa_mem for offline nodes to be first_memory_node.
--- include/asm-generic/topology.h | 3 +++ include/linux/topology.h | 7 +++++++ 2 files changed, 10 insertions(+)
include/asm-generic/topology.h +3 -0
@@ -68,6 +68,9 @@#ifndef set_numa_mem#define set_numa_mem (node)#endif +#ifndef reset_numa_mem +#define reset_numa_mem (node) +#endif #ifndef set_cpu_numa_mem#define set_cpu_numa_mem (cpu, node)#endif
include/linux/topology.h +7 -0
@@ -147,6 +147,13 @@ static inline int node_to_mem_node(int node) }#endif +#ifndef reset_numa_mem +static inline void reset_numa_mem (int node) +{ + _node_numa_mem_[node] = first_memory_node; +} +#endif +#ifndef numa_mem_id/* Returns the number of the nearest Node with memory */ static inline int numa_mem_id (void )
Signature · 1 line
2.18.1
Re: [PATCH v2 3/4] mm: Implement reset_numa_mem
Quoted 5 lines · Srikar Dronamraju
For a memoryless or offline nodes,
node_numa_memrefers to aN_MEMORYfallback node. Currently kernel has an APIset_numa_memthat setsnode_numa_memfor memoryless node. However this API cannot be used for offline nodes. Hence all offline nodes will have theirnode_numa_memset to 0. However systems can themselves have node 0 as offline i.e
That is a significant change to the basic assumptions for memory less nodes. Node 0 needed to have memory and processors. Not sure what else may break.
Michal Hocko
reviewerRe: [PATCH v2 3/4] mm: Implement reset_numa_mem
Quoted 11 lines · Cristopher Lameter
Quoted 5 lines · Srikar Dronamraju
For a memoryless or offline nodes,
node_numa_memrefers to aN_MEMORYfallback node. Currently kernel has an APIset_numa_memthat setsnode_numa_memfor memoryless node. However this API cannot be used for offline nodes. Hence all offline nodes will have theirnode_numa_memset to 0. However systems can themselves have node 0 as offline i.eThat is a significant change to the basic assumptions for memory less nodes. Node 0 needed to have memory and processors. Not sure what else may break.
This assumption is simply incorrect. There many examples but just one from top of my head 3e8589963773 ("memcg: make it work on sparse non-0-node systems"). We simply have to forget that some nodes are special.
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v2 4/4 powerpc/numa: Set fallback nodes for offline nodes
In reply to Srikar DronamrajuCurrently fallback nodes for offline nodes aren't set. Hence by default node 0 ends up being the default node. However node 0 might be offline.
Fix this by explicitly setting fallback node. Ensure first_memory_node is set before kernel does explicit setting of fallback node.
---
Changelog v1 -> v2:
- Handled comments from Bharata B Rao
- Dont use dump_numa_cpu_topology to set fallback nodes
arch/powerpc/include/asm/topology.h | 16 ++++++++++++++++ arch/powerpc/kernel/smp.c | 1 + 2 files changed, 17 insertions(+)
arch/powerpc/include/asm/topology.h +16 -0
@@ -62,6 +62,21 @@ static inline int early_cpu_to_node(int cpu)*/ return (nid < 0) ? 0 : nid; } + +static inline int update_default_numa_mem (void ) +{ +unsigned int node; + +for_each_node (node) { +/* +* For all possible but not yet online nodes, ensure their +* node_numa_mem is set correctly so that kmalloc_node works +* for such nodes. +*/ +if (!node_online (node)) +reset_numa_mem (node); + } +}#else static inline int early_cpu_to_node (int cpu) {return 0 ; }
@@ -90,6 +105,7 @@ static inline int cpu_distance(__be32 *cpu1_assoc, __be32 *cpu2_assoc) return 0; } +static inline int update_default_numa_mem (void ) {}#endif /* CONFIG_NUMA */ #if defined(CONFIG_NUMA ) && defined(CONFIG_PPC_SPLPAR )
arch/powerpc/kernel/smp.c +1 -0
@@ -1383,6 +1383,7 @@ void __init smp_cpus_done(unsigned int max_cpus)if (smp_ops && smp_ops->bringup_done) smp_ops->bringup_done (); +update_default_numa_mem ();dump_numa_cpu_topology ();#ifdef CONFIG_SCHED_SMT
Signature · 1 line
2.18.1
kbuild test robot
build botRe: [PATCH v2 4/4] powerpc/numa: Set fallback nodes for offline nodes
Thank you for the patch! Perhaps something to improve:
Hi Srikar,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on powerpc/next]
[also build test WARNING on next-20200317]
[cannot apply to linus/master asm-generic/master mpe/next v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Srikar-Dronamraju/Fix-kmalloc_node-on-offline-nodes/20200318-180303 base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next config: powerpc-motionpro_defconfig (attached as .config) compiler: powerpc-linux-gcc (GCC) 9.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=9.2.0 make.cross ARCH=powerpc
If you fix the issue, kindly add following tag
All warnings (new ones prefixed by >>):
In file included from include/linux/topology.h:36,
from include/linux/gfp.h:9,
from include/linux/mm.h:10,
from include/linux/bvec.h:13,
from include/linux/blk_types.h:10,
from include/linux/iomap.h:7,
from fs/iomap/trace.c:5:
arch/powerpc/include/asm/topology.h: In function 'update_default_numa_mem':
>> arch/powerpc/include/asm/topology.h:108:1: warning: no return statement in function returning non-void [-Wreturn-type]
108 | static inline int update_default_numa_mem(void) {}
| ^~~~~~--
In file included from include/linux/topology.h:36,
from include/linux/gfp.h:9,
from include/linux/slab.h:15,
from include/linux/crypto.h:19,
from include/crypto/hash.h:11,
from include/linux/uio.h:10,
from include/linux/socket.h:8,
from include/linux/compat.h:15,
from arch/powerpc/kernel/asm-offsets.c:14:
arch/powerpc/include/asm/topology.h: In function 'update_default_numa_mem':
>> arch/powerpc/include/asm/topology.h:108:1: warning: no return statement in function returning non-void [-Wreturn-type]
108 | static inline int update_default_numa_mem(void) {}
| ^~~~~~29 real 8 user 16 sys 85.32% cpu make prepare
vim +108 arch/powerpc/include/asm/topology.h
107
> 108 static inline int update_default_numa_mem(void) {} 109 #endif /* CONFIG_NUMA */
110
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kbuild test robot
build botRe: [PATCH v2 4/4] powerpc/numa: Set fallback nodes for offline nodes
In reply to Srikar DronamrajuThank you for the patch! Yet something to improve:
Hi Srikar,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on powerpc/next]
[also build test ERROR on next-20200317]
[cannot apply to linus/master asm-generic/master mpe/next v5.6-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Srikar-Dronamraju/Fix-kmalloc_node-on-offline-nodes/20200318-180303 base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next config: powerpc-defconfig (attached as .config) compiler: powerpc64-linux-gcc (GCC) 9.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=9.2.0 make.cross ARCH=powerpc
If you fix the issue, kindly add following tag
All error/warnings (new ones prefixed by >>):
In file included from include/linux/topology.h:36,
from include/linux/gfp.h:9,
from include/linux/slab.h:15,
from include/linux/crypto.h:19,
from include/crypto/hash.h:11,
from include/linux/uio.h:10,
from include/linux/socket.h:8,
from include/linux/compat.h:15,
from arch/powerpc/kernel/asm-offsets.c:14:
arch/powerpc/include/asm/topology.h: In function 'update_default_numa_mem':
>> arch/powerpc/include/asm/topology.h:77:4: error: implicit declaration of function 'reset_numa_mem' [-Werror=implicit-function-declaration]
77 | reset_numa_mem(node);
| ^~~~~~~~~~~~~~
arch/powerpc/include/asm/topology.h:79:1: warning: no return statement in function returning non-void [-Wreturn-type]
79 | }
| ^
In file included from include/linux/gfp.h:9,
from include/linux/slab.h:15,
from include/linux/crypto.h:19,
from include/crypto/hash.h:11,
from include/linux/uio.h:10,
from include/linux/socket.h:8,
from include/linux/compat.h:15,
from arch/powerpc/kernel/asm-offsets.c:14:
include/linux/topology.h: At top level:
>> include/linux/topology.h:151:20: warning: conflicting types for 'reset_numa_mem'
151 | static inline void reset_numa_mem(int node)
| ^~~~~~~~~~~~~~
>> include/linux/topology.h:151:20: error: static declaration of 'reset_numa_mem' follows non-static declaration
In file included from include/linux/topology.h:36,
from include/linux/gfp.h:9,
from include/linux/slab.h:15,
from include/linux/crypto.h:19,
from include/crypto/hash.h:11,
from include/linux/uio.h:10,
from include/linux/socket.h:8,
from include/linux/compat.h:15,
from arch/powerpc/kernel/asm-offsets.c:14:
arch/powerpc/include/asm/topology.h:77:4: note: previous implicit declaration of 'reset_numa_mem' was here
77 | reset_numa_mem(node);
| ^~~~~~~~~~~~~~ cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:101: arch/powerpc/kernel/asm-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1112: prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:179: sub-make] Error 2
256 real 58 user 115 sys 67.93% cpu make prepare
vim +/reset_numa_mem +77 arch/powerpc/include/asm/topology.h
65
66 static inline int update_default_numa_mem(void)
67 {
68 unsigned int node;
69
70 for_each_node(node) {
71 /*
72 * For all possible but not yet online nodes, ensure their
73 * node_numa_mem is set correctly so that kmalloc_node works
74 * for such nodes.
75 */
76 if (!node_online(node))
> 77 reset_numa_mem(node);
78 }
79 }
80 #else
81 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org