mm,numa: boot cpu should bound to the node0 when node_off enable
PATCH 1/1 mm,numa: boot cpu should bound to the node0 when node_off enable
At present, boot cpu will bound to a node from device tree when node_off enable. if the node is not initialization, it will lead to a following problem.
next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/mm/numa.c +1 -1
@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off) nid =0 ; cpu_to_node_map[cpu] = nid;
Signature · 1 line
1.7.12.4
Catalin Marinas
maintainerRe: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414
I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 13 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/mm/numa.c +0
-0
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 8 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off) nid =0 ; cpu_to_node_map[cpu] = nid;
The patch looks fine (slight inconsistence from the map_cpu_to_node() callers but I guess we don't want to expose numa_off outside this file). I would however like to see an Ack from Ganapat (cc'ed).
Signature · 1 line
Catalin
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 21 lines · Catalin Marinas
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)
i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when numa_off is set the cpu_to_node_map[cpu] is not used and set to node0 always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
void numa_store_cpu_info(unsigned int cpu)
{
map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
}thanks
Ganapat
Quoted 15 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 6 lines · Ganapatrao Kulkarni
Quoted 4 lines · Catalin Marinas
Quoted 3 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
yes, it will leads to kernel crash. the details is as follows.
Unable to handle kernel paging request at virtual address 00001690 pgd = ffff800001226000 [00001690] *pgd=0000000000000000 Internal error: Oops: 96000004 [#1] SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.27-vhulk3.6.5.aarch64 #1 Hardware name: Hisilicon Hi1612 Development Board (DT) task: ffff80000102b730 ti: ffff800001018000 task.ti: ffff800001018000 PC is at next_zones_zonelist+0x18/0x80 LR is at __build_all_zonelists+0x1e0/0x288 next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414
Quoted 50 lines
Quoted 8 lines
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }thanks
GanapatQuoted 7 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel.
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
In reply to Ganapatrao KulkarniQuoted 42 lines · Ganapatrao Kulkarni
Quoted 20 lines · Catalin Marinas
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu)
Quoted 6 lines
{ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }thanks
Ganapat
The issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when numa_off enable.
__build_all_zonelists will call the cpu_to_node[cpu], but orresponding relation will be obtained from the devicetree. therefore, the issue will come up. The corresponding message is as follows when kdump start. it is obvious that mem range points to the node1 in the devicetree.
Early memory node ranges
node 0: [mem 0x0000005fe0000000-0x0000005fffffffff]
Initmem setup node 0 [mem 0x0000005fe0000000-0x0000005fffffffff]
Unable to handle kernel paging request at virtual address 00001690 pgd = ffff800001226000 [00001690] *pgd=0000000000000000 Internal error: Oops: 96000004 [#1] SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.27-vhulk3.6.5.aarch64 #1 Hardware name: Hisilicon Hi1612 Development Board (DT) task: ffff80000102b730 ti: ffff800001018000 task.ti: ffff800001018000 PC is at next_zones_zonelist+0x18/0x80 LR is at __build_all_zonelists+0x1e0/0x288 next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414
Quoted 16 lines
Quoted 7 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel.
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 47 lines · zhong jiang
Quoted 46 lines · Ganapatrao Kulkarni
Quoted 20 lines · Catalin Marinas
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }
Quoted 5 lines
Quoted 2 lines
thanks
GanapatThe issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.
boot cpu is default mapped to node0
are you running with any other patches?
Quoted 3 lines
__build_all_zonelistswill call the cpu_to_node[cpu], but orresponding relation will be obtained from the devicetree. therefore, the issue will come up.
when numa_off, all cpus are mapped to node0( refer numa_store_cpu_info) and device tree mapping is ignored.
Quoted 40 lines
The corresponding message is as follows when kdump start. it is obvious that mem range points to the node1 in the devicetree.
Early memory node ranges
node 0: [mem 0x0000005fe0000000-0x0000005fffffffff]
Initmem setup node 0 [mem 0x0000005fe0000000-0x0000005fffffffff]Unable to handle kernel paging request at virtual address 00001690 pgd = ffff800001226000 [00001690] *pgd=0000000000000000 Internal error: Oops: 96000004 [#1] SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.27-vhulk3.6.5.aarch64 #1 Hardware name: Hisilicon Hi1612 Development Board (DT) task: ffff80000102b730 ti: ffff800001018000 task.ti: ffff800001018000 PC is at next_zones_zonelist+0x18/0x80 LR is at __build_all_zonelists+0x1e0/0x288 next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414Quoted 16 lines
Quoted 7 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel.
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 48 lines · Ganapatrao Kulkarni
Quoted 47 lines · zhong jiang
Quoted 46 lines · Ganapatrao Kulkarni
Quoted 20 lines · Catalin Marinas
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
thanks
GanapatThe issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.boot cpu is default mapped to node0
are you running with any other patches?
if you added any patch to change this code
/* init boot processor */
cpu_to_node_map[0] = 0;
map_cpu_to_node(0, 0);then adding code to take-care numa_off here might solve your issue.
Quoted 46 lines
Quoted 3 lines
__build_all_zonelistswill call the cpu_to_node[cpu], but orresponding relation will be obtained from the devicetree. therefore, the issue will come up.when
numa_off, all cpus are mapped tonode0( refer numa_store_cpu_info)and device tree mapping is ignored.Quoted 40 lines
The corresponding message is as follows when kdump start. it is obvious that mem range points to the node1 in the devicetree.
Early memory node ranges
node 0: [mem 0x0000005fe0000000-0x0000005fffffffff]
Initmem setup node 0 [mem 0x0000005fe0000000-0x0000005fffffffff]Unable to handle kernel paging request at virtual address 00001690 pgd = ffff800001226000 [00001690] *pgd=0000000000000000 Internal error: Oops: 96000004 [#1] SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.27-vhulk3.6.5.aarch64 #1 Hardware name: Hisilicon Hi1612 Development Board (DT) task: ffff80000102b730 ti: ffff800001018000 task.ti: ffff800001018000 PC is at next_zones_zonelist+0x18/0x80 LR is at __build_all_zonelists+0x1e0/0x288 next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414Quoted 16 lines
Quoted 7 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel.
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 55 lines · Ganapatrao Kulkarni
Quoted 53 lines · Ganapatrao Kulkarni
Quoted 52 lines · zhong jiang
Quoted 42 lines · Ganapatrao Kulkarni
Quoted 20 lines · Catalin Marinas
Quoted 8 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414I think this "problem" is missing a lot of information. Is this supposed to be a kernel panic?
Quoted 8 lines
The patch fix it by fallback to node 0. therefore, the cpu will bound to the node correctly.
Signed-off-by zhongjiang <[email protected]>--- arch/arm64/mm/numa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 10 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 10 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu)Quoted 6 lines
{ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }thanks
GanapatThe issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.
Quoted 8 lines
Quoted 2 lines
boot cpu is default mapped to node0
are you running with any other patches?if you added any patch to change this code
/* init boot processor */cpu_to_node_map[0] = 0; map_cpu_to_node(0, 0);then adding code to take-care
numa_offhere might solve your issue.
but in of_smp_init_cpus, boot cpu will call early_map_cpu_to_node[] to get the relation node. and the node is from devicetree.
you points to the code will be covered with another node. therefore, it is possible that cpu_to_node[cpu] will leads to the incorrect results. therefore, The crash will come up.
Quoted 45 lines
Quoted 43 lines
Quoted 2 lines
__build_all_zonelistswill call the cpu_to_node[cpu], but orresponding relation will be obtained from the devicetree. therefore, the issue will come up.when
numa_off, all cpus are mapped tonode0( refer numa_store_cpu_info)and device tree mapping is ignored.Quoted 39 lines
The corresponding message is as follows when kdump start. it is obvious that mem range points to the node1 in the devicetree.
Early memory node ranges
node 0: [mem 0x0000005fe0000000-0x0000005fffffffff]
Initmem setup node 0 [mem 0x0000005fe0000000-0x0000005fffffffff]Unable to handle kernel paging request at virtual address 00001690 pgd = ffff800001226000 [00001690] *pgd=0000000000000000 Internal error: Oops: 96000004 [#1] SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.27-vhulk3.6.5.aarch64 #1 Hardware name: Hisilicon Hi1612 Development Board (DT) task: ffff80000102b730 ti: ffff800001018000 task.ti: ffff800001018000 PC is at next_zones_zonelist+0x18/0x80 LR is at __build_all_zonelists+0x1e0/0x288 next_zones_zonelist+0x18/0x80 __build_all_zonelists+0x1e0/0x288 build_all_zonelists_init+0x10/0x1c build_all_zonelists+0x114/0x128 start_kernel+0x1a0/0x414Quoted 16 lines
Quoted 7 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 125
Quoted 3 lines
arch/arm64/mm/numa.c +0
-0nid =0 ; cpu_to_node_map[cpu] = nid;The patch looks fine (slight inconsistence from the
map_cpu_to_node()callers but I guess we don't want to exposenuma_offoutside this file). I would however like to see an Ack from Ganapat (cc'ed).Quoted 7 lines
--
Catalin_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel.
.
Catalin Marinas
maintainerRe: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 10 lines · zhong jiang
Quoted 9 lines · Ganapatrao Kulkarni
Quoted 7 lines · Ganapatrao Kulkarni
Quoted 6 lines · zhong jiang
Quoted 5 lines · Ganapatrao Kulkarni
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.
[...]
Quoted 35 lines
Quoted 35 lines
Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 8 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 8 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }Quoted 6 lines
Quoted 3 lines
The issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.boot cpu is default mapped to node0
are you running with any other patches?if you added any patch to change this code
/* init boot processor */cpu_to_node_map[0] = 0; map_cpu_to_node(0, 0);
Quoted 8 lines
Quoted 1 line
then adding code to take-care
numa_offhere might solve your issue.but in
of_smp_init_cpus, boot cpu will call early_map_cpu_to_node[] to get the relation node. and the node is from devicetree.you points to the code will be covered with another node. therefore, it is possible that cpu_to_node[cpu] will leads to the incorrect results. therefore, The crash will come up.
I think I get Ganapat's point. The cpu_to_node_map[0] may be incorrectly set by early_map_cpu_to_node() when called from smp_init_cpus() -> of_parse_and_init_cpus(). However, the cpu_to_node_map[] array is *only* read by numa_store_cpu_info(). This latter function calls map_cpu_to_node() and, if numa_off, will only ever pass 0 as the nid.
Given that the cpu_to_node_map[] array is static, I don't see how any non-zero value could leak outside the arch/arm64/mm/numa.c file.
So please give more details of any additional patches you have on top of mainline or whether you reproduced this issue with the vanilla kernel (since you mentioned kdump, that's not in mainline yet).
Signature · 1 line
Catalin
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 56 lines · Catalin Marinas
Quoted 10 lines · zhong jiang
Quoted 9 lines · Ganapatrao Kulkarni
Quoted 7 lines · Ganapatrao Kulkarni
Quoted 6 lines · zhong jiang
Quoted 5 lines · Ganapatrao Kulkarni
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.[...]
Quoted 31 lines
Quoted 31 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 8 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 8 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }Quoted 5 lines
Quoted 3 lines
The issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.boot cpu is default mapped to node0
are you running with any other patches?if you added any patch to change this code
/* init boot processor */cpu_to_node_map[0] = 0; map_cpu_to_node(0, 0);Quoted 7 lines
Quoted 1 line
then adding code to take-care
numa_offhere might solve your issue.but in
of_smp_init_cpus, boot cpu will call early_map_cpu_to_node[] to get the relation node. and the node is from devicetree.you points to the code will be covered with another node. therefore, it is possible that cpu_to_node[cpu] will leads to the incorrect results. therefore, The crash will come up.
I think I get Ganapat's point. The cpu_to_node_map[0] may be incorrectly set by
early_map_cpu_to_node()when called fromsmp_init_cpus()->of_parse_and_init_cpus(). However, the cpu_to_node_map[] array is *only* read bynuma_store_cpu_info(). This latter function callsmap_cpu_to_node()and, ifnuma_off, will only ever pass 0 as the nid.
Quoted 7 lines
Given that the cpu_to_node_map[] array is static, I don't see how any non-zero value could leak outside the
arch/arm64/mm/numa.cfile.So please give more details of any additional patches you have on top of mainline or whether you reproduced this issue with the vanilla kernel (since you mentioned kdump, that's not in mainline yet).
Thanks for Catalin and Ganapatral.
I am sorry for that. The mainline have solved. The mainline changes is too much, I did not notice.
PATCH 1/1 mm, numa: boot cpu should bound to the node0 when node_off enable
In reply to Catalin MarinasQuoted 35 lines · Catalin Marinas
Quoted 10 lines · zhong jiang
Quoted 9 lines · Ganapatrao Kulkarni
Quoted 7 lines · Ganapatrao Kulkarni
Quoted 6 lines · zhong jiang
Quoted 5 lines · Ganapatrao Kulkarni
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.[...]
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 8 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 8 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
The issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.boot cpu is default mapped to node0
are you running with any other patches?
He applied my patches, which I mentioned these days. I will pay attention to it.
Quoted 28 lines
Quoted 6 lines
Quoted 6 lines
if you added any patch to change this code
/* init boot processor */cpu_to_node_map[0] = 0; map_cpu_to_node(0, 0);Quoted 8 lines
Quoted 1 line
then adding code to take-care
numa_offhere might solve your issue.but in
of_smp_init_cpus, boot cpu will call early_map_cpu_to_node[] to get the relation node. and the node is from devicetree.you points to the code will be covered with another node. therefore, it is possible that cpu_to_node[cpu] will leads to the incorrect results. therefore, The crash will come up.
I think I get Ganapat's point. The cpu_to_node_map[0] may be incorrectly set by
early_map_cpu_to_node()when called fromsmp_init_cpus()->of_parse_and_init_cpus(). However, the cpu_to_node_map[] array is *only* read bynuma_store_cpu_info(). This latter function callsmap_cpu_to_node()and, ifnuma_off, will only ever pass 0 as the nid.Given that the cpu_to_node_map[] array is static, I don't see how any non-zero value could leak outside the
arch/arm64/mm/numa.cfile.So please give more details of any additional patches you have on top of mainline or whether you reproduced this issue with the vanilla kernel (since you mentioned kdump, that's not in mainline yet).
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
In reply to Catalin MarinasQuoted 35 lines · Catalin Marinas
Quoted 10 lines · zhong jiang
Quoted 9 lines · Ganapatrao Kulkarni
Quoted 7 lines · Ganapatrao Kulkarni
Quoted 6 lines · zhong jiang
Quoted 5 lines · Ganapatrao Kulkarni
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · zhongjiang
At present, boot cpu will bound to a node from device tree when
node_offenable. if the node is not initialization, it will lead to a following problem.[...]
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 22 lines
Quoted 8 lines
arch/arm64/mm/numa.c void numa_store_cpu_info(unsigned int cpu) · line 122
Quoted 8 lines
arch/arm64/mm/numa.c +1
-1@@ -119,7 +119,7 @@ void numa_store_cpu_info(unsigned int cpu)void __init early_map_cpu_to_node (unsigned int cpu,int nid) {/* fallback to node 0 */ -if (nid <0 || nid >=MAX_NUMNODES ) +if (nid <0 || nid >=MAX_NUMNODES || numa_off)i did not understood how this line change fixes the issue that you have mentioned (i too not understood fully the issue description) this array used while mapping node id when secondary cores comes up when
numa_offis set the cpu_to_node_map[cpu] is not used and set to node0always( refer function numa_store_cpu_info).. please provide more details to understand the issue you are facing. /*
* Set the cpu to node and mem mapping
*/
voidnuma_store_cpu_info(unsigned int cpu){ map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]); }
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
The issue comes up when we test the kdump. it will leads to kernel crash. when I debug the issue, I find boot cpu actually bound to the node1. while node1 is not real existence when
numa_offenable.boot cpu is default mapped to node0
are you running with any other patches?
He applied my patches, which I mentioned these days.
I chated with ZhongJiang, this problem is only exist for my patches, and no matter whether use kdump or not. Mainline doesn't have this problem.
The details of this problem is(suppose numa_off is true), according to the code execution sequence :
1. setup_arch-->bootmem_init-->arm64_numa_init
When numa_off is true, all memory blocks will add into node 0.
2. setup_arch-->of_smp_init_cpus
I added early_map_cpu_to_node for boot cpu, so that the nid of cpu0 will change to the value read from dt node. With ZhongJiang's patch, it will correct the nid of cpu0 to zero when numa_off is true.
3. build_all_zonelists
Because numa is off, so that only the control block of node 0 had been initialized. So cpu0 with non-zero nid will lead the kernel crash.
4. kernel_init_freeable-->smp_prepare_cpus-->smp_store_cpu_info Set the nid of cpu0 to zero, but it's too late.
5. secondary_start_kernel-->smp_store_cpu_info
Set the nid of other cpus to zero.
I will update my patch series and resend it again.
Best regards,
Town?Thunder
(My Chinese name Zhen Lei direct translation into English)
Quoted 28 lines
Quoted 6 lines
Quoted 6 lines
if you added any patch to change this code
/* init boot processor */cpu_to_node_map[0] = 0; map_cpu_to_node(0, 0);Quoted 8 lines
Quoted 1 line
then adding code to take-care
numa_offhere might solve your issue.but in
of_smp_init_cpus, boot cpu will call early_map_cpu_to_node[] to get the relation node. and the node is from devicetree.you points to the code will be covered with another node. therefore, it is possible that cpu_to_node[cpu] will leads to the incorrect results. therefore, The crash will come up.
I think I get Ganapat's point. The cpu_to_node_map[0] may be incorrectly set by
early_map_cpu_to_node()when called fromsmp_init_cpus()->of_parse_and_init_cpus(). However, the cpu_to_node_map[] array is *only* read bynuma_store_cpu_info(). This latter function callsmap_cpu_to_node()and, ifnuma_off, will only ever pass 0 as the nid.Given that the cpu_to_node_map[] array is static, I don't see how any non-zero value could leak outside the
arch/arm64/mm/numa.cfile.So please give more details of any additional patches you have on top of mainline or whether you reproduced this issue with the vanilla kernel (since you mentioned kdump, that's not in mainline yet).
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 1 line · Leizhen (ThunderTown)
He applied my patches, which I mentioned these days.
[...]
Quoted 1 line
I will update my patch series and resend it again.
To be clear, you plan to send an updated version of:
[PATCH v6 00/14] fix some type infos and bugs for arm64/of numa
so I can ignore v6 of that?
Will
Re: [PATCH] mm, numa: boot cpu should bound to the node0 when node_off enable
Quoted 8 lines · Will Deacon
Quoted 1 line · Leizhen (ThunderTown)
He applied my patches, which I mentioned these days.
[...]
Quoted 1 line
I will update my patch series and resend it again.
To be clear, you plan to send an updated version of:
Yes, but just merge Zhongjiang's patch into mine, only one or two lines changed.
Quoted 4 lines
[PATCH v6 00/14] fix some type infos and bugs for arm64/of numa
so I can ignore v6 of that?
If you have not merged v6 into your branch, I think you can wait my v7. I will send v7 tomorrow.
Quoted 5 lines
Will
.