riscv: Move call to init_cpu_topology() to later initialization stage
PATCH 1/1 riscv: Move call to init_cpu_topology() to later initialization stage
topology_parse_cpu_capacity() is failed to allocate memory with kcalloc() after read "capacity-dmips-mhz" DT parameter in CPU DT nodes. This topology_parse_cpu_capacity() is called from init_cpu_topology(), move call to init_cpu_topology() to later initialization stage (after memory allocation is available).
Note, this refers to ARM64 implementation, call init_cpu_topology() in smp_prepare_cpus().
Tested on Qemu platform.
---
In drivers/base/arch_topology.c: topology_parse_cpu_capacity():
ret = of_property_read_u32(cpu_node, "capacity-dmips-mhz",
&cpu_capacity);
if (!ret) {
if (!raw_capacity) {
raw_capacity = kcalloc(num_possible_cpus(),
sizeof(*raw_capacity),
GFP_KERNEL);
if (!raw_capacity) {
cap_parsing_failed = true;
return false;
}
---
arch/riscv/kernel/smpboot.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)arch/riscv/kernel/smpboot.c +2 -1
@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }void __init smp_prepare_cpus (unsigned int max_cpus)
@@ -48,6 +47,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus)int ret;unsigned int curr_cpuid; +init_cpu_topology (); + curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid);
Signature · 1 line
2.25.1
Quoted 10 lines · Ley Foon Tan
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT nodes. Thistopology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
Hi Ley,
Can you provide the topologies (command lines) tested?
Quoted 2 lines
Signed-off-by Ley Foon Tan <[email protected]>
Fixes tag?
Quoted 44 lines
---
In
drivers/base/arch_topology.c:topology_parse_cpu_capacity():ret = of_property_read_u32(cpu_node, "capacity-dmips-mhz", &cpu_capacity); if (!ret) { if (!raw_capacity) { raw_capacity = kcalloc(num_possible_cpus(), sizeof(*raw_capacity), GFP_KERNEL); if (!raw_capacity) { cap_parsing_failed = true; return false; } --- arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/riscv/kernel/smpboot.c +2
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }void __init smp_prepare_cpus (unsigned int max_cpus)@@ -48,6 +47,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus)int ret;unsigned int curr_cpuid; +init_cpu_topology (); + curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid);Signature · 1 line
2.25.1
Otherwise,
Thanks,
drew
Quoted 21 lines
-----Original Message-----
From: Andrew Jones <[email protected]>
Sent: Tuesday, 3 January, 2023 2:54 PM
To: Leyfoon Tan <[email protected]>
Cc: Palmer Dabbelt <[email protected]>; Paul Walmsley <[email protected]>; Albert Ou <[email protected]>; linux- [email protected]; [email protected]; Ley Foon Tan <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stageQuoted 7 lines · Ley Foon Tan
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT nodes. Thistopology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).
Quoted 8 lines
Quoted 4 lines
Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
Hi Ley,
Can you provide the topologies (command lines) tested?
2 clusters with 2 CPU cores each.
Quoted 5 lines
Quoted 3 lines
Signed-off-by Ley Foon Tan <[email protected]>Fixes tag?
Okay, will send out next revision with Fixes tag.
[...]
Quoted 30 lines
Quoted 8 lines
arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/riscv/kernel/smpboot.c +0
-0arch/riscv/kernel/smpboot.c static DECLARE_COMPLETION(cpu_running); · line 43
Quoted 6 lines
arch/riscv/kernel/smpboot.c +0
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); } void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47,8@@
Quoted 10 lines
arch/riscv/kernel/smpboot.c +0
-0void __init smp_prepare_cpus (unsigned int max_cpus)arch/riscv/kernel/smpboot.c +2
-1int ret;unsigned int curr_cpuid; +init_cpu_topology (); + curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid); --Quoted 2 lines
2.25.1
Otherwise,
Quoted 4 lines
Reviewed-by Andrew Jones <[email protected]>Thanks,
drew
Thanks
Ley Foon
Hello!
Couple comments for you.
+CC Sudeep: I've got a question for you below.
Quoted 6 lines · Leyfoon Tan
Quoted 6 lines
From: Andrew Jones <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stageQuoted 2 lines · Ley Foon Tan
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT
Uhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
nodes. This
topology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
Tested on Qemu platform.
Hi Ley,
Can you provide the topologies (command lines) tested?
2 clusters with 2 CPU cores each.
What's the actual commandline for this? I'm not the best with QEMU, so it'd really be appreciated, given the above.
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Ley Foon Tan <[email protected]>Fixes tag?
Okay, will send out next revision with Fixes tag.
Please don't just send versions to add tags, Palmer can pick them up if/when he applies the patch.
Quoted 15 lines
Fixes 03f11f03dbfe ("RISC-V: Parse cpu topology during boot. ")Quoted 13 lines
Quoted 13 lines
arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/riscv/kernel/smpboot.c +0
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }
Quoted 7 lines
Quoted 7 lines
arch/riscv/kernel/smpboot.c static DECLARE_COMPLETION(cpu_running); · line 43
Quoted 1 line
arch/riscv/kernel/smpboot.c +0
-0void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47,8@@
arch/riscv/kernel/smpboot.c void __init smp_prepare_cpus(unsigned int max_cpus) · line 50
Quoted 5 lines
arch/riscv/kernel/smpboot.c +0
-0void __init smp_prepare_cpus (unsigned int max_cpus)arch/riscv/kernel/smpboot.c +1
-0int ret;unsigned int curr_cpuid; +init_cpu_topology ();
I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
Thanks,
Conor.
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
arch/riscv/kernel/smpboot.c +1
-1+ curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid); --2.25.1
Otherwise,
Reviewed-by Andrew Jones <[email protected]>
Quoted 25 lines
-----Original Message-----
From: Conor Dooley <[email protected]>
Sent: Wednesday, 4 January, 2023 1:08 AM
To: Leyfoon Tan <[email protected]>; Sudeep Holla <[email protected]>
Cc: Andrew Jones <[email protected]>; Palmer Dabbelt <[email protected]>; Paul Walmsley <[email protected]>; Albert Ou <[email protected]>; [email protected]; linux- [email protected]; Ley Foon Tan <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stageHello!
Couple comments for you.
+CC Sudeep: I've got a question for you below.
Quoted 4 lines · Leyfoon Tan
Quoted 4 lines
From: Andrew Jones <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stage On Tue, Jan 03, 2023 at 11:53:16AM +0800, Ley Foon Tan wrote:
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DTUhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.
This DT parameter is not in default Qemu. I've added it for testing (see test steps in below). This is preparation to support asymmetric CPU topology for RISC-V.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
nodes. This
topology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
Hi Ley,
Can you provide the topologies (command lines) tested?
2 clusters with 2 CPU cores each.
What's the actual commandline for this? I'm not the best with QEMU, so it'd really be appreciated, given the above.
I used the Qemu to dump the DTS for 'virt' machine from Qemu, then add the "capacity-dmips-mhz" DT parameter and modify the CPU topology for clusters.
1. Dump the dtb for 'virt' machine:
qemu-system-riscv64 -cpu rv64 -smp 4 -m 2048M -M virt,dumpdtb=qemu_virt.dtb
2. Convert dtb to dts
dtc -I dtb -O dts qemu_virt.dtb > qemu_virt.dts
3. Modifying qemu_virt.dts
4. Compile dts to dtb
dtc -I dts -O dtb qemu_virt.dts > qemu_virt.dtb
5. Pass in "-dtb qemu_virt.dtb" as one of Qemu's argument.
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Ley Foon Tan <[email protected]>Fixes tag?
Okay, will send out next revision with Fixes tag.
Please don't just send versions to add tags, Palmer can pick them up if/when he applies the patch.
Okay. Will let Palmer add tag below if he applies the patch.
Quoted 30 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/riscv/kernel/smpboot.cindex3373df413c88..ddb2afba6d25100644arch/riscv/kernel/smpboot.c +0
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }Quoted 8 lines
Quoted 8 lines
arch/riscv/kernel/smpboot.c static DECLARE_COMPLETION(cpu_running); · line 44
Quoted 2 lines
arch/riscv/kernel/smpboot.c +1
-0void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47 ,8 @@
arch/riscv/kernel/smpboot.c void __init smp_prepare_cpus(unsigned int max_cpus) · line 50
Quoted 5 lines
void
__initsmp_prepare_cpus(unsigned int max_cpus)arch/riscv/kernel/smpboot.c +1
-0int ret;unsigned int curr_cpuid; +init_cpu_topology ();I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
Quoted 14 lines
Thanks,
Conor.Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
arch/riscv/kernel/smpboot.c +1
-1+ curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid); --2.25.1
Otherwise,
Reviewed-by Andrew Jones <[email protected]>
Regards
Ley Foon
Sudeep, see below - I got mixed up & what arm64 does now makes sense to me! Your opinion on the patch, motivation aside, would be nice though.
Hey Leyfoon,
Quoted 5 lines · Leyfoon Tan
Quoted 5 lines
-----Original Message-----
From: Conor Dooley <[email protected]>
Sent: Wednesday, 4 January, 2023 1:08 AM
---8<---
Quoted 8 lines
Quoted 8 lines
To: Leyfoon Tan <[email protected]>; Sudeep Holla <[email protected]>
Cc: Andrew Jones <[email protected]>; Palmer Dabbelt <[email protected]>; Paul Walmsley <[email protected]>; Albert Ou <[email protected]>; [email protected]; linux- [email protected]; Ley Foon Tan <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stage
---8<---
btw, this above is pretty much useless can you can drop it from replies.
Quoted 15 lines
Quoted 15 lines
Hello!
Couple comments for you.
+CC Sudeep: I've got a question for you below.
Quoted 6 lines · Leyfoon Tan
Quoted 6 lines
From: Andrew Jones <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stage On Tue, Jan 03, 2023 at 11:53:16AM +0800,Quoted 2 lines · Ley Foon Tan
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT
Quoted 7 lines
Quoted 4 lines
Uhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.This DT parameter is not in default Qemu. I've added it for testing (see test steps in below). This is preparation to support asymmetric CPU topology for RISC-V.
The property is only valid on arm, so how does arm64 deal with such asymmetric topologies without it? Why should we "fix" something that may never be a valid dts?
Quoted 37 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
nodes. This
topology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
Hi Ley,
Can you provide the topologies (command lines) tested?
2 clusters with 2 CPU cores each.
What's the actual commandline for this? I'm not the best with QEMU, so it'd really be appreciated, given the above.
I used the Qemu to dump the DTS for 'virt' machine from Qemu, then add the "capacity-dmips-mhz" DT parameter and modify the CPU topology for clusters.
1. Dump the dtb for 'virt' machine:
qemu-system-riscv64 -cpu rv64 -smp 4 -m 2048M -M virt,dumpdtb=qemu_virt.dtb2. Convert dtb to dts
dtc -I dtb -O dts qemu_virt.dtb >qemu_virt.dts3. Modifying
qemu_virt.dts4. Compile dts to dtb
dtc -I dts -O dtbqemu_virt.dts> qemu_virt.dtb5. Pass in "-dtb qemu_virt.dtb" as one of Qemu's argument.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Signed-off-by Ley Foon Tan <[email protected]>
Quoted 36 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Fixes tag?
Okay, will send out next revision with Fixes tag.
Please don't just send versions to add tags, Palmer can pick them up if/when he applies the patch.
Okay. Will let Palmer add tag below if he applies the patch.
Fixes 03f11f03dbfe ("RISC-V: Parse cpu topology during boot. ")Quoted 26 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/arch/riscv/kernel/smpboot.cindex3373df413c88..ddb2afba6d25100644arch/riscv/kernel/smpboot.c +0
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }Quoted 8 lines
Quoted 8 lines
arch/riscv/kernel/smpboot.c static DECLARE_COMPLETION(cpu_running); · line 44
Quoted 2 lines
arch/riscv/kernel/smpboot.c +1
-0void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47 ,8 @@
arch/riscv/kernel/smpboot.c void __init smp_prepare_cpus(unsigned int max_cpus) · line 50
Quoted 5 lines
void
__initsmp_prepare_cpus(unsigned int max_cpus)arch/riscv/kernel/smpboot.c +1
-0int ret;unsigned int curr_cpuid; +init_cpu_topology ();I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
I got myself mixed up between places I fiddled with storing the topology, so you can ignore that question Sudeep. Clearly it's the one in smp_callin() that gets called for each CPU.
Woops.
Quoted 3 lines
Quoted 3 lines
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
...but is it still worth moving the function call later to head off any allocation failures if core topology code changes?
Quoted 15 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
arch/riscv/kernel/smpboot.c +1
-1+ curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid); --2.25.1
Otherwise,
Reviewed-by Andrew Jones <[email protected]>Regards
Ley Foon
On Wed, Jan 04, 2023 at 09:49:48AM +0000, Conor Dooley wrote:
[...]
Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
Uhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.This DT parameter is not in default Qemu. I've added it for testing (see test steps in below). This is preparation to support asymmetric CPU topology for RISC-V.
The property is only valid on arm, so how does arm64 deal with such asymmetric topologies without it?
I don't think we can deal with asymmetric topologies without this. Yes we can detect the difference in the CPU types but we can only assume there are symmetric in terms of performance in absence of this property.
Quoted 2 lines
Why should we "fix" something that may never be a valid dts?
I would not say invalid. But surely absence of it must be handled and we do that for sure. IIRC, here the presence of it is causing the issue. And if it is present means someone is trying to build it(I do understand this is Qemu but is quite common these days for power and performance balance in many SoC)
[...]
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
I got myself mixed up between places I fiddled with storing the topology, so you can ignore that question Sudeep. Clearly it's the one in
smp_callin()that gets called for each CPU.
Woops.
Hmm I should have read all the messages in the thread. Doing by date/time didn't work well for me 😄.
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
...but is it still worth moving the function call later to head off any allocation failures if core topology code changes?
Agreed, given how we faced similar issues with cacheinfo on few RISC-V platforms.
Signature · 2 lines
Regards, Sudeep
Hey Sudeep,
Quoted 12 lines · Sudeep Holla
On Wed, Jan 04, 2023 at 09:49:48AM +0000, Conor Dooley wrote:
[...]
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Uhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.This DT parameter is not in default Qemu. I've added it for testing (see test steps in below). This is preparation to support asymmetric CPU topology for RISC-V.
Quoted 6 lines
Quoted 2 lines
The property is only valid on arm, so how does arm64 deal with such asymmetric topologies without it?
I don't think we can deal with asymmetric topologies without this. Yes we can detect the difference in the CPU types but we can only assume there are symmetric in terms of performance in absence of this property.
I looked at the bindings for it and forgot that the arm directory of bindings applies to both arm and arm64. I see now that it is also used on arm64.
Quoted 9 lines
Quoted 3 lines
Why should we "fix" something that may never be a valid dts?
I would not say invalid. But surely absence of it must be handled and we do that for sure. IIRC, here the presence of it is causing the issue. And if it is present means someone is trying to build it(I do understand this is Qemu but is quite common these days for power and performance balance in many SoC)
I said "invalid" as the binding is defined for arm{,64} in arm/cpus.yaml & documented in the same directory in cpu-capacity.txt, but not yet on riscv. All bets are off if your cpu node is using invalid properties IMO, at least this one will fail to boot!
However, I see no reason (at this point) that we should deviate from what arm{,64} is doing & that documenation should probably move to a shared location at some point.
Quoted 11 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
I got myself mixed up between places I fiddled with storing the topology, so you can ignore that question Sudeep. Clearly it's the one in
smp_callin()that gets called for each CPU.
Woops.Hmm I should have read all the messages in the thread. Doing by date/time didn't work well for me 😄.
Meh, my fault for getting confused ;)
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
...but is it still worth moving the function call later to head off any allocation failures if core topology code changes?
Agreed, given how we faced similar issues with cacheinfo on few RISC-V platforms.
Sweet, sounds like a plan to me. I'll go suggest some commit message re-wording I think.
Thanks Sudeep!
Quoted 37 lines · Conor Dooley
Hey Sudeep,
Quoted 12 lines · Sudeep Holla
On Wed, Jan 04, 2023 at 09:49:48AM +0000, Conor Dooley wrote:
[...]
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Uhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
This property should not be valid on anything other than arm AFAICT.This DT parameter is not in default Qemu. I've added it for testing (see test steps in below). This is preparation to support asymmetric CPU topology for RISC-V.
Quoted 6 lines
Quoted 2 lines
The property is only valid on arm, so how does arm64 deal with such asymmetric topologies without it?
I don't think we can deal with asymmetric topologies without this. Yes we can detect the difference in the CPU types but we can only assume there are symmetric in terms of performance in absence of this property.
I looked at the bindings for it and forgot that the arm directory of bindings applies to both arm and arm64. I see now that it is also used on arm64.
Quoted 9 lines
Quoted 3 lines
Why should we "fix" something that may never be a valid dts?
I would not say invalid. But surely absence of it must be handled and we do that for sure. IIRC, here the presence of it is causing the issue. And if it is present means someone is trying to build it(I do understand this is Qemu but is quite common these days for power and performance balance in many SoC)
Quoted 9 lines
I said "invalid" as the binding is defined for arm{,64} in
arm/cpus.yaml& documented in the same directory in cpu-capacity.txt, but not yet on riscv. All bets are off if your cpu node is using invalid properties IMO, at least this one will fail to boot!However, I see no reason (at this point) that we should deviate from what arm{,64} is doing & that documenation should probably move to a shared location at some point.
I prefer making this binding generic rather than patching to handle RISC-V differently in the generic code. Since it is optional, the platform need not use it if it is not needed.
Signature · 2 lines
Regards, Sudeep
Quoted 19 lines · Sudeep Holla
Quoted 17 lines · Conor Dooley
Quoted 10 lines · Sudeep Holla
Quoted 2 lines · Conor Dooley
Why should we "fix" something that may never be a valid dts?
I would not say invalid. But surely absence of it must be handled and we do that for sure. IIRC, here the presence of it is causing the issue. And if it is present means someone is trying to build it(I do understand this is Qemu but is quite common these days for power and performance balance in many SoC)
I said "invalid" as the binding is defined for arm{,64} in
arm/cpus.yaml& documented in the same directory in cpu-capacity.txt, but not yet on riscv. All bets are off if your cpu node is using invalid properties IMO, at least this one will fail to boot!
Quoted 8 lines
Quoted 4 lines
However, I see no reason (at this point) that we should deviate from what arm{,64} is doing & that documenation should probably move to a shared location at some point.
I prefer making this binding generic rather than patching to handle RISC-V differently in the generic code. Since it is optional, the platform need not use it if it is not needed.
Oh yeah, I was not suggesting making changes in the generic code. We just need to change our cpu binding to match the arm cpu binding so that having this property is accepted.
I shall go do that at some point today probably.
Thanks,
Conor.
Quoted 11 lines · Conor Dooley
Hello!
Couple comments for you.
+CC Sudeep: I've got a question for you below.
Quoted 4 lines · Leyfoon Tan
Quoted 4 lines
From: Andrew Jones <[email protected]>
Subject: Re: [PATCH] riscv: Move call toinit_cpu_topology()to later initialization stage
On Tue, Jan 03, 2023 at 11:53:16AM +0800, Ley Foon Tan wrote:
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DTUhh, so where did this "capacity-dmips-mhz" property actually come from? I had a quick check of qemu with grep & I don't see anything there that would add this property.
From the DT properties.
Quoted 2 lines
This property should not be valid on anything other than arm AFAICT.
Not sure if we restrict that on arm/arm64 only, but yes it is mostly used on asymmetric CPU systems.
[...]
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
arch/riscv/kernel/smpboot.c static DECLARE_COMPLETION(cpu_running); · line 43
Quoted 11 lines
arch/riscv/kernel/smpboot.c +0
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); } void __init smp_prepare_cpus(unsigned int max_cpus) @@ -48,6 +47,8@@
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
arch/riscv/kernel/smpboot.c void __init smp_prepare_cpus(unsigned int max_cpus) · line 50
Quoted 5 lines
arch/riscv/kernel/smpboot.c +0
-0void __init smp_prepare_cpus (unsigned int max_cpus)arch/riscv/kernel/smpboot.c +1
-0int ret;unsigned int curr_cpuid; +init_cpu_topology ();I know arm64 does this, but there is any real reason for us to do so? @Sudeep, do you know why arm64 calls that each time?
Not sure what you are referring as called each time. IIUC smp_prepare_cpus() must be called only once on the primary during the boot to prepare for the secondary boot. The difference is by this stage we know the max possible CPU and supply the same to the call.
Quoted 5 lines
Or if it is worth "saving" that call on riscv, since arm64 is clearly happily calling it for many years & calling it later would likely head off a good few allocation issues (like the one we saw with the topology reworking a few months ago).
Yes the failure seems like similar issue we saw with cacheinfo and early allocation on RISC-V. However I can't recall why we have done this way on arm64 and not in smp_prepare_boot_cpu() like in RISC-V.
Not sure if that was any helpful.
Signature · 2 lines
Regards, Sudeep
Hey Ley Foon Tan,
Apologies for my various bits of confusion.
Quoted 10 lines · Ley Foon Tan
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT nodes. Thistopology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
I'd like to suggest a change to the commit message:
``` If "capacity-dmips-mhz" is present in a CPU DT node, topology_parse_cpu_capacity() will fail to allocate memory. arm64, with which this code path is shared, does not call topology_parse_cpu_capacity() until later in boot where memory allocation is available. While "capacity-dmips-mhz" is not yet a valid property on RISC-V, invalid properties should be ignored rather than cause issues. Move init_cpu_topology(), which calls topology_parse_cpu_capacity(), to a later initialization stage, to match arm64. As a side effect of this change, RISC-V is "protected" from changes to core topology code that would work on arm64 where memory allocation is safe but on RISC-V isn't. ```
You don't need to use exactly that, but with something along those lines:
Thanks,
Conor.
Quoted 46 lines
Signed-off-by Ley Foon Tan <[email protected]>---
In
drivers/base/arch_topology.c:topology_parse_cpu_capacity():ret = of_property_read_u32(cpu_node, "capacity-dmips-mhz", &cpu_capacity); if (!ret) { if (!raw_capacity) { raw_capacity = kcalloc(num_possible_cpus(), sizeof(*raw_capacity), GFP_KERNEL); if (!raw_capacity) { cap_parsing_failed = true; return false; } --- arch/riscv/kernel/smpboot.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/riscv/kernel/smpboot.c +2
-1@@ -39,7 +39,6 @@ static DECLARE_COMPLETION(cpu_running);void __init smp_prepare_boot_cpu (void ) { -init_cpu_topology (); }void __init smp_prepare_cpus (unsigned int max_cpus)@@ -48,6 +47,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus)int ret;unsigned int curr_cpuid; +init_cpu_topology (); + curr_cpuid =smp_processor_id ();store_cpu_topology (curr_cpuid);numa_store_cpu_info (curr_cpuid);Signature · 1 line
2.25.1
Quoted 32 lines · Ley Foon Tan
Quoted 11 lines
topology_parse_cpu_capacity()is failed to allocate memory withkcalloc()after read "capacity-dmips-mhz" DT parameter in CPU DT nodes. Thistopology_parse_cpu_capacity()is called frominit_cpu_topology(), move call toinit_cpu_topology()to later initialization stage (after memory allocation is available).Note, this refers to ARM64 implementation, call
init_cpu_topology()insmp_prepare_cpus().Tested on Qemu platform.
I'd like to suggest a change to the commit message:
``` If "capacity-dmips-mhz" is present in a CPU DT node, topology_parse_cpu_capacity() will fail to allocate memory. arm64, with which this code path is shared, does not call topology_parse_cpu_capacity() until later in boot where memory allocation is available. While "capacity-dmips-mhz" is not yet a valid property on RISC-V, invalid properties should be ignored rather than cause issues. Move init_cpu_topology(), which calls topology_parse_cpu_capacity(), to a later initialization stage, to match arm64. As a side effect of this change, RISC-V is "protected" from changes to core topology code that would work on arm64 where memory allocation is safe but on RISC-V isn't. ```
Quoted 6 lines
You don't need to use exactly that, but with something along those lines:
Reviewed-by Conor Dooley <[email protected]>Thanks,
Conor.
Hi Conor
Thanks for your suggestions. Will send the v2 and update the commit message.
Regards
Ley Foon