CPU_METHOD_OF_DECLARE() with Linux as non-secure OS
Hello,
I'm working on a dual-core SoC (Cortex-A9 MPCore) with Linux running as the non-secure OS. The secure OS implements some of the APIs used in OMAP2 via SMC.
To start the second core, I used to define .smp in my DT_MACHINE_START definition, but Arnd wrote: "There are other ways to do [.smp and .restart] too, see CPU_METHOD_OF_DECLARE() and register_restart_handler()."
Is CPU_METHOD_OF_DECLARE() the way to go?
Is it "compatible" with executing smc to ask the secure OS to do whatever is needed to start the second core? (I don't understand why starting cores is secure OS responsibility.)
I've located three relevant secure OS services:get_aux_core_bootset_aux_core_boot0set_aux_core_boot_addr
I'm looking for guidance. What mach-* should I take as a reference to implement "second core startup from non-secure OS"?
An ARM engineer suggested PSCI a while back, but I can't change the secure OS API at this time.
Regards.
Quoted 11 lines · Mason
Hello,
I'm working on a dual-core SoC (Cortex-A9 MPCore) with Linux running as the non-secure OS. The secure OS implements some of the APIs used in OMAP2 via SMC.
To start the second core, I used to define .smp in my
DT_MACHINE_STARTdefinition, but Arnd wrote: "There are other ways to do [.smp and .restart] too, seeCPU_METHOD_OF_DECLARE()andregister_restart_handler()."Is
CPU_METHOD_OF_DECLARE()the way to go?
Yes.
Quoted 3 lines
Is it "compatible" with executing smc to ask the secure OS to do whatever is needed to start the second core? (I don't understand why starting cores is secure OS responsibility.)
Yes, that is what PSCI does. In your case, these are not PSCI calls though?
Quoted 8 lines
I've located three relevant secure OS services:
get_aux_core_bootset_aux_core_boot0set_aux_core_boot_addrI'm looking for guidance. What mach-* should I take as a reference to implement "second core startup from non-secure OS"?
I don't think there is a mach example that matches. PSCI would be the closest example. But doing a smc call versus writing some registers are not all that different and there are numerous examples of the latter.
Rob
Quoted 15 lines · Mason
Hello,
I'm working on a dual-core SoC (Cortex-A9 MPCore) with Linux running as the non-secure OS. The secure OS implements some of the APIs used in OMAP2 via SMC.
To start the second core, I used to define .smp in my
DT_MACHINE_STARTdefinition, but Arnd wrote: "There are other ways to do [.smp and .restart] too, seeCPU_METHOD_OF_DECLARE()andregister_restart_handler()."Is
CPU_METHOD_OF_DECLARE()the way to go?
Is it "compatible" with executing smc to ask the secure OS to do whatever is needed to start the second core? (I don't understand why starting cores is secure OS responsibility.)
Quoted 7 lines
I've located three relevant secure OS services:
get_aux_core_bootset_aux_core_boot0set_aux_core_boot_addrI'm looking for guidance. What mach-* should I take as a reference to implement "second core startup from non-secure OS"?
See https://github.com/mansr/linux-tangox/commit/717dd5fab87aa90255460ba409970331cb577247
Signature · 2 lines
M?ns Rullg?rd mans at mansr.com
Quoted 20 lines · M?ns Rullg?rd
Quoted 18 lines · Mason
I'm working on a dual-core SoC (Cortex-A9 MPCore) with Linux running as the non-secure OS. The secure OS implements some of the APIs used in OMAP2 via SMC.
To start the second core, I used to define .smp in my
DT_MACHINE_STARTdefinition, but Arnd wrote: "There are other ways to do [.smp and .restart] too, seeCPU_METHOD_OF_DECLARE()andregister_restart_handler()."Is
CPU_METHOD_OF_DECLARE()the way to go?
Is it "compatible" with executing smc to ask the secure OS to do whatever is needed to start the second core? (I don't understand why starting cores is secure OS responsibility.)I've located three relevant secure OS services:
get_aux_core_bootset_aux_core_boot0set_aux_core_boot_addr
Quoted 4 lines
Quoted 2 lines
I'm looking for guidance. What mach-* should I take as a reference to implement "second core startup from non-secure OS"?
See https://github.com/mansr/linux-tangox/commit/717dd5fab87aa90255460ba409970331cb577247
I discussed cpu1 startup with the firmware author; he said the secure OS does all the heavy lifting, Linux is only expected to provide secondary_startup. AFAICT, we don't need to jump hoops like mach-vexpress did.
I don't think you can define the cpus node in your smp87xx.dtsi because smp8756 is single-core.
BTW, I see that you've tried to enable L2C. I would expect your board would panic?
http://thread.gmane.org/gmane.linux.ports.arm.kernel/447493
Regards.
Quoted 25 lines · Mason
Quoted 23 lines · M?ns Rullg?rd
Quoted 20 lines · Mason
I'm working on a dual-core SoC (Cortex-A9 MPCore) with Linux running as the non-secure OS. The secure OS implements some of the APIs used in OMAP2 via SMC.
To start the second core, I used to define .smp in my
DT_MACHINE_STARTdefinition, but Arnd wrote: "There are other ways to do [.smp and .restart] too, seeCPU_METHOD_OF_DECLARE()andregister_restart_handler()."Is
CPU_METHOD_OF_DECLARE()the way to go?
Is it "compatible" with executing smc to ask the secure OS to do whatever is needed to start the second core? (I don't understand why starting cores is secure OS responsibility.)I've located three relevant secure OS services:
get_aux_core_bootset_aux_core_boot0set_aux_core_boot_addrI'm looking for guidance. What mach-* should I take as a reference to implement "second core startup from non-secure OS"?
Quoted 6 lines
Quoted 1 line
See https://github.com/mansr/linux-tangox/commit/717dd5fab87aa90255460ba409970331cb577247
I discussed cpu1 startup with the firmware author; he said the secure OS does all the heavy lifting, Linux is only expected to provide
secondary_startup. AFAICT, we don't need to jump hoops like mach-vexpress did.
No major hoops really, and it's nice to have there as an extra safety check.
Quoted 2 lines
I don't think you can define the cpus node in your
smp87xx.dtsibecause smp8756 is single-core.
OK, I'll fix that.
Quoted 4 lines
BTW, I see that you've tried to enable L2C. I would expect your board would panic?
I hacked around that, and everything works fine. Nice performance boost too.
There's also something wrong with the L2C-310 aux control register setting. The SMC call ID from OMAP (0x109) which is also used in some Sigma code I found somewhere doesn't seem to do anything, so the register is left at the value set by the secure boot code. Perhaps you can check with your firmware guy if there's another way of writing that register.
Signature · 2 lines
M?ns Rullg?rd mans at mansr.com
Quoted 6 lines · M?ns Rullg?rd
There's also something wrong with the L2C-310 aux control register setting. The SMC call ID from OMAP (
0x109) which is also used in some Sigma code I found somewhere doesn't seem to do anything, so the register is left at the value set by the secure boot code. Perhaps you can check with your firmware guy if there's another way of writing that register.
IIRC, only debug firmware allows writes to L2 AUXCTRL (after filtering some of the bits out), while production firmware ignores them completely.
IME, the smc handler should default to return ENOTSUP; that way, when a syscall disappears due to ifdef-ery, the caller gets a meaningful answer.
We just had an interesting internal discussion about L2 AUXCTRL. For my education, what value would you like to write to AUXCTRL? :-)
Regards.
Quoted 12 lines · Mason
Quoted 6 lines · M?ns Rullg?rd
There's also something wrong with the L2C-310 aux control register setting. The SMC call ID from OMAP (
0x109) which is also used in some Sigma code I found somewhere doesn't seem to do anything, so the register is left at the value set by the secure boot code. Perhaps you can check with your firmware guy if there's another way of writing that register.IIRC, only debug firmware allows writes to L2 AUXCTRL (after filtering some of the bits out), while production firmware ignores them completely.
Quoted 6 lines
IME, the smc handler should default to return ENOTSUP; that way, when a syscall disappears due to ifdef-ery, the caller gets a meaningful answer.
We just had an interesting internal discussion about L2 AUXCTRL. For my education, what value would you like to write to AUXCTRL? :-)
The best value to use depends on the workload, so it would be nice to be able to control all the purely performance related bits. I see no possible benefit in restricting the non-secure kernel from writing these.
Signature · 2 lines
M?ns Rullg?rd mans at mansr.com
Quoted 18 lines · M?ns Rullg?rd
Quoted 16 lines · Mason
Quoted 6 lines · M?ns Rullg?rd
There's also something wrong with the L2C-310 aux control register setting. The SMC call ID from OMAP (
0x109) which is also used in some Sigma code I found somewhere doesn't seem to do anything, so the register is left at the value set by the secure boot code. Perhaps you can check with your firmware guy if there's another way of writing that register.IIRC, only debug firmware allows writes to L2 AUXCTRL (after filtering some of the bits out), while production firmware ignores them completely.
IME, the smc handler should default to return ENOTSUP; that way, when a syscall disappears due to ifdef-ery, the caller gets a meaningful answer.
Quoted 7 lines
Quoted 2 lines
We just had an interesting internal discussion about L2 AUXCTRL. For my education, what value would you like to write to AUXCTRL? :-)
The best value to use depends on the workload, so it would be nice to be able to control all the purely performance related bits. I see no possible benefit in restricting the non-secure kernel from writing these.
For the record, the latest firmware uses 0x72860401.
[ 0] Full Line of Zero enable
[10] High Priority for SO and Dev Reads enable
[11] Store buffer device limitation disable
[12] Exclusive cache configuration disable
[13] Shared Attribute Invalidate disable
[16] Associativity 8-way
[17-19] Way-size = 64
[20] Event monitor bus disable
[21] Parity disable
[22] Shared attribute override disable
[23:24] Force write allocate = 0x1
[25] Cache replacement policy = 1
[26] Non-secure lock-down disable
[27] Non-secure interrupt access control disable
[28] Data prefetch enable
[29] Instruction prefetch enable
[30] Early BRESP enable
Everyone, feel free to comment on these choices ;-)
Regards.
Quoted 4 lines · M?ns Rullg?rd
The best value to use depends on the workload, so it would be nice to be able to control all the purely performance related bits. I see no possible benefit in restricting the non-secure kernel from writing these.
For the most case, the auxiliary control register settings are static. Once the L2 cache is enabled, you can't write to the control register until the L2 cache is disabled, and you can't sensibly do that on a running system without taking the system down.
Those which can be tuned at runtime (the prefetch enables) are also available in the prefetch control register, along with the prefetch offset.
The auxiliary control register is really to do with configuring the cache to the system its in rather than about performance.
Eg, you could have the FLZ bit set in the L2 cache auxiliary register if its wired to a Cortex A9 CPU and the FLZ signal is wired. That would be perfectly reasonable, provided the FLZ bit in the Cortex A9's control register is disabled when the L2 cache is otherwise disabled.
(I guess I ought to quieten down the pr_err() in that case...)
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
Quoted 7 lines · Russell King - ARM Linux
Eg, you could have the FLZ bit set in the L2 cache auxiliary register if its wired to a Cortex A9 CPU and the FLZ signal is wired. That would be perfectly reasonable, provided the FLZ bit in the Cortex A9's control register is disabled when the L2 cache is otherwise disabled.
(I guess I ought to quieten down the
pr_err()in that case...)
Based on the pr_err(), I thought it was considered an error, so I asked the firmware author to leave L2AUX.FLZ cleared, and only set it just before the cache is enabled, and clear it after the cache is disabled.
Do you remember why you considered it an error at the time?
Regards.
Quoted 23 lines · Mason
Quoted 21 lines · M?ns Rullg?rd
Quoted 18 lines · Mason
Quoted 6 lines · M?ns Rullg?rd
There's also something wrong with the L2C-310 aux control register setting. The SMC call ID from OMAP (
0x109) which is also used in some Sigma code I found somewhere doesn't seem to do anything, so the register is left at the value set by the secure boot code. Perhaps you can check with your firmware guy if there's another way of writing that register.IIRC, only debug firmware allows writes to L2 AUXCTRL (after filtering some of the bits out), while production firmware ignores them completely.
IME, the smc handler should default to return ENOTSUP; that way, when a syscall disappears due to ifdef-ery, the caller gets a meaningful answer.
We just had an interesting internal discussion about L2 AUXCTRL. For my education, what value would you like to write to AUXCTRL? :-)
Quoted 6 lines
Quoted 4 lines
The best value to use depends on the workload, so it would be nice to be able to control all the purely performance related bits. I see no possible benefit in restricting the non-secure kernel from writing these.
For the record, the latest firmware uses
0x72860401.
My SMP8759 firmware sets it to 0x62460801.
Quoted 1 line
[ 0] Full Line of Zero enable
This is only takes effect if the feature is also enabled in the cp15 aux control register, so it's safe to always set it. It would of course be nice to have a way of updating the cp15 aux control register...
Quoted 1 line
[10] High Priority for SO and Dev Reads enable
The best value here depends on the workload. It would be nice for the system integrator to have control over it.
Quoted 1 line
[11] Store buffer device limitation disable
Ditto.
Quoted 1 line
[12] Exclusive cache configuration disable
Like FLZ, this has a corresponding cp15 bit. It is a potentially useful feature, so again it would be nice to have control over it.
Quoted 2 lines
[13] Shared Attribute Invalidate disable
[22] Shared attribute override disable
Not sure about these.
Quoted 2 lines
[16] Associativity 8-way
[17-19] Way-size = 64
These must match the actual hardware.
Quoted 1 line
[20] Event monitor bus disable
OK to disable.
Quoted 1 line
[21] Parity disable
Must match hardware.
Quoted 1 line
[23:24] Force write allocate =
0x1
Why was this changed? Write-allocate can be useful, and this attribute is normally supplied by the page table. Forcing it off should only be done to work around broken hardware or software that can't be fixed.
Quoted 1 line
[25] Cache replacement policy = 1
Again, the best value depends on the workload. Better to let the OS choose.
Quoted 1 line
[26] Non-secure lock-down disable
The Linux setup code has this comment:
/*
* Always enable non-secure access to the lockdown registers -
* we write to them as part of the L2C enable sequence so they
* need to be accessible.
*/
The lockdown registers are only accessed if this bit is actually set, so leaving it off is harmless provided the cache is left unlocked by the secure firmware.
Quoted 1 line
[27] Non-secure interrupt access control disable
Access to the interrupt control registers is useful in conjunction with the event counters.
Quoted 2 lines
[28] Data prefetch enable
[29] Instruction prefetch enable
These are accessible also from the prefetch control register, so as long as that is accessible, the values set here don't matter.
Quoted 1 line
[30] Early BRESP enable
Linux wants this enabled. It is generally good for performance.
Signature · 2 lines
M?ns Rullg?rd mans at mansr.com