Slow booting on x15
Subsystems (4)
Hi,
Has anyone noticed that booting would have slowed down in 5.8 and in 5.9-rcs? I'm booting x15 with my custom config builder, which uses omap2plus_defconfig as a base. I see:
v5.7
[ 0.597765] cpuidle: using governor menu [ 0.793198] No ATAGs?
[ 9.654611] Run /sbin/init as init process
5.8
[ 0.616660] cpuidle: using governor menu [ 4.439856] No ATAGs?
[ 15.484710] Run /sbin/init as init process
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?
[ 19.522536] Run /sbin/init as init process
So v5.7 boots quickly, and init starts around 9.6 secs. 5.8 is much slower, and 5.9-rc5 even slower.
My config builder enables all kinds of debug options, so possibly it could be just more debug features or such having been implemented, or enabled in omap2plus_defconfig...
Tomi
Signature · 2 lines
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
Quoted 17 lines · Tomi Valkeinen
Hi,
Has anyone noticed that booting would have slowed down in 5.8 and in 5.9-rcs? I'm booting x15 with my custom config builder, which uses
omap2plus_defconfigas a base. I see:v5.7
[ 0.597765] cpuidle: using governor menu [ 0.793198] No ATAGs?[ 9.654611] Run /sbin/init as init process5.8
[ 0.616660] cpuidle: using governor menu [ 4.439856] No ATAGs?[ 15.484710] Run /sbin/init as init process
Quoted 8 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processSo v5.7 boots quickly, and init starts around 9.6 secs. 5.8 is much slower, and 5.9-rc5 even slower.
I've tracked this down to device_link_add_missing_supplier_links() taking about 40+ms to run, for every device added to the system. Given the large number of devices, this adds up to 10s on v5.9-rc5, when called during the initial of_platform_populate().
It doesn't seem there's a single culprit here (although I may have missed it), "just" lots of walks through the whole device tree that end up taking way too much time. I'm not planning to investigate this further at this point.
Quoted 2 lines
My config builder enables all kinds of debug options, so possibly it could be just more debug features or such having been implemented, or enabled in
omap2plus_defconfig...
Signature · 3 lines
Regards, Laurent Pinchart
Tony Lindgren
maintainerQuoted 32 lines · Laurent Pinchart
Quoted 17 lines · Tomi Valkeinen
Hi,
Has anyone noticed that booting would have slowed down in 5.8 and in 5.9-rcs? I'm booting x15 with my custom config builder, which uses
omap2plus_defconfigas a base. I see:v5.7
[ 0.597765] cpuidle: using governor menu [ 0.793198] No ATAGs?[ 9.654611] Run /sbin/init as init process5.8
[ 0.616660] cpuidle: using governor menu [ 4.439856] No ATAGs?[ 15.484710] Run /sbin/init as init processQuoted 8 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processSo v5.7 boots quickly, and init starts around 9.6 secs. 5.8 is much slower, and 5.9-rc5 even slower.
I've tracked this down to
device_link_add_missing_supplier_links()taking about 40+ms to run, for every device added to the system. Given the large number of devices, this adds up to 10s on v5.9-rc5, when called during the initialof_platform_populate().
Quoted 4 lines
It doesn't seem there's a single culprit here (although I may have missed it), "just" lots of walks through the whole device tree that end up taking way too much time. I'm not planning to investigate this further at this point.
Hmm so do you have some commit that causes this slowness?
In general, we should now be looking each device at module_init time only when configured in the dts. I guess it's possible we still have some extra walking of the whole device tree left but this sounds like a more generic issue though and not related to the omap dts conversion.
Regards,
Tony
Hi Tony,
Quoted 33 lines · Tony Lindgren
Quoted 32 lines · Laurent Pinchart
Quoted 17 lines · Tomi Valkeinen
Hi,
Has anyone noticed that booting would have slowed down in 5.8 and in 5.9-rcs? I'm booting x15 with my custom config builder, which uses
omap2plus_defconfigas a base. I see:v5.7
[ 0.597765] cpuidle: using governor menu [ 0.793198] No ATAGs?[ 9.654611] Run /sbin/init as init process5.8
[ 0.616660] cpuidle: using governor menu [ 4.439856] No ATAGs?[ 15.484710] Run /sbin/init as init processQuoted 8 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processSo v5.7 boots quickly, and init starts around 9.6 secs. 5.8 is much slower, and 5.9-rc5 even slower.
I've tracked this down to
device_link_add_missing_supplier_links()taking about 40+ms to run, for every device added to the system. Given the large number of devices, this adds up to 10s on v5.9-rc5, when called during the initialof_platform_populate().
Quoted 6 lines
Quoted 4 lines
It doesn't seem there's a single culprit here (although I may have missed it), "just" lots of walks through the whole device tree that end up taking way too much time. I'm not planning to investigate this further at this point.
Hmm so do you have some commit that causes this slowness?
No, I haven't had time to bisect this.
Quoted 5 lines
In general, we should now be looking each device at
module_inittime only when configured in the dts. I guess it's possible we still have some extra walking of the whole device tree left but this sounds like a more generic issue though and not related to the omap dts conversion.
There's a call to of_platform_populate() in pdata_quirks_init(), and that takes between 5 and 10 seconds to run, depending on the board.
Signature · 3 lines
Regards, Laurent Pinchart
Tony Lindgren
maintainerQuoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 33 lines · Tony Lindgren
Quoted 32 lines · Laurent Pinchart
Quoted 17 lines · Tomi Valkeinen
Hi,
Has anyone noticed that booting would have slowed down in 5.8 and in 5.9-rcs? I'm booting x15 with my custom config builder, which uses
omap2plus_defconfigas a base. I see:v5.7
[ 0.597765] cpuidle: using governor menu [ 0.793198] No ATAGs?[ 9.654611] Run /sbin/init as init process5.8
[ 0.616660] cpuidle: using governor menu [ 4.439856] No ATAGs?[ 15.484710] Run /sbin/init as init processQuoted 8 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processSo v5.7 boots quickly, and init starts around 9.6 secs. 5.8 is much slower, and 5.9-rc5 even slower.
I've tracked this down to
device_link_add_missing_supplier_links()taking about 40+ms to run, for every device added to the system. Given the large number of devices, this adds up to 10s on v5.9-rc5, when called during the initialof_platform_populate().Quoted 6 lines
Quoted 4 lines
It doesn't seem there's a single culprit here (although I may have missed it), "just" lots of walks through the whole device tree that end up taking way too much time. I'm not planning to investigate this further at this point.
Hmm so do you have some commit that causes this slowness?
No, I haven't had time to bisect this.
Quoted 8 lines
Quoted 5 lines
In general, we should now be looking each device at
module_inittime only when configured in the dts. I guess it's possible we still have some extra walking of the whole device tree left but this sounds like a more generic issue though and not related to the omap dts conversion.There's a call to
of_platform_populate()inpdata_quirks_init(), and that takes between 5 and 10 seconds to run, depending on the board.
If that is the issue we may be able to improve it a bit by probing the interconnects separately for less deferred probe cases.
Regards,
Tony
Tony Lindgren
maintainerQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init process
Hmm v5.9-rc1 here with NFSroot shows this for me:
[ 11.704511] Run /sbin/init as init process
And v5.9-rc6 shows:
[ 13.120913] Run /sbin/init as init process
With at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with omap2plus_defconfig.
Regards,
Tony
Quoted 9 lines · Tony Lindgren
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processHmm v5.9-rc1 here with NFSroot shows this for me:
Quoted 8 lines
[ 11.704511] Run /sbin/init as init processAnd v5.9-rc6 shows:
[ 13.120913] Run /sbin/init as init processWith at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with
omap2plus_defconfig.
I'm also using nfsroot, so I didn't put too much weight on the "Run /sbin/init as init process" line, as it could also be caused the network or the nfs server. But the "No ATAGs" lines already show an increase from 0.8s to 6.7s from v5.7 to v5.9-rc5.
Tomi
Signature · 2 lines
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
Tony Lindgren
maintainerQuoted 14 lines · Tomi Valkeinen
Quoted 13 lines · Tony Lindgren
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processHmm v5.9-rc1 here with NFSroot shows this for me:
[ 11.704511] Run /sbin/init as init processAnd v5.9-rc6 shows:
Quoted 8 lines
Quoted 4 lines
[ 13.120913] Run /sbin/init as init processWith at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with
omap2plus_defconfig.I'm also using nfsroot, so I didn't put too much weight on the "Run
/sbin/initas init process" line, as it could also be caused the network or the nfs server. But the "No ATAGs" lines already show an increase from 0.8s to 6.7s from v5.7 to v5.9-rc5.
That's strange, I'm seeing this with v5.9-rc6:
[ 2.357585] No ATAGs?
Regards,
Tony
Hi Tony,
Quoted 15 lines · Tony Lindgren
Quoted 14 lines · Tomi Valkeinen
Quoted 13 lines · Tony Lindgren
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processHmm v5.9-rc1 here with NFSroot shows this for me:
[ 11.704511] Run /sbin/init as init processAnd v5.9-rc6 shows:
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
[ 13.120913] Run /sbin/init as init processWith at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with
omap2plus_defconfig.I'm also using nfsroot, so I didn't put too much weight on the "Run
/sbin/initas init process" line, as it could also be caused the network or the nfs server. But the "No ATAGs" lines already show an increase from 0.8s to 6.7s from v5.7 to v5.9-rc5.
Yes, that's the main issue.
Quoted 3 lines
That's strange, I'm seeing this with v5.9-rc6:
[ 2.357585] No ATAGs?
What's your test platform ?
Still, 1.5s spent there is quite a lot.
Signature · 3 lines
Regards, Laurent Pinchart
Tony Lindgren
maintainerQuoted 27 lines · Laurent Pinchart
Hi Tony,
Quoted 15 lines · Tony Lindgren
Quoted 14 lines · Tomi Valkeinen
Quoted 13 lines · Tony Lindgren
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processHmm v5.9-rc1 here with NFSroot shows this for me:
[ 11.704511] Run /sbin/init as init processAnd v5.9-rc6 shows:
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
[ 13.120913] Run /sbin/init as init processWith at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with
omap2plus_defconfig.I'm also using nfsroot, so I didn't put too much weight on the "Run
/sbin/initas init process" line, as it could also be caused the network or the nfs server. But the "No ATAGs" lines already show an increase from 0.8s to 6.7s from v5.7 to v5.9-rc5.
Quoted 7 lines
Yes, that's the main issue.
Quoted 3 lines
That's strange, I'm seeing this with v5.9-rc6:
[ 2.357585] No ATAGs?What's your test platform ?
This is from beagle x15. I wonder how Tomi is seeing over six seconds at that point though while I'm seeing two something.
Quoted 1 line
Still, 1.5s spent there is quite a lot.
Probably that's when module_init runs and we probe almost everything. Might be worth profiling to see if we can optimize out some full dtb tree searches for example.
Regards,
Tony
Quoted 30 lines · Tony Lindgren
Quoted 29 lines · Laurent Pinchart
Hi Tony,
Quoted 15 lines · Tony Lindgren
Quoted 14 lines · Tomi Valkeinen
Quoted 13 lines · Tony Lindgren
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Tomi Valkeinen
Quoted 5 lines
v5.9-rc5
[ 0.706165] cpuidle: using governor menu [ 6.761812] No ATAGs?[ 19.522536] Run /sbin/init as init processHmm v5.9-rc1 here with NFSroot shows this for me:
[ 11.704511] Run /sbin/init as init processAnd v5.9-rc6 shows:
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
[ 13.120913] Run /sbin/init as init processWith at least two seconds spent on waiting for Ethernet PHY to negotiate the speed. This is with
omap2plus_defconfig.I'm also using nfsroot, so I didn't put too much weight on the "Run
/sbin/initas init process" line, as it could also be caused the network or the nfs server. But the "No ATAGs" lines already show an increase from 0.8s to 6.7s from v5.7 to v5.9-rc5.Yes, that's the main issue.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
That's strange, I'm seeing this with v5.9-rc6:
[ 2.357585] No ATAGs?What's your test platform ?
This is from beagle x15. I wonder how Tomi is seeing over six seconds at that point though while I'm seeing two something.
And I see 10s on the same platform.
Quoted 5 lines
Quoted 1 line
Still, 1.5s spent there is quite a lot.
Probably that's when
module_initruns and we probe almost everything. Might be worth profiling to see if we can optimize out some full dtb tree searches for example.
As far as I can tell, module_init() runs later than that. If we probed everything there would be lots of messages printed during that period of time.
Signature · 3 lines
Regards, Laurent Pinchart
Tony Lindgren
maintainerQuoted 5 lines · Laurent Pinchart
Quoted 2 lines · Tony Lindgren
This is from beagle x15. I wonder how Tomi is seeing over six seconds at that point though while I'm seeing two something.
And I see 10s on the same platform.
Hmm I wonder why that does not happen to me? I don't have any displays connected currently, maybe that makes a difference?
Can you guys try to bisect it down?
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
Still, 1.5s spent there is quite a lot.
Probably that's when
module_initruns and we probe almost everything. Might be worth profiling to see if we can optimize out some full dtb tree searches for example.As far as I can tell,
module_init()runs later than that. If we probed everything there would be lots of messages printed during that period of time.
Hmm OK. Sorry no ideas other than than me not having displays connected currently.
Regards,
Tony
Quoted 18 lines · Tony Lindgren
Quoted 5 lines · Laurent Pinchart
Quoted 2 lines · Tony Lindgren
This is from beagle x15. I wonder how Tomi is seeing over six seconds at that point though while I'm seeing two something.
And I see 10s on the same platform.
Hmm I wonder why that does not happen to me? I don't have any displays connected currently, maybe that makes a difference?
Can you guys try to bisect it down?
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
Still, 1.5s spent there is quite a lot.
Probably that's when
module_initruns and we probe almost everything. Might be worth profiling to see if we can optimize out some full dtb tree searches for example.
Quoted 6 lines
Quoted 3 lines
As far as I can tell,
module_init()runs later than that. If we probed everything there would be lots of messages printed during that period of time.Hmm OK. Sorry no ideas other than than me not having displays connected currently.
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init process
v5.9-rc7
[ 15.085373] Run /sbin/init as init process
It appears to be 'fixed' in next-20200928: the board does not even boot.
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process
-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:
- Péter
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
Tony Lindgren
maintainerQuoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:
Booting with initcall_debug I see this with current Linux next:
...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with initcall_debug?
Regards,
Tony
Tony Lindgren
maintainerQuoted 26 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
Quoted 14 lines
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?
And the long time above with customize_machine() ends up being pdata_quirks_init() calling of_platform_populate().
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?
But maybe the long delay is something else for you guys so please check.
Regards,
Tony
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 26 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
Quoted 14 lines
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().
That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?
Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024K
after:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process
----
drivers/of/platform.c +2 -2
@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {
@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );
@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }
Signature · 2 lines
Best regards, grygorii
Hi Grygorii et al,
Quoted 6 lines · Grygorii Strashko
Quoted 5 lines · Laurent Pinchart
Quoted 4 lines · Tony Lindgren
Quoted 4 lines
Quoted 4 lines
-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:
1. Is irq=POLL normal behavior for your board?
2. As this is a Micrel PHY, perhaps you are affected by the changes in the configuration defaults in commit bcf3440c6dd78bfe ("net: phy:
micrel: add phy-mode support for the KSZ9031 PHY")?
See also the performance drop figures in the description of quick fix 9b23203c32ee02cd ("ravb: Mask PHY mode to avoid inserting delays twice") (and better solution
https://lore.kernel.org/linux-renesas-soc/[email protected])
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
hi Geert,
Quoted 16 lines · Geert Uytterhoeven
Hi Grygorii et al,
Quoted 6 lines · Grygorii Strashko
Quoted 5 lines · Laurent Pinchart
Quoted 4 lines · Tony Lindgren
Quoted 4 lines
Quoted 4 lines
-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:1. Is irq=POLL normal behavior for your board?
2. As this is a Micrel PHY, perhaps you are affected by the changes in the configuration defaults in commitbcf3440c6dd78bfe("net: phy:
micrel: add phy-mode support for the KSZ9031 PHY")?
Quoted 4 lines
See also the performance drop figures in the description of quick fix
9b23203c32ee02cd("ravb: Mask PHY mode to avoid inserting delays twice") (and better solution
https://lore.kernel.org/linux-renesas-soc/[email protected])
It's not about Ethernet PHY and I've tried different board am571x-idk.
The boot delay introduced first very early during boot:
Quoted 11 lines
Quoted 8 lines
[ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().
And was narrowed down to ^ of_platform_populate() - above customize_machine() costs 2sec.
related discussion https://lkml.org/lkml/2020/6/17/452
Signature · 2 lines
Best regards, grygorii
Quoted 102 lines · Grygorii Strashko
Quoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );
drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();
Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
To explain more, if you call fw_devlink_pause/resume() inside of_platform_populate() you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to use defer_fw_devlink_lock to manage the list used by fw_devlink_pause/resume().
I'll add more details later. But yeah, this patch isn't safe as is.
-Saravana
Quoted 116 lines · Saravana Kannan
Quoted 102 lines · Grygorii Strashko
Quoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
Quoted 8 lines
To explain more, if you call fw_devlink_pause/resume() inside
of_platform_populate()you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to usedefer_fw_devlink_lockto manage the list used by fw_devlink_pause/resume().I'll add more details later. But yeah, this patch isn't safe as is.
I assume you going to fix it your way, right?
Just FYI. On ARM main DT population path is:
arch_initcall(customize_machine)
machine_desc->init_machine(); of_platform_default_populate(); -- or -- of_platform_populate();
the of_platform_default_populate_init() called later due to make/linker dependencies.
As result, existing pause/resume optimizations don't fully working.
Thank you.
Signature · 2 lines
Best regards, grygorii
Quoted 127 lines · Grygorii Strashko
Quoted 123 lines · Saravana Kannan
Quoted 102 lines · Grygorii Strashko
Quoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
To explain more, if you call fw_devlink_pause/resume() inside
of_platform_populate()you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to usedefer_fw_devlink_lockto manage the list used by fw_devlink_pause/resume().
Quoted 3 lines
Quoted 1 line
I'll add more details later. But yeah, this patch isn't safe as is.
I assume you going to fix it your way, right?
I'll be glad to fix this if I find a way out. But currently, there's no "my way" for a generic fix to use inside of_platform_populate(). However...
Quoted 11 lines
Just FYI. On ARM main DT population path is:
arch_initcall(customize_machine)machine_desc->init_machine(); of_platform_default_populate(); -- or -- of_platform_populate();the
of_platform_default_populate_init()called later due to make/linker dependencies.As result, existing pause/resume optimizations don't fully working.
We can just add this optimization around all the other places the top level addition of DT devices is done. I wasn't aware of (more like forgot) the init_machine() path.
Can you give the call path in your case? Just to make it a bit easier for me.
Thanks,
Saravana
Quoted 148 lines · Saravana Kannan
Quoted 127 lines · Grygorii Strashko
Quoted 123 lines · Saravana Kannan
Quoted 102 lines · Grygorii Strashko
Quoted 45 lines · Laurent Pinchart
Hi Tony,
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
To explain more, if you call fw_devlink_pause/resume() inside
of_platform_populate()you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to usedefer_fw_devlink_lockto manage the list used by fw_devlink_pause/resume().Quoted 3 lines
Quoted 1 line
I'll add more details later. But yeah, this patch isn't safe as is.
I assume you going to fix it your way, right?
I'll be glad to fix this if I find a way out. But currently, there's no "my way" for a generic fix to use inside
of_platform_populate(). However...Quoted 10 lines
Just FYI. On ARM main DT population path is:
arch_initcall(customize_machine)machine_desc->init_machine(); of_platform_default_populate(); -- or -- of_platform_populate();the
of_platform_default_populate_init()called later due to make/linker dependencies.
Quoted 7 lines
Quoted 1 line
As result, existing pause/resume optimizations don't fully working.
We can just add this optimization around all the other places the top level addition of DT devices is done. I wasn't aware of (more like forgot) the
init_machine()path.Can you give the call path in your case? Just to make it a bit easier for me.
I think I have a simple solution. I can just move the fw_devlink_pause/resume() inside of_platform_default_populate() and call those only if "root" == NULL.
-Saravana
Hi Saravana,
Quoted 144 lines · Saravana Kannan
Quoted 143 lines · Saravana Kannan
Quoted 118 lines · Grygorii Strashko
Quoted 117 lines · Saravana Kannan
Quoted 97 lines · Grygorii Strashko
Quoted 43 lines · Laurent Pinchart
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
To explain more, if you call fw_devlink_pause/resume() inside
of_platform_populate()you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to usedefer_fw_devlink_lockto manage the list used by fw_devlink_pause/resume().Quoted 3 lines
Quoted 1 line
I'll add more details later. But yeah, this patch isn't safe as is.
I assume you going to fix it your way, right?
I'll be glad to fix this if I find a way out. But currently, there's no "my way" for a generic fix to use inside
of_platform_populate(). However...Quoted 11 lines
Just FYI. On ARM main DT population path is:
arch_initcall(customize_machine)machine_desc->init_machine(); of_platform_default_populate(); -- or -- of_platform_populate();the
of_platform_default_populate_init()called later due to make/linker dependencies.As result, existing pause/resume optimizations don't fully working.
We can just add this optimization around all the other places the top level addition of DT devices is done. I wasn't aware of (more like forgot) the
init_machine()path.
Quoted 5 lines
Quoted 1 line
Can you give the call path in your case? Just to make it a bit easier for me.
I think I have a simple solution. I can just move the fw_devlink_pause/resume() inside
of_platform_default_populate()and call those only if "root" == NULL.
I'd be happy to test a patch. On my device, the initial of_platform_populate() call from machine_desc->init_machine() takes between 6 and 10 seconds to run with v5.9-rc5, compared to 200ms on v5.7. That's a fairly bad regression, all the people who have worked hard so reduce boot time would really hate this :-)
Signature · 3 lines
Regards, Laurent Pinchart
Quoted 154 lines · Laurent Pinchart
Hi Saravana,
Quoted 144 lines · Saravana Kannan
Quoted 143 lines · Saravana Kannan
Quoted 118 lines · Grygorii Strashko
Quoted 117 lines · Saravana Kannan
Quoted 97 lines · Grygorii Strashko
Quoted 43 lines · Laurent Pinchart
Quoted 42 lines · Tony Lindgren
Quoted 40 lines · Tony Lindgren
Quoted 10 lines · Peter Ujfalusi
Fwiw on my beagle x15
v5.8
[ 9.908787] Run /sbin/init as init processv5.9-rc7
[ 15.085373] Run /sbin/init as init processIt appears to be 'fixed' in next-20200928: the board does not even boot.
Yeah so it seems :(
Quoted 8 lines
next-20200928 on omap5
[ 9.936806] Run /sbin/init as init process-rc7 spends most of it's time:
[ 7.635530] Micrel KSZ9031 Gigabit PHY 48485000.mdio:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY] (mii_bus:phy_addr=48485000.mdio:01, irq=POLL) [ 14.956671] cpsw 48484000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off [ 15.005211] IP-Config: Complete:Booting with
initcall_debugI see this with current Linux next:...
[ 1.697313] cpuidle: using governor menu [ 1.701353] initcall init_menu+0x0/0xc returned 0 after 0 usecs [ 1.707458] calling gpmc_init+0x0/0x10 @ 1 [ 1.711784] initcall gpmc_init+0x0/0x10 returned 0 after 0 usecs [ 1.717974] calling omap3_l3_init+0x0/0x10 @ 1 [ 1.722653] initcall omap3_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.729201] calling omap_l3_init+0x0/0x10 @ 1 [ 1.733791] initcall omap_l3_init+0x0/0x10 returned 0 after 0 usecs [ 1.740314] calling gate_vma_init+0x0/0x70 @ 1 [ 1.744976] initcall gate_vma_init+0x0/0x70 returned 0 after 0 usecs [ 1.751522] calling customize_machine+0x0/0x30 @ 1 [ 3.823114] initcall customize_machine+0x0/0x30 returned 0 after 2011718 usecs [ 3.830566] calling init_atags_procfs+0x0/0xec @ 1 [ 3.835583] No ATAGs?Quoted 10 lines
Quoted 2 lines
And the long time above with
customize_machine()ends up beingpdata_quirks_init()callingof_platform_populate().That's what the delay is for me (I think I've reported that initially).
Quoted 4 lines
Quoted 2 lines
Laurent & Tomi, care to check what you guys see in the slow booting case after booting with
initcall_debug?But maybe the long delay is something else for you guys so please check.
It's all devlink :( Looks like sometimes, improvements (PM) could became so complicated that time required to execute such algorithms may completely eliminate all expected benefits. Will not be surprised if PM consumption also increased instead of decreasing in some cases.
not sure if it's 100% correct, but below diff reduces boot time from 7.6sec to 3.7sec :P
before:
[ 0.053870] cpuidle: using governor menu [ 2.505971] No ATAGs? ... [ 7.562317] Freeing unused kernel memory: 1024Kafter:
[ 0.053800] cpuidle: using governor menu [ 0.136853] No ATAGs? [ 3.716218] devtmpfs: mounted [ 3.719628] Freeing unused kernel memory: 1024K [ 3.724266] Run /sbin/init as init process----
drivers/of/platform.c +2
-0@@ -481,6 +481,7 @@ int of_platform_populate(struct device_node *root,pr_debug (" starting at: %pOF\n" , root);device_links_supplier_sync_state_pause (); +fw_devlink_pause ();for_each_child_of_node (root, child) { rc =of_platform_bus_create (child, matches, lookup, parent, true);if (rc) {@@ -488,6 +489,7 @@ int of_platform_populate(struct device_node *root,break ; } } +fw_devlink_resume ();device_links_supplier_sync_state_resume ();of_node_set_flag (root,OF_POPULATED_BUS );drivers/of/platform.c static int __init of_platform_default_populate_init(void) · line 543
Quoted 6 lines
drivers/of/platform.c +0
-2@@ -538,9 +540,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();Your analysis is right, but this change is not safe. You'll get an unlocked linked list trampling if you call it outside of where it's called now. That's explicitly why I didn't do it the way this patch does it.
To explain more, if you call fw_devlink_pause/resume() inside
of_platform_populate()you can end up calling it in the context of another device's probe function. When a device's probe function is called, a has a bunch of other locks held and you'll cause a deadlock. To avoid that, I had to usedefer_fw_devlink_lockto manage the list used by fw_devlink_pause/resume().Quoted 3 lines
Quoted 1 line
I'll add more details later. But yeah, this patch isn't safe as is.
I assume you going to fix it your way, right?
I'll be glad to fix this if I find a way out. But currently, there's no "my way" for a generic fix to use inside
of_platform_populate(). However...Quoted 11 lines
Just FYI. On ARM main DT population path is:
arch_initcall(customize_machine)machine_desc->init_machine(); of_platform_default_populate(); -- or -- of_platform_populate();the
of_platform_default_populate_init()called later due to make/linker dependencies.As result, existing pause/resume optimizations don't fully working.
We can just add this optimization around all the other places the top level addition of DT devices is done. I wasn't aware of (more like forgot) the
init_machine()path.Quoted 5 lines
Quoted 1 line
Can you give the call path in your case? Just to make it a bit easier for me.
I think I have a simple solution. I can just move the fw_devlink_pause/resume() inside
of_platform_default_populate()and call those only if "root" == NULL.
Quoted 5 lines
I'd be happy to test a patch. On my device, the initial
of_platform_populate()call frommachine_desc->init_machine()takes between 6 and 10 seconds to run with v5.9-rc5, compared to 200ms on v5.7. That's a fairly bad regression, all the people who have worked hard so reduce boot time would really hate this :-)
Thanks! Patch coming up in a few minutes.
-Saravana
PATCH 1/1 of: platform: Batch fwnode parsing in the init_machine() path
When commit 93d2e4322aa7 ("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through the init_machine() path.
This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside of_platform_default_populate().
--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
drivers/of/platform.c +15 -4
@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret; + +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent); + +if (!root) +fw_devlink_resume (); +return ret; }EXPORT_SYMBOL_GPL (of_platform_default_populate);
@@ -538,9 +551,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }
Signature · 1 line
2.28.0.709.gb0816b6eb0-goog
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Hi Saravana,
Thank you for the patch.
Quoted 7 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?
:-(
Quoted 44 lines
Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +15
-4@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret; + +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent); + +if (!root) +fw_devlink_resume (); +return ret; }EXPORT_SYMBOL_GPL (of_platform_default_populate);@@ -538,9 +551,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 18 lines · Laurent Pinchart
Hi Saravana,
Thank you for the patch.
Quoted 7 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?
Quoted 6 lines
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?:-(
This is kinda expected :( because omap2 arch doesn't call of_platform_default_populate()
Call path:board-generic.c
DT_MACHINE_START()
.init_machine = omap_generic_init,
omap_generic_init()
pdata_quirks_init(omap_dt_match_table); of_platform_populate(NULL, omap_dt_match_table, omap_auxdata_lookup, NULL);
Other affected platforms
arm: mach-ux500
some mips
some powerpc
there are also case when a lot of devices placed under bus node, in such case
of_platform_populate() calls from bus drivers will also suffer from this issue.
I think one option could be to add some parameter to _populate() or introduce new api.
By the way, is there option to disable this feature at all?
Is there Kconfig option?
Is there any reasons why such complex and time consuming code added to the kernel and not implemented on DTC level?
Also, I've came with another diff, pls check.
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 5.9.0-rc6-01791-g9acba6b38757-dirty (grygorii@grygorii-XPS-13-9370) (arm-linux-gnueabihf-gcc (GNU Toolcha0 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt: Machine model: TI AM5718 IDK ... [ 0.053443] cpuidle: using governor ladder [ 0.053470] cpuidle: using governor menu [ 0.089304] No ATAGs? ... [ 3.092291] devtmpfs: mounted [ 3.095804] Freeing unused kernel memory: 1024K [ 3.100483] Run /sbin/init as init process
------ >< ---
drivers/of/platform.c +7 -2
@@ -514,6 +514,12 @@ static const struct of_device_id reserved_mem_matches[] = { {} }; +static int __init of_platform_fw_devlink_pause (void ) +{ +fw_devlink_pause (); +} +core_initcall (of_platform_fw_devlink_pause); +static int __init of_platform_default_populate_init (void ) {struct device_node *node;
@@ -538,9 +544,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }
@@ -548,6 +552,7 @@ arch_initcall_sync(of_platform_default_populate_init);static int __init of_platform_sync_state_init (void ) { +fw_devlink_resume ();device_links_supplier_sync_state_resume ();return 0 ; }
Signature · 2 lines
Best regards, grygorii
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 103 lines · Grygorii Strashko
Quoted 18 lines · Laurent Pinchart
Hi Saravana,
Thank you for the patch.
Quoted 7 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu
[ 12.349476] No ATAGs?Quoted 6 lines
After the patch:
[ 0.650460] cpuidle: using governor menu
[ 12.262101] No ATAGs?:-(
This is kinda expected :( because omap2 arch doesn't call
of_platform_default_populate()Call path:
board-generic.c
DT_MACHINE_START()
.init_machine =omap_generic_init,omap_generic_init()
pdata_quirks_init(omap_dt_match_table);of_platform_populate(NULL, omap_dt_match_table, omap_auxdata_lookup, NULL);Other affected platforms
arm: mach-ux500
some mips
some powerpcthere are also case when a lot of devices placed under bus node, in such case
of_platform_populate() calls from bus drivers will also suffer from this issue.I think one option could be to add some parameter to
_populate()or introduce new api.By the way, is there option to disable this feature at all?
Is thereKconfigoption?
Is there any reasons why such complex and time consuming code added to the kernel and not implemented on DTC level?Also, I've came with another diff, pls check.
[ 0.000000] Booting Linux on physical CPU
0x0
[ 0.000000] Linux version 5.9.0-rc6-01791-g9acba6b38757-dirty (grygorii@grygorii-XPS-13-9370) (arm-linux-gnueabihf-gcc (GNU Toolcha0 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] OF: fdt: Machine model: TI AM5718 IDK
...
[ 0.053443] cpuidle: using governor ladder
[ 0.053470] cpuidle: using governor menu
[ 0.089304] No ATAGs?
...
[ 3.092291] devtmpfs: mounted
[ 3.095804] Freeing unused kernel memory: 1024K
[ 3.100483] Run/sbin/initas init process------ >< ---
drivers/of/platform.c +6
-2@@ -514,6 +514,12 @@ static const struct of_device_id reserved_mem_matches[] = { {} }; +static int __init of_platform_fw_devlink_pause (void ) +{ + fw_devlink_pause(); +} +core_initcall (of_platform_fw_devlink_pause); + static int __init of_platform_default_populate_init(void) { struct device_node *node;@@ -538,9 +544,7 @@ static int __init of_platform_default_populate_init(void) } /* Populate everything else. */ - fw_devlink_pause(); of_platform_default_populate(NULL, NULL, NULL); - fw_devlink_resume(); return 0; }
drivers/of/platform.c arch_initcall_sync(of_platform_default_populate_init); · line 555
Quoted 4 lines
drivers/of/platform.c +1
-0@@ -548,6 +552,7 @@ arch_initcall_sync(of_platform_default_populate_init); static int __init of_platform_sync_state_init(void) { + fw_devlink_resume();
^ it seems has to be done earlier, like +static int __init of_platform_fw_devlink_resume(void) +{
+ fw_devlink_resume();
+ return 0;
+} +device_initcall_sync(of_platform_fw_devlink_resume);
Quoted 6 lines
drivers/of/platform.c +0
-0device_links_supplier_sync_state_resume(); return 0; }
Signature · 2 lines
Best regards, grygorii
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 113 lines · Grygorii Strashko' via kernel-team
Quoted 103 lines · Grygorii Strashko
Quoted 18 lines · Laurent Pinchart
Hi Saravana,
Thank you for the patch.
Quoted 7 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?Quoted 6 lines
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?:-(
This is kinda expected :( because omap2 arch doesn't call
of_platform_default_populate()Call path:
board-generic.c
DT_MACHINE_START()
.init_machine =omap_generic_init,omap_generic_init() pdata_quirks_init(omap_dt_match_table); of_platform_populate(NULL, omap_dt_match_table, omap_auxdata_lookup, NULL);Other affected platforms
arm: mach-ux500
some mips
some powerpcthere are also case when a lot of devices placed under bus node, in such case
of_platform_populate()calls from bus drivers will also suffer from this issue.I think one option could be to add some parameter to
_populate()or introduce new api.By the way, is there option to disable this feature at all?
Is thereKconfigoption?
Is there any reasons why such complex and time consuming code added to the kernel and not implemented on DTC level?Also, I've came with another diff, pls check.
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 5.9.0-rc6-01791-g9acba6b38757-dirty (grygorii@grygorii-XPS-13-9370) (arm-linux-gnueabihf-gcc (GNU Toolcha0 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt: Machine model: TI AM5718 IDK ... [ 0.053443] cpuidle: using governor ladder [ 0.053470] cpuidle: using governor menu [ 0.089304] No ATAGs? ... [ 3.092291] devtmpfs: mounted [ 3.095804] Freeing unused kernel memory: 1024K [ 3.100483] Run /sbin/init as init process------ >< ---
drivers/of/platform.c +6
-2@@ -514,6 +514,12 @@ static const struct of_device_id reserved_mem_matches[] = { {} }; +static int __init of_platform_fw_devlink_pause (void ) +{ +fw_devlink_pause (); +} +core_initcall (of_platform_fw_devlink_pause); +static int __init of_platform_default_populate_init (void ) {struct device_node *node;@@ -538,9 +544,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }drivers/of/platform.c arch_initcall_sync(of_platform_default_populate_init); · line 555
Quoted 4 lines
drivers/of/platform.c +1
-0@@ -548,6 +552,7 @@ arch_initcall_sync(of_platform_default_populate_init);static int __init of_platform_sync_state_init (void ) { +fw_devlink_resume ();^ it seems has to be done earlier, like
Review
Quoted 6 lines
(quoted diff) +6
-0+static int __init of_platform_fw_devlink_resume(void) +{ + fw_devlink_resume(); + return 0; +} +device_initcall_sync(of_platform_fw_devlink_resume);
This will mean no device will probe until device_initcall_sync(). Unfortunately, I don't think we can make such a sweeping assumption.
-Saravana
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 110 lines · Saravana Kannan
Quoted 108 lines · Grygorii Strashko' via kernel-team
Quoted 104 lines · Grygorii Strashko
Quoted 18 lines · Laurent Pinchart
Hi Saravana,
Thank you for the patch.
Quoted 7 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?Quoted 6 lines
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?:-(
This is kinda expected :( because omap2 arch doesn't call
of_platform_default_populate()Call path:
board-generic.c
DT_MACHINE_START()
.init_machine =omap_generic_init,omap_generic_init() pdata_quirks_init(omap_dt_match_table); of_platform_populate(NULL, omap_dt_match_table, omap_auxdata_lookup, NULL);Other affected platforms
arm: mach-ux500
some mips
some powerpcthere are also case when a lot of devices placed under bus node, in such case
of_platform_populate()calls from bus drivers will also suffer from this issue.I think one option could be to add some parameter to
_populate()or introduce new api.By the way, is there option to disable this feature at all?
Is thereKconfigoption?
Is there any reasons why such complex and time consuming code added to the kernel and not implemented on DTC level?Also, I've came with another diff, pls check.
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 5.9.0-rc6-01791-g9acba6b38757-dirty (grygorii@grygorii-XPS-13-9370) (arm-linux-gnueabihf-gcc (GNU Toolcha0 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt: Machine model: TI AM5718 IDK ... [ 0.053443] cpuidle: using governor ladder [ 0.053470] cpuidle: using governor menu [ 0.089304] No ATAGs? ... [ 3.092291] devtmpfs: mounted [ 3.095804] Freeing unused kernel memory: 1024K [ 3.100483] Run /sbin/init as init process------ >< ---
drivers/of/platform.c +6
-2@@ -514,6 +514,12 @@ static const struct of_device_id reserved_mem_matches[] = { {} }; +static int __init of_platform_fw_devlink_pause (void ) +{ +fw_devlink_pause (); +} +core_initcall (of_platform_fw_devlink_pause); +static int __init of_platform_default_populate_init (void ) {struct device_node *node;@@ -538,9 +544,7 @@ static int __init of_platform_default_populate_init(void) }/* Populate everything else. */ -fw_devlink_pause ();of_platform_default_populate (NULL ,NULL ,NULL ); -fw_devlink_resume ();return 0 ; }@@ -548,6 +552,7 @@ arch_initcall_sync(of_platform_default_populate_init);
Quoted 14 lines
Review
Quoted 11 lines
drivers/of/platform.c arch_initcall_sync(of_platform_default_populate_init); · line 555
Quoted 3 lines
drivers/of/platform.c +1
-0static int __init of_platform_sync_state_init (void ) { +fw_devlink_resume ();^ it seems has to be done earlier, like
(quoted diff) +6
-0+static int __init of_platform_fw_devlink_resume(void) +{ + fw_devlink_resume(); + return 0; +} +device_initcall_sync(of_platform_fw_devlink_resume);This will mean no device will probe until
device_initcall_sync(). Unfortunately, I don't think we can make such a sweeping assumption.
Could you answer below questions, pls?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
By the way, is there option to disable this feature at all? Is there
Kconfigoption?
Signature · 2 lines
Best regards, grygorii
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
In reply to Saravana KannanQuoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;
drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate() vs. of_platform_populate() is just a different match table. I don't think the behavior should otherwise be different.
There's also of_platform_probe() which has slightly different matching behavior. It should not behave differently either with respect to devlinks.
Rob
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 43 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.
Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.
So I'm trying to do this only when the top level devices are added for the first time. of_platform_default_populate() seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default to of_platform_populate().
Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
-Saravana
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Hi Saravana,
Quoted 54 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().
Quoted 2 lines
Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 58 lines · Laurent Pinchart
Hi Saravana,
Quoted 54 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().
Quoted 7 lines
Quoted 2 lines
Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
-Saravana
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Hi Saravana,
Quoted 59 lines · Saravana Kannan
Quoted 58 lines · Laurent Pinchart
Quoted 56 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Quoted 7 lines
Quoted 4 lines
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
I've replied to the patch:
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?
I'm using an AM57xx EVM, whose DT is not upstream, but it's essentially a am57xx-beagle-x15-revb1.dts (it includes that DTS) with a few additional nodes for GPIO keys, LCD panel, backlight and touchscreen.
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
hi Saravana,
Quoted 77 lines · Laurent Pinchart
Hi Saravana,
Quoted 59 lines · Saravana Kannan
Quoted 58 lines · Laurent Pinchart
Quoted 56 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Quoted 7 lines
Quoted 4 lines
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
I've replied to the patch:
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?
Quoted 9 lines
After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?I'm using an AM57xx EVM, whose DT is not upstream, but it's essentially a
am57xx-beagle-x15-revb1.dts(it includes that DTS) with a few additional nodes for GPIO keys, LCD panel, backlight and touchscreen.
hope you are receiving my mails as I've provided you with all required information already [1]
with below diff:
[ 4.177231] Freeing unused kernel memory: 1024K [ 4.181892] Run /sbin/init as init process
The best time with [2] is
[ 3.100483] Run /sbin/init as init process
Still 1 sec lose.
Pls understand an issue - requirements here are like 500ms boot with can, Ethernet, camera and display on ;(
[1] https://lore.kernel.org/patchwork/patch/1316134/#1511276
[2] https://lore.kernel.org/patchwork/patch/1316134/#1511435
arch/arm/mach-omap2/pdata-quirks.c +2 -0
@@ -591,7 +591,9 @@ void __init pdata_quirks_init(const struct of_device_id *omap_dt_match_table)if (of_machine_is_compatible ("ti,omap3" ))omap3_mcbsp_init ();pdata_quirks_check (auxdata_quirks); +fw_devlink_pause ();of_platform_populate (NULL , omap_dt_match_table, omap_auxdata_lookup,NULL ); +fw_devlink_resume ();pdata_quirks_check (pdata_quirks); }
Signature · 2 lines
Best regards, grygorii
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 86 lines · Grygorii Strashko' via kernel-team
hi Saravana,
Quoted 82 lines · Laurent Pinchart
Hi Saravana,
Quoted 59 lines · Saravana Kannan
Quoted 58 lines · Laurent Pinchart
Quoted 56 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Quoted 7 lines
Quoted 4 lines
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
I've replied to the patch:
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?
Quoted 6 lines
Quoted 4 lines
I'm using an AM57xx EVM, whose DT is not upstream, but it's essentially a
am57xx-beagle-x15-revb1.dts(it includes that DTS) with a few additional nodes for GPIO keys, LCD panel, backlight and touchscreen.hope you are receiving my mails as I've provided you with all required information already [1]
Laurent/Grygorii,
Looks like I'm definitely missing emails. Sorry about the confusion.
I have some other urgent things on my plate right now. Is it okay if I get to this in a day or two? In the end, we'll find a solution that addresses most/all of the delay.
Thanks,
Saravana
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Hi Saravana,
Quoted 89 lines · Saravana Kannan
Quoted 81 lines · Grygorii Strashko' via kernel-team
Quoted 80 lines · Laurent Pinchart
Quoted 59 lines · Saravana Kannan
Quoted 58 lines · Laurent Pinchart
Quoted 56 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Quoted 7 lines
Quoted 4 lines
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
I've replied to the patch:
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?Quoted 6 lines
Quoted 4 lines
I'm using an AM57xx EVM, whose DT is not upstream, but it's essentially a
am57xx-beagle-x15-revb1.dts(it includes that DTS) with a few additional nodes for GPIO keys, LCD panel, backlight and touchscreen.hope you are receiving my mails as I've provided you with all required information already [1]
Quoted 7 lines
Laurent/Grygorii,
Looks like I'm definitely missing emails. Sorry about the confusion.
I have some other urgent things on my plate right now. Is it okay if I get to this in a day or two? In the end, we'll find a solution that addresses most/all of the delay.
No issue on my side.
By the way, during initial investigations, I've traced code paths to figure out if there was a particular step that would consume a large amount of time, and found out that of_platform_populate() ends up executing devlink-related code that seems to have an O(n^3) complexity on the number of devices, with a few dozens of milliseconds for each iteration. That's a very bad complexity.
Signature · 3 lines
Regards, Laurent Pinchart
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 95 lines · Laurent Pinchart
Hi Saravana,
Quoted 91 lines · Saravana Kannan
Quoted 81 lines · Grygorii Strashko' via kernel-team
Quoted 80 lines · Laurent Pinchart
Quoted 59 lines · Saravana Kannan
Quoted 58 lines · Laurent Pinchart
Quoted 56 lines · Saravana Kannan
Quoted 42 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time. As I said in the other email, we can't add fw_devlink_pause/resume() by default toof_platform_populate().Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Quoted 7 lines
Quoted 4 lines
I'm not an expert in this domain, but before investigating it, would you be able to share a hack patch that implements this (in the most simple way) to check if it actually fixes the delays I experience on my system ?
So I take it the patch I sent out didn't work for you? Can you tell me what machine/DT you are using?
I've replied to the patch:
Based on v5.9-rc5, before the patch:
[ 0.652887] cpuidle: using governor menu [ 12.349476] No ATAGs?After the patch:
[ 0.650460] cpuidle: using governor menu [ 12.262101] No ATAGs?Quoted 6 lines
Quoted 4 lines
I'm using an AM57xx EVM, whose DT is not upstream, but it's essentially a
am57xx-beagle-x15-revb1.dts(it includes that DTS) with a few additional nodes for GPIO keys, LCD panel, backlight and touchscreen.hope you are receiving my mails as I've provided you with all required information already [1]
Laurent/Grygorii,
Quoted 7 lines
Quoted 5 lines
Looks like I'm definitely missing emails. Sorry about the confusion.
I have some other urgent things on my plate right now. Is it okay if I get to this in a day or two? In the end, we'll find a solution that addresses most/all of the delay.
No issue on my side.
Hi Laurent,
Sorry it took awhile for me to get back to this.
Can you try throwing around fw_devlink_pause/resume() around the of_platform_populate() call in arch/arm/mach-omap2/pdata-quirks.c? Just trying to verify the cause/fix.
If it fixes the issue, then considering Rob's comments [1], a good short term solution might be to have the suggestion above and some way to do pause/resume only when the top level devices are added.
Quoted 6 lines
By the way, during initial investigations, I've traced code paths to figure out if there was a particular step that would consume a large amount of time, and found out that
of_platform_populate()ends up executing devlink-related code that seems to have an O(n^3) complexity on the number of devices, with a few dozens of milliseconds for each iteration. That's a very bad complexity.
As you said, the complexity of fw_devlink parsing can be O(N^2). There are other ways to improve it to make it O(N) but it has a bunch of additional complexity and memory increase. When I tried to do it that way the first time, I was question whether O(N^2) actually translated to measurable difference. Looks like we do now :) I have something in mind for how to do it with O(N) complexity, but I expect it to take a while to get in. So in the meantime, I'm thinking of using fw_devlink_pause/resume() as a short term optimization.
-Saravana
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
Quoted 3 lines · Saravana Kannan
Can you try throwing around fw_devlink_pause/resume() around the
of_platform_populate()call inarch/arm/mach-omap2/pdata-quirks.c? Just trying to verify the cause/fix.
AM5 EVM on v5.10-rc1:
[ 1.139945] cpuidle: using governor menu [ 13.126461] No ATAGs?
After adding fw_devlink_pause/resume around of_platform_populate:
[ 1.139587] cpuidle: using governor menu [ 1.899913] No ATAGs?
Tomi
Signature · 2 lines
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
Re: [PATCH v1] of: platform: Batch fwnode parsing in the init_machine() path
In reply to Saravana KannanQuoted 49 lines · Saravana Kannan
Quoted 43 lines · Rob Herring
Quoted 26 lines · Saravana Kannan
When commit
93d2e4322aa7("of: platform: Batch fwnode parsing when adding all top level devices") optimized the fwnode parsing when all top level devices are added, it missed out optimizing this for platform where the top level devices are added through theinit_machine()path.This commit does the optimization for all paths by simply moving the fw_devlink_pause/resume() inside
of_platform_default_populate().Reported-by Tomi Valkeinen <[email protected]>Signed-off-by Saravana Kannan <[email protected]>--- drivers/of/platform.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)drivers/of/platform.c +1
-2@@ -501,8 +501,21 @@ int of_platform_default_populate(struct device_node *root,const struct of_dev_auxdata *lookup, struct device *parent) { - return of_platform_populate(root, of_default_bus_match_table, lookup, -parent ); + int ret;drivers/of/platform.c int of_platform_default_populate(struct device_node *root, · line 514
Quoted 10 lines
drivers/of/platform.c +10
-0+ +/* +* fw_devlink_pause/resume() are only safe to be called around top +* level device addition due to locking constraints. +*/ +if (!root) +fw_devlink_pause (); + + ret =of_platform_populate (root, of_default_bus_match_table, lookup, + parent);
of_platform_default_populate()vs.of_platform_populate()is just a different match table. I don't think the behavior should otherwise be different.Quoted 3 lines
There's also
of_platform_probe()which has slightly different matching behavior. It should not behave differently either with respect to devlinks.
Quoted 8 lines
So I'm trying to do this only when the top level devices are added for the first time.
of_platform_default_populate()seems to be the most common path. For other cases, I think we just need to call fw_devlink_pause/resume() wherever the top level devices are added for the first time.As I said in the other email, we can't add fw_devlink_pause/resume() by default to
of_platform_populate().
If you detect it's the first time, you could?
Quoted 3 lines
Do you have other ideas for achieving "call fw_devlink_pause/resume() only when top level devices are added for the first time"?
Eliminate the cases not using of_platform_default_populate().
There's 2 main reasons for the non default cases. The first is auxdata. Really, for any modern platform that people care about (and care about the boot time), they should not be using auxdata. That's just for the DT transition. You know, a temporary thing from 9 years ago.
The 2nd is having some parent device. This is typically an soc_device. I really think this is kind of dumb. We should either have the parent device always or never. After all, everything's an SoC right? Of course changing that will break some Android systems since they like to use non-ABI sysfs device paths.
There could also be some initcall ordering issues. IIRC, in the last round of cleanups in this area, at91 gpio/pinctrl had an issue with that. I think I have a half done fix for that I started.
Rob