Do not gate ge0/1 and runit clocks on Kirkwood
Subsystems (6)
v2 cover Do not gate ge0/1 and runit clocks on Kirkwood
Hi,
kernel 3.8-rc5 will hang on kirkwood DT if the Ethernet driver is built as a module or when no driver claiming the runit clock is built in. (Usually, at least the serial driver is built in, but it won't request the clock if "clock-frequency" is given in DT.) In the past, we fixed this by keeping the clocks ticking.
These two patches re-enable the ethernet fix for DT and declare the "runit" clock as not to gated even if unused.
Changes:
v2:
- Do not turn on ge[01] clocks unconditionally. Instead, fix the previous way we handled this on kirkwood.
Simon Baatz (2):
ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
clk: mvebu: Do not gate runit clock on Kirkwood
arch/arm/mach-kirkwood/common.c | 13 +++++++++---- drivers/clk/mvebu/clk-gating-ctrl.c | 5 ++++- 2 files changed, 13 insertions(+), 5 deletions(-)
Signature · 1 line
1.7.9.5
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Commit 1611f87 (ARM: Kirkwood: switch to DT clock providers) broke the functions to initialize the ethernet interfaces (kirkwood_ge00_init() and kirkwood_ge01_init()). In the DT case, the functions could not enable the correct clocks.
Fix this by looking up the clocks through the device name.
--- arch/arm/mach-kirkwood/common.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
arch/arm/mach-kirkwood/common.c +9 -4
@@ -218,11 +218,10 @@ static struct clk __init *kirkwood_register_gate_fn(const char *name, bit_idx, 0, &gating_lock, fn_en, fn_dis); } -static struct clk *ge0, *ge1;void __init kirkwood_clk_init (void ) { -struct clk *runit, *sata0, *sata1, *usb0, *sdio; +struct clk *runit, *ge0, *ge1, *sata0, *sata1, *usb0, *sdio;struct clk *crypto, *xor0, *xor1, *pex0, *pex1, *audio; tclk = clk_register_fixed_rate(NULL, "tclk", NULL,
@@ -288,12 +287,15 @@ void __init kirkwood_ehci_init(void)****************************************************************************/ void __init kirkwood_ge00_init (struct mv643xx_eth_platform_data *eth_data) { +struct clk *ge0;orion_ge00_init (eth_data,GE00_PHYS_BASE ,IRQ_KIRKWOOD_GE00_SUM ,IRQ_KIRKWOOD_GE00_ERR ,1600 );/* The interface forgets the MAC address assigned by u-boot if the clock is turned off, so claim the clk now. */ -clk_prepare_enable (ge0); + ge0 =clk_get_sys (MV643XX_ETH_NAME ".0" ,NULL ); +if (!IS_ERR (ge0)) +clk_prepare_enable (ge0); }
@@ -302,10 +304,13 @@ void __init kirkwood_ge00_init(struct mv643xx_eth_platform_data *eth_data)****************************************************************************/ void __init kirkwood_ge01_init (struct mv643xx_eth_platform_data *eth_data) { +struct clk *ge1;orion_ge01_init (eth_data,GE01_PHYS_BASE ,IRQ_KIRKWOOD_GE01_SUM ,IRQ_KIRKWOOD_GE01_ERR ,1600 ); -clk_prepare_enable (ge1); + ge1 =clk_get_sys (MV643XX_ETH_NAME ".1" ,NULL ); +if (!IS_ERR (ge1)) +clk_prepare_enable (ge1); }
Signature · 1 line
1.7.9.5
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 23 lines · Simon Baatz
Commit 1611f87 (ARM: Kirkwood: switch to DT clock providers) broke the functions to initialize the ethernet interfaces (
kirkwood_ge00_init()andkirkwood_ge01_init()). In the DT case, the functions could not enable the correct clocks.Fix this by looking up the clocks through the device name.
Signed-off-by Simon Baatz<[email protected]>--- arch/arm/mach-kirkwood/common.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)arch/arm/mach-kirkwood/common.c +1
-0tclk = clk_register_fixed_rate(NULL, "tclk", NULL,@@ -288,12 +287,15 @@ void __init kirkwood_ehci_init(void)****************************************************************************/ void __init kirkwood_ge00_init (struct mv643xx_eth_platform_data *eth_data) { +struct clk *ge0;
arch/arm/mach-kirkwood/common.c void __init kirkwood_ehci_init(void) · line 299
Quoted 10 lines
arch/arm/mach-kirkwood/common.c +3
-1orion_ge00_init (eth_data,GE00_PHYS_BASE ,IRQ_KIRKWOOD_GE00_SUM ,IRQ_KIRKWOOD_GE00_ERR ,1600 );/* The interface forgets the MAC address assigned by u-boot if the clock is turned off, so claim the clk now. */ -clk_prepare_enable (ge0); + ge0 =clk_get_sys (MV643XX_ETH_NAME ".0" ,NULL ); +if (!IS_ERR (ge0)) +clk_prepare_enable (ge0); }
Simon,
Jason posted a patch set that makes mv643xx DT compatible. IMHO this patch is obsolete when we have DT support for mv643xx. kirkwood_ge00/01_init will not be called with DT support at all.
I agree that loosing the MAC address _is_ an issue but there must be another way to retain it during gated ge clocks than not gate the clocks at all.
I can think of some ways to retain it but don't know what is the most common with linux:
- make u-boot pass it through cmdline and let mv643xx get it from there
- have kirkwood's common code grab it before clk gates kick in
I will do some tests on dove if it also suffers from loosing it's MAC on gated clock.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Hi Sebastian,
Quoted 3 lines · Sebastian Hesselbarth
Jason posted a patch set that makes mv643xx DT compatible. IMHO this patch is obsolete when we have DT support for mv643xx. kirkwood_ge00/01_init will not be called with DT support at all.
I know. This is to fix a regression in 3.8 (which currently does not boot with my "modularized" config).
As said in my other mail to Jason, we will need something more clever in 3.9.
- Simon
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 7 lines · Simon Baatz
Quoted 3 lines · Sebastian Hesselbarth
Jason posted a patch set that makes mv643xx DT compatible. IMHO this patch is obsolete when we have DT support for mv643xx. kirkwood_ge00/01_init will not be called with DT support at all.
I know. This is to fix a regression in 3.8 (which currently does not boot with my "modularized" config).
Well, patching kirkwood_ge00/01_init() will also affect non-DT code so there is a great potential to break something else.
If it is a DT-only issue please move the "always enable ge clocks" fix to kirkwood_clk_legacy_init() as it only called by DT enabled boards.
If there is no other issue than keeping the clocks running because of not loosing the MAC address, I prefer not to fix it at all.
Looks like modular gbe on kirkwood is just unsupported on 3.8.
Sebastian
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Simon BaatzQuoted 9 lines · Simon Baatz
Hi Sebastian,
Quoted 3 lines · Sebastian Hesselbarth
Jason posted a patch set that makes mv643xx DT compatible. IMHO this patch is obsolete when we have DT support for mv643xx. kirkwood_ge00/01_init will not be called with DT support at all.
I know. This is to fix a regression in 3.8 (which currently does not boot with my "modularized" config).
Simon,
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing mv643xx_eth does work just fine here on Dove.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 11 lines · Sebastian Hesselbarth
Quoted 9 lines · Simon Baatz
Hi Sebastian,
Quoted 3 lines · Sebastian Hesselbarth
Jason posted a patch set that makes mv643xx DT compatible. IMHO this patch is obsolete when we have DT support for mv643xx. kirkwood_ge00/01_init will not be called with DT support at all.
I know. This is to fix a regression in 3.8 (which currently does not boot with my "modularized" config).
Quoted 6 lines
Simon,
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.
I believe Simon's issue is that the mv643xx_eth driver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.
thx,
Jason.
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 9 lines · Jason Cooper
Quoted 4 lines
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.I believe Simon's issue is that the
mv643xx_ethdriver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.
Loading and unloading the driver module hangs because of the missing clk_prepeare_enable in shared driver part. This should be fixed by the patch I sent you.
Dove and Kirkwood have the same gbe internally and I can boot into Dove without mv643xx_eth, load, unload, reload the module and it always finds its MAC address.
I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 15 lines · Sebastian Hesselbarth
Quoted 9 lines · Jason Cooper
Quoted 4 lines
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.I believe Simon's issue is that the
mv643xx_ethdriver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.Loading and unloading the driver module hangs because of the missing
clk_prepeare_enablein shared driver part. This should be fixed by the patch I sent you.
Quoted 7 lines
Dove and Kirkwood have the same gbe internally and I can boot into Dove without
mv643xx_eth, load, unload, reload the module and it always finds its MAC address.I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Ok, I just wanted to make sure we understood his problem correctly, and if possible, reproduce it.
Simon, can you give us some steps to reproduce this on our side so we can see exactly what's happening?
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Hi Jason,
Quoted 24 lines · Jason Cooper
Quoted 15 lines · Sebastian Hesselbarth
Quoted 9 lines · Jason Cooper
Quoted 4 lines
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.I believe Simon's issue is that the
mv643xx_ethdriver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.Loading and unloading the driver module hangs because of the missing
clk_prepeare_enablein shared driver part. This should be fixed by the patch I sent you.Quoted 7 lines
Dove and Kirkwood have the same gbe internally and I can boot into Dove without
mv643xx_eth, load, unload, reload the module and it always finds its MAC address.I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Quoted 5 lines
Ok, I just wanted to make sure we understood his problem correctly, and if possible, reproduce it.
Simon, can you give us some steps to reproduce this on our side so we can see exactly what's happening?
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers, mv643xx_eth is build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading mv643xx_eth)
non-DT: Boots ok
3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots ok
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots ok
3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
DT: Boots, but no MAC
non-DT: Boots, but no MAC
hth,
Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 30 lines · Simon Baatz
Hi Jason,
Quoted 27 lines · Jason Cooper
Quoted 15 lines · Sebastian Hesselbarth
Quoted 9 lines · Jason Cooper
Quoted 4 lines
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.I believe Simon's issue is that the
mv643xx_ethdriver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.Loading and unloading the driver module hangs because of the missing
clk_prepeare_enablein shared driver part. This should be fixed by the patch I sent you.Quoted 7 lines
Dove and Kirkwood have the same gbe internally and I can boot into Dove without
mv643xx_eth, load, unload, reload the module and it always finds its MAC address.I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Ok, I just wanted to make sure we understood his problem correctly, and if possible, reproduce it.
Quoted 8 lines
Quoted 2 lines
Simon, can you give us some steps to reproduce this on our side so we can see exactly what's happening?
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers,
mv643xx_ethis build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.
Let's back up. If you config early printk and COMMON_CLK_DEBUG and a vanilla v3.8-rc5 with your current config, what happens?
Could you also please try kirkwood_defconfig and tell us if it boots?
thx,
Jason.
Quoted 32 lines
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading
mv643xx_eth)
non-DT: Boots ok3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots ok3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots ok3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
DT: Boots, but no MAC
non-DT: Boots, but no MAChth,
Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 34 lines · Jason Cooper
Quoted 33 lines · Simon Baatz
Hi Jason,
Quoted 29 lines · Jason Cooper
Quoted 15 lines · Sebastian Hesselbarth
Quoted 9 lines · Jason Cooper
Quoted 4 lines
I _cannot_ confirm that gbe is loosing its MAC address on Dove. I will post a follow-up patch to Jason's cleanup patches that will also grab a clock for smi. With that patch insmod'ing/rmmod'ing
mv643xx_ethdoes work just fine here on Dove.I believe Simon's issue is that the
mv643xx_ethdriver is not loaded at boot, it's clocks get gated, then when he loads the driver, there is no mac address. Is that correct Simon? I don't think unloading the driver after boot will trigger this regression.Loading and unloading the driver module hangs because of the missing
clk_prepeare_enablein shared driver part. This should be fixed by the patch I sent you.Quoted 7 lines
Dove and Kirkwood have the same gbe internally and I can boot into Dove without
mv643xx_eth, load, unload, reload the module and it always finds its MAC address.I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Ok, I just wanted to make sure we understood his problem correctly, and if possible, reproduce it.
Simon, can you give us some steps to reproduce this on our side so we can see exactly what's happening?
Quoted 8 lines
Quoted 5 lines
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers,
mv643xx_ethis build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.Let's back up. If you config early printk and
COMMON_CLK_DEBUGand a vanilla v3.8-rc5 with your current config, what happens?
I thought it's clear what happens. "runit" is requested in the dts by serial, spi, watchdog, nand, i2c. Each of these are either not built or built as a module in my config, except serial.
of_serial.c does the following in of_platform_serial_setup():
if (of_property_read_u32(np, "clock-frequency", &clk)) {
/* Get clk rate through clk driver if present */
info->clk = clk_get(&ofdev->dev, NULL);
if (IS_ERR(info->clk)) {
dev_warn(&ofdev->dev,
"clk or clock-frequency not defined\n");
return PTR_ERR(info->clk);
}
clk_prepare_enable(info->clk);
clk = clk_get_rate(info->clk);
}and since "clock-frequency" is still given in the standard DTS for the IB-NAS 6210, it does not enable the clock.
Thus, no driver uses the clock, it gets disabled and the system locks up.
The system boots when removing "clock-frequency" from the DTS.
Which lead to two questions:
- Is it safe to remove "clock-frequency" now that we have the clocks in place?
- The behavior of of_serial.c looks strange to me, is this really the desired behavior?
For the "runit" clock, this means that any time you hit a configuration that does not keep the clock enabled, the system will lockup. (We have gone through this more than once now. Especially, this hits you hard when you try to support a new board and disable as much as possible in the config/dts for a start...).
Thus, we should keep it enabled even if it is unused, which is exactly the purpose of CLK_IGNORE_UNUSED if I understood this correctly.
Quoted 1 line
Could you also please try
kirkwood_defconfigand tell us if it boots?
It is very easy to get my system to boot with a stock kernel. Just build one of the drivers I use directly into the kernel and it boots (up to the point where the gbe driver is loaded if built as a module, of course).
Quoted 8 lines
Quoted 8 lines
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading
mv643xx_eth)
non-DT: Boots ok
In the DT case ge0 and ge1 are NULL, and thus, kirkwood_ge0[01]_init() tries to enable NULL clocks, but not the gbe clocks. -> The clocks get disabled.
As described in Sebastian's patch (and also found by Andrew some time ago), this leads to a hanging system when loading the gbe driver.
Quoted 4 lines
Quoted 4 lines
3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots ok
Since the clocks are found now and kept enabled, DT behaves the same as non-DT.
Quoted 6 lines
Quoted 6 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots ok
This is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
Quoted 6 lines
Quoted 6 lines
3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
DT: Boots, but no MAC
non-DT: Boots, but no MAC
I think non-DT and DT gated clocks behave differently, since the non-DT ones also enable/disable the PHY. I explicitly removed the clk_prepare_enable() from kirkwood_ge0[01]_init() in order to see if this makes a difference.
(Which reminds me, that we wanted to implement the PHY enabling in the driver.)
Apparently, it does not make a difference.
- Simon
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 8 lines · Simon Baatz
Quoted 7 lines · Jason Cooper
Quoted 5 lines · Simon Baatz
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers,
mv643xx_ethis build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.
Quoted 6 lines
Quoted 2 lines
Let's back up. If you config early printk and
COMMON_CLK_DEBUGand a vanilla v3.8-rc5 with your current config, what happens?I thought it's clear what happens. "runit" is requested in the dts by serial, spi, watchdog, nand, i2c. Each of these are either not built or built as a module in my config, except serial.
Simon,
it is clear what happens but just to blindly disable clock gating will not help here. What you are suffering are several issues not only one.
Quoted 22 lines
of_serial.cdoes the following inof_platform_serial_setup():if (of_property_read_u32(np, "clock-frequency",&clk)) { /* Get clk rate through clk driver if present */ info->clk = clk_get(&ofdev->dev, NULL); if (IS_ERR(info->clk)) { dev_warn(&ofdev->dev, "clk or clock-frequency not defined\n"); return PTR_ERR(info->clk); } clk_prepare_enable(info->clk); clk = clk_get_rate(info->clk); }and since "clock-frequency" is still given in the standard DTS for the IB-NAS 6210, it does not enable the clock.
Thus, no driver uses the clock, it gets disabled and the system locks up.
The system boots when removing "clock-frequency" from the DTS.
Quoted 3 lines
Which lead to two questions:
- Is it safe to remove "clock-frequency" now that we have the clocks in place?
Safe, as long as you replace it with the corresponding clocks=<&..> property.
Quoted 2 lines
- The behavior of
of_serial.clooks strange to me, is this really the desired behavior?
I guess it is, rely on clock-frequency because a lot of boards still use it. There was no clocks property in of_serial when we started with kirkwood and dove. I think we can switch now to clocks property which will also remove all hard coded occurrences of tclk frequency.
Quoted 5 lines
For the "runit" clock, this means that any time you hit a configuration that does not keep the clock enabled, the system will lockup. (We have gone through this more than once now. Especially, this hits you hard when you try to support a new board and disable as much as possible in the config/dts for a start...).
Well, if there is no device/driver using runit it should be safe to disable it. If there is a driver using it, we need a clocks property for it. And as you already stated, serial is using it. And you want serial in every minimal kernel, don't you?
Quoted 3 lines
Thus, we should keep it enabled even if it is unused, which is exactly the purpose of
CLK_IGNORE_UNUSEDif I understood this correctly.
True, but only if it is that important that it should _never_ be gated. As long as it is 'only' used by peripheral devices, it should be safe to gate it.
Quoted 16 lines
Quoted 1 line
Could you also please try
kirkwood_defconfigand tell us if it boots?It is very easy to get my system to boot with a stock kernel. Just build one of the drivers I use directly into the kernel and it boots (up to the point where the gbe driver is loaded if built as a module, of course).
Quoted 8 lines
Quoted 8 lines
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading
mv643xx_eth)
non-DT: Boots ok
Quoted 6 lines
In the DT case ge0 and ge1 are NULL, and thus, kirkwood_ge0[01]_init() tries to enable NULL clocks, but not the gbe clocks. -> The clocks get disabled.
As described in Sebastian's patch (and also found by Andrew some time ago), this leads to a hanging system when loading the gbe driver.
If I got all your configs right, this DT case is with serial but no clocks property? All other runit "users" are _not_ built into the kernel? Maybe it is just a coincidence that it fails when loading eth but I guess it hangs because of runit getting gated while serial accessing it. When you replace serial's clock-frequency with clocks property to "runit" it shouldn't hang.
(Issue 1: gated runit clock hangs kernel due to serial access)
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots okSince the clocks are found now and kept enabled, DT behaves the same as non-DT.
Quoted 1 line
Quoted 1 line
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots okThis is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
Quoted 6 lines
Quoted 6 lines
3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
DT: Boots, but no MAC
non-DT: Boots, but no MAC
Ok, here my patch for smi clock enables gbe clock before accessing gbe registers.
(Issue 2: gated gbe clock hangs kernel due to smi access)
Quoted 4 lines
I think non-DT and DT gated clocks behave differently, since the non-DT ones also enable/disable the PHY. I explicitly removed the
clk_prepare_enable()from kirkwood_ge0[01]_init() in order to see if this makes a difference.
True, DT gated clocks don't disable PHYs (and never will).
Quoted 4 lines
(Which reminds me, that we wanted to implement the PHY enabling in the driver.)
Apparently, it does not make a difference.
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 12 lines · Sebastian Hesselbarth
Quoted 11 lines · Simon Baatz
Quoted 9 lines · Jason Cooper
Quoted 5 lines · Simon Baatz
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers,
mv643xx_ethis build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.Let's back up. If you config early printk and
COMMON_CLK_DEBUGand a vanilla v3.8-rc5 with your current config, what happens?
Quoted 8 lines
Quoted 3 lines
I thought it's clear what happens. "runit" is requested in the dts by serial, spi, watchdog, nand, i2c. Each of these are either not built or built as a module in my config, except serial.
Simon,
it is clear what happens but just to blindly disable clock gating will not help here. What you are suffering are several issues not only one.
Agreed.
Quoted 22 lines
Quoted 22 lines
of_serial.cdoes the following inof_platform_serial_setup():if (of_property_read_u32(np, "clock-frequency",&clk)) { /* Get clk rate through clk driver if present */ info->clk = clk_get(&ofdev->dev, NULL); if (IS_ERR(info->clk)) { dev_warn(&ofdev->dev, "clk or clock-frequency not defined\n"); return PTR_ERR(info->clk); } clk_prepare_enable(info->clk); clk = clk_get_rate(info->clk); }and since "clock-frequency" is still given in the standard DTS for the IB-NAS 6210, it does not enable the clock.
Thus, no driver uses the clock, it gets disabled and the system locks up.
The system boots when removing "clock-frequency" from the DTS.
Quoted 5 lines
Quoted 3 lines
Which lead to two questions:
- Is it safe to remove "clock-frequency" now that we have the clocks in place?Safe, as long as you replace it with the corresponding clocks=<&..> property.
For serial, this is in kirkwood.dtsi, so we are fine there.
Quoted 50 lines
Quoted 2 lines
- The behavior of
of_serial.clooks strange to me, is this really the desired behavior?I guess it is, rely on clock-frequency because a lot of boards still use it. There was no clocks property in
of_serialwhen we started with kirkwood and dove. I think we can switch now to clocks property which will also remove all hard coded occurrences of tclk frequency.Quoted 5 lines
For the "runit" clock, this means that any time you hit a configuration that does not keep the clock enabled, the system will lockup. (We have gone through this more than once now. Especially, this hits you hard when you try to support a new board and disable as much as possible in the config/dts for a start...).
Well, if there is no device/driver using runit it should be safe to disable it. If there is a driver using it, we need a clocks property for it. And as you already stated, serial is using it. And you want serial in every minimal kernel, don't you?
Quoted 3 lines
Thus, we should keep it enabled even if it is unused, which is exactly the purpose of
CLK_IGNORE_UNUSEDif I understood this correctly.True, but only if it is that important that it should _never_ be gated. As long as it is 'only' used by peripheral devices, it should be safe to gate it.
Quoted 16 lines
Quoted 1 line
Could you also please try
kirkwood_defconfigand tell us if it boots?It is very easy to get my system to boot with a stock kernel. Just build one of the drivers I use directly into the kernel and it boots (up to the point where the gbe driver is loaded if built as a module, of course).
Quoted 8 lines
Quoted 8 lines
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading
mv643xx_eth)
non-DT: Boots okQuoted 6 lines
In the DT case ge0 and ge1 are NULL, and thus, kirkwood_ge0[01]_init() tries to enable NULL clocks, but not the gbe clocks. -> The clocks get disabled.
As described in Sebastian's patch (and also found by Andrew some time ago), this leads to a hanging system when loading the gbe driver.
Quoted 8 lines
If I got all your configs right, this DT case is with serial but no clocks property? All other runit "users" are _not_ built into the kernel? Maybe it is just a coincidence that it fails when loading eth but I guess it hangs because of runit getting gated while serial accessing it. When you replace serial's clock-frequency with clocks property to "runit" it shouldn't hang.
(Issue 1: gated runit clock hangs kernel due to serial access)
This can be fixed with a patch removing clock-frequency from all kirkwood boards (dove, orion, etc?) I'll gin this up and add it to the havoc ;-)
Quoted 22 lines
Quoted 22 lines
Quoted 4 lines
Quoted 4 lines
3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots okSince the clocks are found now and kept enabled, DT behaves the same as non-DT.
Quoted 6 lines
Quoted 6 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots okThis is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
Quoted 4 lines
Quoted 4 lines
3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
DT: Boots, but no MAC
non-DT: Boots, but no MACOk, here my patch for smi clock enables gbe clock before accessing gbe registers.
(Issue 2: gated gbe clock hangs kernel due to smi access)
Here, I'm going to wait for Florian's mvmdio changes to settle out and then readdress this. My current understanding is that there will only be one DT node for this. iiuc, each board dts will have to say which gate clocks to consume to prevent the mvmdio node in the dtsi from consuming both unconditionally.
Quoted 7 lines
Quoted 4 lines
I think non-DT and DT gated clocks behave differently, since the non-DT ones also enable/disable the PHY. I explicitly removed the
clk_prepare_enable()from kirkwood_ge0[01]_init() in order to see if this makes a difference.True, DT gated clocks don't disable PHYs (and never will).
Quoted 8 lines
Quoted 4 lines
(Which reminds me, that we wanted to implement the PHY enabling in the driver.)
Apparently, it does not make a difference.
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
A patch to mv643xx_eth to pull this from DT should solve this.
thx,
Jason.
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 6 lines · Jason Cooper
Quoted 1 line · Sebastian Hesselbarth
(Issue 1: gated runit clock hangs kernel due to serial access)
This can be fixed with a patch removing clock-frequency from all kirkwood boards (dove, orion, etc?) I'll gin this up and add it to the havoc ;-)
For kirkwood.dtsi it is already there as you stated, but yes, it should be removed from the board files including kirkwood.dtsi and also the comments in kirkwood.dtsi. I'll prepare a patch for dove.dtsi.
Quoted 7 lines
Quoted 1 line
(Issue 2: gated gbe clock hangs kernel due to smi access)
Here, I'm going to wait for Florian's mvmdio changes to settle out and then readdress this. My current understanding is that there will only be one DT node for this. iiuc, each board dts will have to say which gate clocks to consume to prevent the mvmdio node in the dtsi from consuming both unconditionally.
Ok, feel free to consume the patch ;)
Quoted 5 lines
Quoted 3 lines
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
A patch to
mv643xx_ethto pull this from DT should solve this.
Great.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 9 lines · Sebastian Hesselbarth
Quoted 6 lines · Jason Cooper
Quoted 3 lines · Sebastian Hesselbarth
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
A patch to
mv643xx_ethto pull this from DT should solve this.Great.
Did I just volunteer myself for that too? :-)
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 7 lines · Jason Cooper
Quoted 6 lines · Sebastian Hesselbarth
Quoted 3 lines
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
A patch to
mv643xx_ethto pull this from DT should solve this.
Somewhere, Jason Gunthorpe shared his patch to do this. I'll poke around for it and try to get it merged in.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 10 lines · Jason Cooper
Quoted 7 lines · Jason Cooper
Quoted 6 lines · Sebastian Hesselbarth
Quoted 3 lines
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
A patch to
mv643xx_ethto pull this from DT should solve this.Somewhere, Jason Gunthorpe shared his patch to do this. I'll poke around for it and try to get it merged in.
Yes, you asked for the doc update and I haven't had a moment to get a tree setup for that..
Here are some words though:
- local-mac-address : Optional, the MAC address to assign to the device. If not specified then the MAC address in the HW registers is used, but the driver can not be made modular.
drivers/net/ethernet/marvell/mv643xx_eth.c +6 -0
@@ -2891,6 +2891,8 @@ static int mv643xx_eth_probe(struct platform_device *pdev)struct mv643xx_eth_private *mp;struct net_device *dev;struct resource *res; +const u8 *mac; +int len;int err;if (pdev->dev.of_node) {
@@ -2912,6 +2914,10 @@ static int mv643xx_eth_probe(struct platform_device *pdev)else pd->phy_addr =MV643XX_ETH_PHY_ADDR_DEFAULT ; + mac =of_get_property (pdev->dev.of_node,"local-mac-address" , &len); +if (mac && len ==6 ) +memcpy (pd->mac_addr, mac,sizeof pd->mac_addr); + np =of_parse_phandle (pdev->dev.of_node,"mdio" ,0 );if (np) { pd->shared =of_find_device_by_node (np);
Signature · 1 line
1.7.5.4
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 7 lines · Jason Gunthorpe
Quoted 6 lines · Jason Cooper
Quoted 6 lines · Jason Cooper
Quoted 5 lines · Sebastian Hesselbarth
Quoted 3 lines
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
A patch to
mv643xx_ethto pull this from DT should solve this.Somewhere, Jason Gunthorpe shared his patch to do this. I'll poke around for it and try to get it merged in.
Yes, you asked for the doc update and I haven't had a moment to get a tree setup for that..
Ahh, that's right. Sorry, wasn't trying to put pressure on you. Just trying to make a reminder to myself of what to look for when I get to around to it. (I have a separate IMAP folder for such things ;-) )
Quoted 5 lines
Here are some words though:
- local-mac-address : Optional, the MAC address to assign to the device. If not specified then the MAC address in the HW registers is used, but the driver can not be made modular.
I like it. If I haven't seen anything when I get to this, I'll add this to your patch. Please Ack if it's ok. I prefer not to change peoples patches without approval.
thx,
Jason.
Quoted 28 lines
drivers/net/ethernet/marvell/mv643xx_eth.c +6
-0@@ -2891,6 +2891,8 @@ static int mv643xx_eth_probe(struct platform_device *pdev)struct mv643xx_eth_private *mp;struct net_device *dev;struct resource *res; +const u8 *mac; +int len;int err;if (pdev->dev.of_node) {@@ -2912,6 +2914,10 @@ static int mv643xx_eth_probe(struct platform_device *pdev)else pd->phy_addr =MV643XX_ETH_PHY_ADDR_DEFAULT ; + mac =of_get_property (pdev->dev.of_node,"local-mac-address" , &len); +if (mac && len ==6 ) +memcpy (pd->mac_addr, mac,sizeof pd->mac_addr); + np =of_parse_phandle (pdev->dev.of_node,"mdio" ,0 );if (np) { pd->shared =of_find_device_by_node (np);Signature · 1 line
1.7.5.4
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 6 lines · Jason Cooper
Quoted 2 lines
Yes, you asked for the doc update and I haven't had a moment to get a tree setup for that..
Ahh, that's right. Sorry, wasn't trying to put pressure on you. Just trying to make a reminder to myself of what to look for when I get to around to it. (I have a separate IMAP folder for such things ;-) )
No worries, right now I'm in a spot where the lastest ARM stuff has diverged so much that it is a major job to get my custom board working against the latest tree, so I'm kinda stuck waiting on some spare time/things to settle out...
Quoted 9 lines
Quoted 5 lines
Here are some words though:
- local-mac-address : Optional, the MAC address to assign to the device. If not specified then the MAC address in the HW registers is used, but the driver can not be made modular.
I like it. If I haven't seen anything when I get to this, I'll add this to your patch. Please Ack if it's ok. I prefer not to change peoples patches without approval.
Please feel free to merge this text and the patch into Ian's DT patch, or rebase ontop of the latest version as you like:
Thanks for sorting this ethernet stuff!
Jason
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 12 lines · Sebastian Hesselbarth
Quoted 11 lines · Simon Baatz
Quoted 9 lines · Jason Cooper
Quoted 5 lines · Simon Baatz
Nothing special here. My config originally based on a Debian config for a Kirkwood kernel. Thus, amongst other drivers,
mv643xx_ethis build as a module and is loaded from an initrd.
Because all relevant drivers are loaded as modules, I need my runit patch to boot at all.Let's back up. If you config early printk and
COMMON_CLK_DEBUGand a vanilla v3.8-rc5 with your current config, what happens?
Quoted 8 lines
Quoted 3 lines
I thought it's clear what happens. "runit" is requested in the dts by serial, spi, watchdog, nand, i2c. Each of these are either not built or built as a module in my config, except serial.
Simon,
it is clear what happens but just to blindly disable clock gating will not help here. What you are suffering are several issues not only one.
Sure it is more than one issue. That's why I ran different scenarios.
Quoted 43 lines
Quoted 23 lines
of_serial.cdoes the following inof_platform_serial_setup():if (of_property_read_u32(np, "clock-frequency",&clk)) { /* Get clk rate through clk driver if present */ info->clk = clk_get(&ofdev->dev, NULL); if (IS_ERR(info->clk)) { dev_warn(&ofdev->dev, "clk or clock-frequency not defined\n"); return PTR_ERR(info->clk); } clk_prepare_enable(info->clk); clk = clk_get_rate(info->clk); }and since "clock-frequency" is still given in the standard DTS for the IB-NAS 6210, it does not enable the clock.
Thus, no driver uses the clock, it gets disabled and the system locks up.
The system boots when removing "clock-frequency" from the DTS.Quoted 3 lines
Which lead to two questions:
- Is it safe to remove "clock-frequency" now that we have the clocks in place?Safe, as long as you replace it with the corresponding clocks=<&..> property.
Quoted 2 lines
- The behavior of
of_serial.clooks strange to me, is this really the desired behavior?I guess it is, rely on clock-frequency because a lot of boards still use it. There was no clocks property in
of_serialwhen we started with kirkwood and dove. I think we can switch now to clocks property which will also remove all hard coded occurrences of tclk frequency.Quoted 5 lines
For the "runit" clock, this means that any time you hit a configuration that does not keep the clock enabled, the system will lockup. (We have gone through this more than once now. Especially, this hits you hard when you try to support a new board and disable as much as possible in the config/dts for a start...).
Quoted 6 lines
Well, if there is no device/driver using runit it should be safe to disable it. If there is a driver using it, we need a clocks property for it. And as you already stated, serial is using it. And you want serial in every minimal kernel, don't you?
From f479db4:
Marvell engineers tell us:
It seems that many units use the RUNIT clock.
SPI, UART, NAND, TWSI, ...
So it's not possible to clock gate it.
Currently the SPI, NAND and TWSI driver will clk_prepaure_enable() this clk, but since we have no idea what ... is, and turning this clk off results in a hard lock, unconditionally enable runit.
Thus, if we know better than that, that's fine, but please don't tell me that this is "blindly" enabling clocks.
Quoted 32 lines
Quoted 4 lines
Thus, we should keep it enabled even if it is unused, which is exactly the purpose of
CLK_IGNORE_UNUSEDif I understood this correctly.True, but only if it is that important that it should _never_ be gated. As long as it is 'only' used by peripheral devices, it should be safe to gate it.
Quoted 16 lines
Quoted 1 line
Could you also please try
kirkwood_defconfigand tell us if it boots?It is very easy to get my system to boot with a stock kernel. Just build one of the drivers I use directly into the kernel and it boots (up to the point where the gbe driver is loaded if built as a module, of course).
Quoted 8 lines
Quoted 8 lines
Here are my findings with various patches: ("non-DT" means booting the IBNAS6210 with machine ID 1680 ???Marvell DB-88F6281-BP Development Board???, which works reasonably well)
3.8-rc5 + runit patch:
DT: Hangs at boot (when loading
mv643xx_eth)
non-DT: Boots okQuoted 6 lines
In the DT case ge0 and ge1 are NULL, and thus, kirkwood_ge0[01]_init() tries to enable NULL clocks, but not the gbe clocks. -> The clocks get disabled.
As described in Sebastian's patch (and also found by Andrew some time ago), this leads to a hanging system when loading the gbe driver.
Quoted 6 lines
If I got all your configs right, this DT case is with serial but no clocks property? All other runit "users" are _not_ built into the kernel? Maybe it is just a coincidence that it fails when loading eth but I guess it hangs because of runit getting gated while serial accessing it. When you replace serial's clock-frequency with clocks property to "runit" it shouldn't hang.
No, this is the SMI hang already. As stated its "3.8-rc5 + runit patch". So runit is running.
And sure, removing "clock-frequency" gets the kernel booting until the gbe driver loads, as I said above in the same mail.
Quoted 1 line
(Issue 1: gated runit clock hangs kernel due to serial access)
Yes, and issue 1a: gated runit clock hangs kernel due to "...".
I can't get a stock kernel to boot when additionally disabling the serial driver altogether (and compiling the Ethernet driver into the kernel to avoid its lockup).
When the runit clock is enabled all the time, it works. I have no idea where it gets stuck, since I have no console. It looks like the SATA module does something (telling from the LEDs), but booting does not continue.
So, we have a statement from Marvell not to gate it, we know that it is needed for "...", can we please not gate it?
Quoted 38 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
3.8-rc5 + runit patch + ge00/ge01 patch:
DT: Boots ok
non-DT: Boots okSince the clocks are found now and kept enabled, DT behaves the same as non-DT.
Quoted 1 line
Quoted 1 line
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots okThis is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
Quoted 6 lines
Quoted 6 lines
3.8-rc5 + runit + using Sebastians patch + clks not ticking at module load:
DT: Boots, but no MAC
non-DT: Boots, but no MACOk, here my patch for smi clock enables gbe clock before accessing gbe registers.
(Issue 2: gated gbe clock hangs kernel due to smi access)
Quoted 4 lines
I think non-DT and DT gated clocks behave differently, since the non-DT ones also enable/disable the PHY. I explicitly removed the
clk_prepare_enable()from kirkwood_ge0[01]_init() in order to see if this makes a difference.True, DT gated clocks don't disable PHYs (and never will).
Quoted 8 lines
Quoted 4 lines
(Which reminds me, that we wanted to implement the PHY enabling in the driver.)
Apparently, it does not make a difference.
Leaves Issue 3, gbe forgets about its MAC address when gated or powered down. That should be done with local-mac-address passed by DT enabled u-boot or any other (dirty) ATAG hack ;)
Fine with all of that. But: I am talking about 3.8 all the time. We have three options for issues 2 and 3 from my point of view:
1. We do proper fixes in 3.8 for issues 2 and 3.
2. We fix this regression by not gating the clock in both the DT and the non-DT case, preferably by using the correct clocks in kirkwood_ge0[01]_init(). Additionally, Jason promises that all gets well with the DT aware driver in 3.9. ;-)
3. Do nothing. Simply accept that we broke modular Ethernet for DT because some code relies on two pointers being set. When we introduced a new code path for DT, we forgot about these pointers. Bad luck, the solution was not nice anyway and we will do proper fixes in 3.9.
As should be clear by now, I think we should go for option 2.
- Simon
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 14 lines · Simon Baatz
Quoted 4 lines · Sebastian Hesselbarth
Well, if there is no device/driver using runit it should be safe to disable it. If there is a driver using it, we need a clocks property for it. And as you already stated, serial is using it. And you want serial in every minimal kernel, don't you?
From f479db4:
Marvell engineers tell us:
It seems that many units use the RUNIT clock.
SPI, UART, NAND, TWSI, ...
So it's not possible to clock gate it.
Quoted 7 lines
Currently the SPI, NAND and TWSI driver will
clk_prepaure_enable()this clk, but since we have no idea what ... is, and turning this clk off results in a hard lock, unconditionally enable runit.Thus, if we know better than that, that's fine, but please don't tell me that this is "blindly" enabling clocks.
Hmm, should have seen that comment ealier. Based on your/Andrew's statement above using CLK_IGNORE_UNUSED on runit maybe is the best.
So I guess we take patch 2/2 and extend DT clock gating for .flags later?
Quoted 2 lines
So, we have a statement from Marvell not to gate it, we know that it is needed for "...", can we please not gate it?
Ack.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots okThis is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
For the smi clock issue: DT is fixed by the smi clock patch, right? non-DT should be fixed in kirkwood_clk_init() with an orion_clkdev_add() alias for MV643XX_ETH_SHARED_NAME ".0" and ".1" respectively.
For the MAC address issue: non-DT doesn't need to be fixed, right? DT clock gates should be fixed in kirkwood_legacy_clk_init which will also save the clk_get_sys() call. We can easily remove it when mv643xx_eth catches the MAC address from either local-mac-address or ATAG.
Quoted 10 lines
Fine with all of that. But: I am talking about 3.8 all the time. We have three options for issues 2 and 3 from my point of view:
1. We do proper fixes in 3.8 for issues 2 and 3.
2. We fix this regression by not gating the clock in both the DT and the non-DT case, preferably by using the correct clocks in kirkwood_ge0[01]_init(). Additionally, Jason promises that all gets well with the DT aware driver in 3.9. ;-)
Quoted 7 lines
3. Do nothing. Simply accept that we broke modular Ethernet for DT because some code relies on two pointers being set. When we introduced a new code path for DT, we forgot about these pointers. Bad luck, the solution was not nice anyway and we will do proper fixes in 3.9.
As should be clear by now, I think we should go for option 2.
Agreed, do you think all issues you are suffering will be solved with:
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds MV643XX_ETH_SHARED_NAME ".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)
- Some patch that adds clk_prepare_enable to ge0/ge1 clocks to kirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)
I'll prepare the latter patches and post them.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 15 lines · Sebastian Hesselbarth
Quoted 14 lines · Simon Baatz
Quoted 4 lines · Sebastian Hesselbarth
Well, if there is no device/driver using runit it should be safe to disable it. If there is a driver using it, we need a clocks property for it. And as you already stated, serial is using it. And you want serial in every minimal kernel, don't you?
From f479db4:
Marvell engineers tell us:
It seems that many units use the RUNIT clock.
SPI, UART, NAND, TWSI, ...
So it's not possible to clock gate it.
Quoted 7 lines
Quoted 7 lines
Currently the SPI, NAND and TWSI driver will
clk_prepaure_enable()this clk, but since we have no idea what ... is, and turning this clk off results in a hard lock, unconditionally enable runit.Thus, if we know better than that, that's fine, but please don't tell me that this is "blindly" enabling clocks.
'blindly' is the correct term. I'm sorry, I'm not trying to be difficult, but I don't like applying a 'fix' because a 'Marvell engineer' was too f'ing lazy to figure out wtf was going on with the runit clock.
So what happens if runit is gated? Here's what I did to find out:
######
$ git checkout -b runit v3.8-rc5 $ make kirkwood_defconfig $ ./scripts/config -e COMMON_CLK_DEBUG -e NETCONSOLE \ -d SERIAL_OF_PLATFORM -d ORION_WATCHDOG \ -d SPI_ORION -d MTD_NAND_ORION -d I2C_MV64XXX
$ make uImage dtbs
# edit kernel cmdline in u-boot:
Marvell>> setenv bootargs 'console=null rootwait root=/dev/sda2 [email protected]/eth0, at 192.168.1.196/ff:ff:ff:ff:ff:ff' Marvell>> boot
$ socat udp-recv:6666 - <small pause here> EXT3-fs (sda2): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (sda2): using internal journal <smaller pause here>> Unable to handle kernel NULL pointer dereference at virtual address 00000000 #######
It boots all the way up to mounting the USB attached uSD card rootfs.
A few notes about this test setup:
- the rootfs still has a getty on ttyS0 (could be causing above NULL?)
- I haven't setup networking on it yet (I was just debugging mv643xx_eth gated clock issue).
All this tells me the kernel *does* boot with runit gated since all drivers that would grab it are disabled. Tomorrow I'll setup dhcp and sshd and try to mount debugfs to see if runit is indeed disabled. Unless someone gets to it before me ;-)
I'll get to the below tomorrow.
thx,
Jason.
Quoted 46 lines
Hmm, should have seen that comment ealier. Based on your/Andrew's statement above using
CLK_IGNORE_UNUSEDon runit maybe is the best.So I guess we take patch 2/2 and extend DT clock gating for .flags later?
Quoted 2 lines
So, we have a statement from Marvell not to gate it, we know that it is needed for "...", can we please not gate it?
Ack.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
3.8-rc5 + runit + Sebastians get smi clock patch (modified to use legacy clock names):
DT: Boots, but no MAC
non-DT: Boots okThis is the behavior originally found by Andrew. The clock patch eliminates the lockup, but the hardware forgets its MAC address.
For the smi clock issue: DT is fixed by the smi clock patch, right? non-DT should be fixed in
kirkwood_clk_init()with anorion_clkdev_add()alias forMV643XX_ETH_SHARED_NAME".0" and ".1" respectively.For the MAC address issue: non-DT doesn't need to be fixed, right? DT clock gates should be fixed in
kirkwood_legacy_clk_initwhich will also save theclk_get_sys()call. We can easily remove it whenmv643xx_ethcatches the MAC address from either local-mac-address or ATAG.Quoted 16 lines
Fine with all of that. But: I am talking about 3.8 all the time. We have three options for issues 2 and 3 from my point of view:
1. We do proper fixes in 3.8 for issues 2 and 3.
2. We fix this regression by not gating the clock in both the DT and the non-DT case, preferably by using the correct clocks in kirkwood_ge0[01]_init(). Additionally, Jason promises that all gets well with the DT aware driver in 3.9. ;-)
3. Do nothing. Simply accept that we broke modular Ethernet for DT because some code relies on two pointers being set. When we introduced a new code path for DT, we forgot about these pointers. Bad luck, the solution was not nice anyway and we will do proper fixes in 3.9.
Quoted 6 lines
Quoted 1 line
As should be clear by now, I think we should go for option 2.
Agreed, do you think all issues you are suffering will be solved with:
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
fix for v3.8-rc
Quoted 13 lines
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds
MV643XX_ETH_SHARED_NAME".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)- Some patch that adds
clk_prepare_enableto ge0/ge1 clocks tokirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)I'll prepare the latter patches and post them.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 21 lines · Sebastian Hesselbarth
above using
CLK_IGNORE_UNUSEDon runit maybe is the best.Quoted 10 lines
Fine with all of that. But: I am talking about 3.8 all the time. We have three options for issues 2 and 3 from my point of view:
1. We do proper fixes in 3.8 for issues 2 and 3.
2. We fix this regression by not gating the clock in both the DT and the non-DT case, preferably by using the correct clocks in kirkwood_ge0[01]_init(). Additionally, Jason promises that all gets well with the DT aware driver in 3.9. ;-)
Quoted 7 lines
3. Do nothing. Simply accept that we broke modular Ethernet for DT because some code relies on two pointers being set. When we introduced a new code path for DT, we forgot about these pointers. Bad luck, the solution was not nice anyway and we will do proper fixes in 3.9.
As should be clear by now, I think we should go for option 2.
Agreed, do you think all issues you are suffering will be solved with:
Quoted 8 lines
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds
MV643XX_ETH_SHARED_NAME".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)
I think your patch to get the smi clock is intended for device tree. Thus, the driver won't use these aliases, right?
Quoted 3 lines
- Some patch that adds
clk_prepare_enableto ge0/ge1 clocks tokirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)
I like mine better, since it only enables the clocks of the interfaces that are initialized in the init code. I tested it with non-DT as well. But either is fine with me.
Quoted 1 line
I'll prepare the latter patches and post them.
Thanks!
- Simon
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 4 lines · Simon Baatz
Quoted 3 lines · Sebastian Hesselbarth
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
Quoted 8 lines
Quoted 5 lines
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds
MV643XX_ETH_SHARED_NAME".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)I think your patch to get the smi clock is intended for device tree. Thus, the driver won't use these aliases, right?
Actually, both patches above will not fix modular ethernet for 3.8-rc as shared driver is probed before core driver and not requesting any clk at all. The "NET: mv643xx: get smi clock from device tree" patch is based on Jason's attempt to separate shared driver.
If we need to fix modular ethernet now, we also need to add a clk_get to shared ethernet.
But yes, DT doesn't need any clock aliases.
Quoted 7 lines
Quoted 3 lines
- Some patch that adds
clk_prepare_enableto ge0/ge1 clocks tokirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)I like mine better, since it only enables the clocks of the interfaces that are initialized in the init code. I tested it with non-DT as well. But either is fine with me.
I know the difference, but here it is not only about fixing an issue but have it cleanly removed later on. But I don't have a strong opinion on that and maybe Andrew or Jason should coordinate what must be fixed now and how we do it.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Hi Sebastian,
Quoted 28 lines · Sebastian Hesselbarth
Quoted 4 lines · Simon Baatz
Quoted 3 lines · Sebastian Hesselbarth
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
Quoted 8 lines
Quoted 5 lines
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds
MV643XX_ETH_SHARED_NAME".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)I think your patch to get the smi clock is intended for device tree. Thus, the driver won't use these aliases, right?
Actually, both patches above will not fix modular ethernet for 3.8-rc as shared driver is probed before core driver and not requesting any clk at all. The "NET: mv643xx: get smi clock from device tree" patch is based on Jason's attempt to separate shared driver.
If we need to fix modular ethernet now, we also need to add a
clk_getto shared ethernet.But yes, DT doesn't need any clock aliases.
Quoted 4 lines
Quoted 3 lines
- Some patch that adds
clk_prepare_enableto ge0/ge1 clocks tokirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)
Quoted 8 lines
Quoted 3 lines
I like mine better, since it only enables the clocks of the interfaces that are initialized in the init code. I tested it with non-DT as well. But either is fine with me.
I know the difference, but here it is not only about fixing an issue but have it cleanly removed later on. But I don't have a strong opinion on that and maybe Andrew or Jason should coordinate what must be fixed now and how we do it.
Nitpicking here: For a DT aware driver on a DT board, the calls to kirkwood_ge0[01]_init() need to be removed anyway (this is already part of Jason's patches) and the clocks will not be enabled. Thus, there is not need to cleanup anything for DT when keeping the clk_prepare_enable in those functions (beyond what we need to do anyhow).
But now I will shut up. I fully agree that letting Jason/Andrew decide what to do is the best way forward.
- Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 28 lines · Sebastian Hesselbarth
Quoted 4 lines · Simon Baatz
Quoted 3 lines · Sebastian Hesselbarth
- [PATCH v2 2/2] clk: mvebu: Do not gate runit clock on Kirkwood (no lockup for minimal kernel configs)
Quoted 8 lines
Quoted 5 lines
- [PATCH] NET: mv643xx: get smi clock from device tree (no lockup for modular DT ethernet)
- Some patch that adds
MV643XX_ETH_SHARED_NAME".0" and ".1" clk aliases (no lockup for modular non-DT ethernet)I think your patch to get the smi clock is intended for device tree. Thus, the driver won't use these aliases, right?
Actually, both patches above will not fix modular ethernet for 3.8-rc as shared driver is probed before core driver and not requesting any clk at all. The "NET: mv643xx: get smi clock from device tree" patch is based on Jason's attempt to separate shared driver.
If we need to fix modular ethernet now, we also need to add a
clk_getto shared ethernet.But yes, DT doesn't need any clock aliases.
Quoted 4 lines
Quoted 3 lines
- Some patch that adds
clk_prepare_enableto ge0/ge1 clocks tokirkwood_legacy_clk_init()
(retain MAC address for modular DT ethernet)
Quoted 8 lines
Quoted 3 lines
I like mine better, since it only enables the clocks of the interfaces that are initialized in the init code. I tested it with non-DT as well. But either is fine with me.
I know the difference, but here it is not only about fixing an issue but have it cleanly removed later on. But I don't have a strong opinion on that and maybe Andrew or Jason should coordinate what must be fixed now and how we do it.
I agree that Simon's is nicer (per device disabling). However, Sebastian is correct, his is easier to remove later on once we have proper DT bindings in mv643xx_eth.
As it stands, there are three patches to fix this issue:
ARM: kirkwood: of_serial: fix clock gating by removing clock-frequency
ARM: kirkwood: provide ge clock aliases for shared smi
ARM: kirkwood: fix to retain gbe MAC addresses for DT kernels
wrt to runit gating, the only case we are not covering is if of_serial is a module (and so is everything else using the runit clk). That's *really* rare. If someone embarks down that path, they get the responsibility of not writing to all the deactivated registers. ;-)
wrt to ge losing mac addresses, both DT and non-DT booting are covered by Sebastian's patches, for non-DT aware mv643xx_eth.
Once we add proper DT support to mv643xx_eth, the local-mac-address will need to be defined in the dtb. We should probably poke the other Jason to see if there is a cooresponding u-boot patch to his local-mac-address patch.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 4 lines · Jason Cooper
Once we add proper DT support to
mv643xx_eth, the local-mac-address will need to be defined in the dtb. We should probably poke the other Jason to see if there is a cooresponding u-boot patch to his local-mac-address patch.
Sorry I don't have a u-boot fix, we don't use u-boot here - but local-mac-address is common on other ARM drivers so I'd hope there is a u-boot facility for stuffing it already?
Jason
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 10 lines · Jason Gunthorpe
Quoted 4 lines · Jason Cooper
Once we add proper DT support to
mv643xx_eth, the local-mac-address will need to be defined in the dtb. We should probably poke the other Jason to see if there is a cooresponding u-boot patch to his local-mac-address patch.Sorry I don't have a u-boot fix, we don't use u-boot here - but local-mac-address is common on other ARM drivers so I'd hope there is a u-boot facility for stuffing it already?
a quick search of the u-boot source reveals common/fdt_support.c:477:
do_fixup_by_path(fdt, path, "local-mac-address",
&mac_addr, 6, 1);
which is indeed called from bootm if u-boot is compiled with OF_LIBFDT.
I'll test this when I work on the mv643xx_eth DT binding series.
thx,
Jason.
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Jason CooperQuoted 5 lines · Jason Cooper
As it stands, there are three patches to fix this issue:
ARM: kirkwood:
of_serial: fix clock gating by removing clock-frequency
ARM: kirkwood: provide ge clock aliases for shared smi
ARM: kirkwood: fix to retain gbe MAC addresses for DT kernels
Actually, for the second patch I got distracted by the smi split patch set floating around. But that is not in current kernel and smi will not request any clock at all.
If Simon can hit another round of testing without second patch included and agrees, I suggest to keep it for next release.
Quoted 2 lines
wrt to ge losing mac addresses, both DT and non-DT booting are covered by Sebastian's patches, for non-DT aware
mv643xx_eth.
non-DT already ungates ge0/1 clocks on registration and cannot loose its mac address, not my fix.
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 7 lines · Sebastian Hesselbarth
Quoted 5 lines · Jason Cooper
As it stands, there are three patches to fix this issue:
ARM: kirkwood:
of_serial: fix clock gating by removing clock-frequency
ARM: kirkwood: provide ge clock aliases for shared smi
ARM: kirkwood: fix to retain gbe MAC addresses for DT kernels
Quoted 6 lines
Actually, for the second patch I got distracted by the smi split patch set floating around. But that is not in current kernel and smi will not request any clock at all.
If Simon can hit another round of testing without second patch included and agrees, I suggest to keep it for next release.
Ok, I'll wait for Simon on this. fixes can go anytime, so no rush. Better to get it right the first (second?) time out.
Quoted 5 lines
Quoted 2 lines
wrt to ge losing mac addresses, both DT and non-DT booting are covered by Sebastian's patches, for non-DT aware
mv643xx_eth.non-DT already ungates ge0/1 clocks on registration and cannot loose its mac address, not my fix.
Yes, bad wording on my part. I meant that everything was magically fixed by Sebastian and that his thesis should be approved without challenge. ;-)
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Hi Jason,
Quoted 12 lines · Jason Cooper
Quoted 11 lines · Sebastian Hesselbarth
Quoted 5 lines · Jason Cooper
As it stands, there are three patches to fix this issue:
ARM: kirkwood:
of_serial: fix clock gating by removing clock-frequency
ARM: kirkwood: provide ge clock aliases for shared smi
ARM: kirkwood: fix to retain gbe MAC addresses for DT kernelsActually, for the second patch I got distracted by the smi split patch set floating around. But that is not in current kernel and smi will not request any clock at all.
Quoted 5 lines
Quoted 2 lines
If Simon can hit another round of testing without second patch included and agrees, I suggest to keep it for next release.
Ok, I'll wait for Simon on this. fixes can go anytime, so no rush. Better to get it right the first (second?) time out.
Ok, will do. For the first patch you already have my Tested-by. Sebastian is right on the second one, it does not add value for 3.8 yet. I will test the third patch as soon as I find time.
- Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Jason CooperHi Jason, Sebastian,
Quoted 5 lines · Jason Cooper
wrt to runit gating, the only case we are not covering is if
of_serialis a module (and so is everything else using the runit clk). That's *really* rare. If someone embarks down that path, they get the responsibility of not writing to all the deactivated registers. ;-)
With the serial driver now enabling runit it is really rare, but where is your enthusiasm to get to the bottom of it? At least we have indications that there really is something in "..." (my box stops somewhere when no driver enables runit)
Sebastian, are you still interested in the .flags stuff from the runit patch or do you see no need now since "ddr" is the only exception anyway?
- Simon
Sebastian Hesselbarth
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 9 lines · Simon Baatz
Hi Jason, Sebastian,
Quoted 5 lines · Jason Cooper
wrt to runit gating, the only case we are not covering is if
of_serialis a module (and so is everything else using the runit clk). That's *really* rare. If someone embarks down that path, they get the responsibility of not writing to all the deactivated registers. ;-)
Quoted 4 lines
With the serial driver now enabling runit it is really rare, but where is your enthusiasm to get to the bottom of it? At least we have indications that there really is something in "..." (my box stops somewhere when no driver enables runit)
I think watchdog could be an issue, Jason had it disabled and he was able to run it, right? I don't have a strong opinion on that, but I'd disable every clock you can - OTOH runit will be enabled anyway if you choose to have serial.
Quoted 3 lines
Sebastian, are you still interested in the .flags stuff from the runit patch or do you see no need now since "ddr" is the only exception anyway?
I don't expect to use it on Dove but it should be good to have for Kirkwood and maybe Armada XP/370 too. You prepare a patch?
Sebastian
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 15 lines · Sebastian Hesselbarth
Quoted 9 lines · Simon Baatz
Hi Jason, Sebastian,
Quoted 5 lines · Jason Cooper
wrt to runit gating, the only case we are not covering is if
of_serialis a module (and so is everything else using the runit clk). That's *really* rare. If someone embarks down that path, they get the responsibility of not writing to all the deactivated registers. ;-)Quoted 4 lines
With the serial driver now enabling runit it is really rare, but where is your enthusiasm to get to the bottom of it? At least we have indications that there really is something in "..." (my box stops somewhere when no driver enables runit)
Quoted 4 lines
I think watchdog could be an issue, Jason had it disabled and he was able to run it, right? I don't have a strong opinion on that, but I'd disable every clock you can - OTOH runit will be enabled anyway if you choose to have serial.
I got the list of modules be searching kirkwood.dtsi for '&gate_clk 7'. the watchdog made the list.
Once I can get a proper test environment setup, my first goal is to show runit gated from a command prompt. The second goal is to use it in a *really* locked down / low power firewall gateway. There is no timeline for goal number two. ;-)
Quoted 6 lines
Quoted 3 lines
Sebastian, are you still interested in the .flags stuff from the runit patch or do you see no need now since "ddr" is the only exception anyway?
I don't expect to use it on Dove but it should be good to have for Kirkwood and maybe Armada XP/370 too. You prepare a patch?
I'm thinking we probably don't need it atm.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Simon BaatzQuoted 11 lines · Simon Baatz
Hi Jason, Sebastian,
Quoted 5 lines · Jason Cooper
wrt to runit gating, the only case we are not covering is if
of_serialis a module (and so is everything else using the runit clk). That's *really* rare. If someone embarks down that path, they get the responsibility of not writing to all the deactivated registers. ;-)With the serial driver now enabling runit it is really rare, but where is your enthusiasm to get to the bottom of it?
No one responded to my email. I figured I got a little too intense and decided to let it go. Didn't want to be the ranting a-hole (too late). :-)
If you're interested, I still have a few ideas. One was to wire two USB serial adapters end to end to create a different console (console=/dev/ttyUSB0,115200, getty, etc). Since they would be going over usb, that's a different clock, so it should work and provide us with a safety net.
I'll see if I can dig up a few spare cables and try it out over the next few days. Priority is to get the pull requests for v3.9 in, though.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 5 lines · Jason Cooper
If you're interested, I still have a few ideas. One was to wire two USB serial adapters end to end to create a different console (console=/dev/ttyUSB0,115200, getty, etc). Since they would be going over usb, that's a different clock, so it should work and provide us with a safety net.
I can't recall, can you still use JTAG once the CPU has hung on a mbus access?
If so memory dumping the console ring, or cpu registers would get the answer pretty directly..
My guesses would be the RTC and/or GPIO blocks (the GPIO blinker needs a clock), based on table 94.
Jason
Andrew Lunn
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 2 lines
My guesses would be the RTC and/or GPIO blocks (the GPIO blinker needs a clock), based on table 94.
I've used AT91 parts where you can do GPO with the clock disabled, but GPI needed a ticking clock. So, yes, GPIO is a good candidate for ruint clock as well.
Looking through the data sheets, and comparing against the gated clocks, we have the following without their own clock:
RTC, I2C (a.k.a. TWI), UART, NAND, SPI, Watchdog, eFuse,
Andrew
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 11 lines · Andrew Lunn
Quoted 2 lines
My guesses would be the RTC and/or GPIO blocks (the GPIO blinker needs a clock), based on table 94.
I've used AT91 parts where you can do GPO with the clock disabled, but GPI needed a ticking clock. So, yes, GPIO is a good candidate for ruint clock as well.
Looking through the data sheets, and comparing against the gated clocks, we have the following without their own clock:
RTC, I2C (a.k.a. TWI), UART, NAND, SPI, Watchdog, eFuse,
Hmm..
If watchdog is on the runit clock then the bridge registers and thus the timer are on the runit clock, so the whole point would be moot.
Any easy test would be to boot a system with a minimal DT, basically serial only, and have the kernel disable the runit clock, read a register from one of those blocks, enable the clock and print OK. The ones that lock up need the runit clock for sure. 7 boots should answer the question :)
My guess is that all the peripherals behind mbus unit 0x1 (see table 94, and table 96) are controlled by that clock gate. The other gates seem to be organized by mbus unit, and there is something very tidy about that from a hardware perspective :)
Jason
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Jason GunthorpeHi Jason,
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Jason Cooper
If you're interested, I still have a few ideas. One was to wire two USB serial adapters end to end to create a different console (console=/dev/ttyUSB0,115200, getty, etc). Since they would be going over usb, that's a different clock, so it should work and provide us with a safety net.
Quoted 8 lines
I can't recall, can you still use JTAG once the CPU has hung on a mbus access?
If so memory dumping the console ring, or cpu registers would get the answer pretty directly..
My guesses would be the RTC and/or GPIO blocks (the GPIO blinker needs a clock), based on table 94.
These guesses seem to be dead on:
Moved the RTC module and GPIO modules (keys, leds) out of the way, to see whether they cause the boot with disabled runit to lock up.
System boots now and SSH login is possible!
# mount -t debugfs debugfs /sys/kernel/debug # cat /sys/kernel/debug/clk/tclk/runit/clk_enable_count 0 # insmod ./gpio_keys.ko
System locks up.
and, after a reboot:
# insmod ./rtc-mv.ko
System locks up.
Bingo!
- Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 34 lines · Simon Baatz
Hi Jason,
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Jason Cooper
If you're interested, I still have a few ideas. One was to wire two USB serial adapters end to end to create a different console (console=/dev/ttyUSB0,115200, getty, etc). Since they would be going over usb, that's a different clock, so it should work and provide us with a safety net.
Quoted 8 lines
I can't recall, can you still use JTAG once the CPU has hung on a mbus access?
If so memory dumping the console ring, or cpu registers would get the answer pretty directly..
My guesses would be the RTC and/or GPIO blocks (the GPIO blinker needs a clock), based on table 94.
These guesses seem to be dead on:
Moved the RTC module and GPIO modules (keys, leds) out of the way, to see whether they cause the boot with disabled runit to lock up.
System boots now and SSH login is possible!
# mount -t debugfs debugfs /sys/kernel/debug # cat /sys/kernel/debug/clk/tclk/runit/clk_enable_count 0 # insmod ./gpio_keys.koSystem locks up.
Quoted 7 lines
and, after a reboot:
# insmod ./rtc-mv.koSystem locks up.
Bingo!
Awesome! Thanks for running the test Simon! I see Andrew sent two patches hopefully fixing these lockups. I'll wait for your tested-by and then apply all four to mvebu/fixes:
ARM: kirkwood: of_serial: fix clock gating by removing clock-frequency
ARM: kirkwood: fix to retain gbe MAC addresses for DT kernels
gpio: mvebu: Add clk support to prevent lockup
rtc: rtc-mv: Add support for clk to avoid lockups
I'll push Jason's local-mac-address patch to v3.9.
That should cover everything.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 3 lines · Sebastian Hesselbarth
I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
FWIW, there are two block power management controls on the kirkwood chips, one is documented to a clock gate, and one is documented to a be a power down. I wouldn't expect the clock gate to have a problem with the MAC address, but the powerdown I would expect to wipe it..
Other varients might have only the powerdown..
That said, I'm not sure what Linux uses on Kirkwood, it ideally should be using both, I think.
Jason
Andrew Lunn
maintainerPATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 9 lines · Jason Gunthorpe
Quoted 3 lines · Sebastian Hesselbarth
I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
FWIW, there are two block power management controls on the kirkwood chips, one is documented to a clock gate, and one is documented to a be a power down.
Hi Jason
Are you referring to
Memory Power Management Control Register
Offset: 0x20118
Bit Field Type / Init Val Description
0 GE0_Mem_PD RW GbE0 Memory Power Down:
0x0 0 = Functional
1 = PowerDown
1 PEX0_Mem_PD RW PCI Express0 Memory Power Down;
0x0 0 = Functional
1 = PowerDown
2 USB0_Mem_PD RW USB0 Memory Power Down:
0x0 0 = Functional
1 = PowerDown
etc.
Quoted 2 lines
I wouldn't expect the clock gate to have a problem
with the MAC address, but the powerdown I would expect to wipe it..
As far as i know, we don't touch it. However, as a debug exercise, it would be good to print its value at boot, at ethernet driver load time, after unused clocks are disabled, when the ethernet clock is enabled by the driver, etc.
Andrew
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 7 lines · Andrew Lunn
Quoted 6 lines · Jason Gunthorpe
Quoted 3 lines · Sebastian Hesselbarth
I just want Simon to confirm that Kirkwood's gbe is really loosing the contents of its MAC address registers during gated clocks, which is from a HW point of view very unlikely.
Quoted 8 lines
Quoted 3 lines
FWIW, there are two block power management controls on the kirkwood chips, one is documented to a clock gate, and one is documented to a be a power down.
Are you referring to
Memory Power Management Control Register
Offset:0x20118
Yes, and the other is 'Clock Gating Control Register' at 2011c.
I suppose Linux should really used both for maximum power savings, probably in a clock off, then power off kind of pattern??
Jason
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Sebastian HesselbarthQuoted 8 lines · Sebastian Hesselbarth
I agree that loosing the MAC address _is_ an issue but there must be another way to retain it during gated ge clocks than not gate the clocks at all.
I can think of some ways to retain it but don't know what is the most common with linux:
- make u-boot pass it through cmdline and let mv643xx get it from there
- have kirkwood's common code grab it before clk gates kick in
The cannonical solution here is to have a DT attribute 'local-mac-address' that is filled in by the bootloader rather than attempting to pass the mac address to the kernel through the ethernet controller registers.
Until the bootloaders are fixed a reasonable hack is to have the platform startup code look for an all-zeros local-mac-address in the DT and if found then copy the MAC from the ethernet registers into the DT. Then the ethernet driver can safely be a module since the MAC is captured in the DT.
I've been using this patch here on top of the original Ian Molton patch.
Jason: Can you include something like this as well?
drivers/net/ethernet/marvell/mv643xx_eth.c +6 -0
@@ -2891,6 +2891,8 @@ static int mv643xx_eth_probe(struct platform_device *pdev)struct mv643xx_eth_private *mp;struct net_device *dev;struct resource *res; +const u8 *mac; +int len;int err;if (pdev->dev.of_node) {
@@ -2912,6 +2914,10 @@ static int mv643xx_eth_probe(struct platform_device *pdev)else pd->phy_addr =MV643XX_ETH_PHY_ADDR_DEFAULT ; + mac =of_get_property (pdev->dev.of_node,"local-mac-address" , &len); +if (mac && len ==6 ) +memcpy (pd->mac_addr, mac,sizeof pd->mac_addr); + np =of_parse_phandle (pdev->dev.of_node,"mdio" ,0 );if (np) { pd->shared =of_find_device_by_node (np);
Signature · 1 line
1.7.5.4
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Thanks Jason,
A question for Simon below
Quoted 11 lines · Jason Gunthorpe
Quoted 8 lines · Sebastian Hesselbarth
I agree that loosing the MAC address _is_ an issue but there must be another way to retain it during gated ge clocks than not gate the clocks at all.
I can think of some ways to retain it but don't know what is the most common with linux:
- make u-boot pass it through cmdline and let mv643xx get it from there
- have kirkwood's common code grab it before clk gates kick in
Quoted 4 lines
The cannonical solution here is to have a DT attribute 'local-mac-address' that is filled in by the bootloader rather than attempting to pass the mac address to the kernel through the ethernet controller registers.
Simon,
Can you try this in conjunction with enabling ARM_ATAG_DTB_COMPAT ? Does that solve your issue?
thx,
Jason.
Quoted 39 lines
Until the bootloaders are fixed a reasonable hack is to have the platform startup code look for an all-zeros local-mac-address in the DT and if found then copy the MAC from the ethernet registers into the DT. Then the ethernet driver can safely be a module since the MAC is captured in the DT.
I've been using this patch here on top of the original Ian Molton patch.
Jason: Can you include something like this as well?
drivers/net/ethernet/marvell/mv643xx_eth.c +6
-0@@ -2891,6 +2891,8 @@ static int mv643xx_eth_probe(struct platform_device *pdev)struct mv643xx_eth_private *mp;struct net_device *dev;struct resource *res; +const u8 *mac; +int len;int err;if (pdev->dev.of_node) {@@ -2912,6 +2914,10 @@ static int mv643xx_eth_probe(struct platform_device *pdev)else pd->phy_addr =MV643XX_ETH_PHY_ADDR_DEFAULT ; + mac =of_get_property (pdev->dev.of_node,"local-mac-address" , &len); +if (mac && len ==6 ) +memcpy (pd->mac_addr, mac,sizeof pd->mac_addr); + np =of_parse_phandle (pdev->dev.of_node,"mdio" ,0 );if (np) { pd->shared =of_find_device_by_node (np);Signature · 1 line
1.7.5.4
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 4 lines · Jason Cooper
Simon,
Can you try this in conjunction with enabling
ARM_ATAG_DTB_COMPAT? Does that solve your issue?
I use
CONFIG_ARM_ATAG_DTB_COMPAT=y CONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=y
in my standard configuration already. I don't use the original U-Boot that came with the device, but a mainline one which supports my box.
- Simon
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
Quoted 6 lines · Simon Baatz
Quoted 4 lines · Jason Cooper
Simon,
Can you try this in conjunction with enabling
ARM_ATAG_DTB_COMPAT? Does that solve your issue?
Quoted 7 lines
I use
CONFIG_ARM_ATAG_DTB_COMPAT=yCONFIG_ARM_ATAG_DTB_COMPAT_CMDLINE_FROM_BOOTLOADER=yin my standard configuration already. I don't use the original U-Boot that came with the device, but a mainline one which supports my box.
Ahh, I thought you were using the stock bootloader that came with the device. Thanks for clearing that up.
thx,
Jason.
PATCH v2 1/2 ARM: kirkwood: Ensure that kirkwood_ge0[01]_init() finds its clock
In reply to Jason GunthorpeQuoted 22 lines · Jason Gunthorpe
Quoted 8 lines · Sebastian Hesselbarth
I agree that loosing the MAC address _is_ an issue but there must be another way to retain it during gated ge clocks than not gate the clocks at all.
I can think of some ways to retain it but don't know what is the most common with linux:
- make u-boot pass it through cmdline and let mv643xx get it from there
- have kirkwood's common code grab it before clk gates kick inThe cannonical solution here is to have a DT attribute 'local-mac-address' that is filled in by the bootloader rather than attempting to pass the mac address to the kernel through the ethernet controller registers.
Until the bootloaders are fixed a reasonable hack is to have the platform startup code look for an all-zeros local-mac-address in the DT and if found then copy the MAC from the ethernet registers into the DT. Then the ethernet driver can safely be a module since the MAC is captured in the DT.
Quoted 4 lines
I've been using this patch here on top of the original Ian Molton patch.
Jason: Can you include something like this as well?
Seems reasonable. Mind updating the binding docs?
I'd also like to confirm that legacy, factory-installed bootloaders are passing the mac addr via a special ATAG. My cursory check of the current mainline u-boot code doesn't include it, but Marvell's version may have.
thx,
Jason.
Quoted 32 lines
drivers/net/ethernet/marvell/mv643xx_eth.c +6
-0@@ -2891,6 +2891,8 @@ static int mv643xx_eth_probe(struct platform_device *pdev)struct mv643xx_eth_private *mp;struct net_device *dev;struct resource *res; +const u8 *mac; +int len;int err;if (pdev->dev.of_node) {@@ -2912,6 +2914,10 @@ static int mv643xx_eth_probe(struct platform_device *pdev)else pd->phy_addr =MV643XX_ETH_PHY_ADDR_DEFAULT ; + mac =of_get_property (pdev->dev.of_node,"local-mac-address" , &len); +if (mac && len ==6 ) +memcpy (pd->mac_addr, mac,sizeof pd->mac_addr); + np =of_parse_phandle (pdev->dev.of_node,"mdio" ,0 );if (np) { pd->shared =of_find_device_by_node (np);Signature · 7 lines
1.7.5.4 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel at lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v2 2/2 clk: mvebu: Do not gate runit clock on Kirkwood
In reply to Simon BaatzCommit f479db "ARM: Kirkwood: Ensure runit clock always ticks." made sure that the runit clock always ticks on Kirkwood.
When moving the clock gating to clk-gating-ctrl.c for Kirkwood DT devices, this change was disabled. Set the CLK_IGNORE_UNUSED flag for "runit" to ensure that it always ticks.
--- drivers/clk/mvebu/clk-gating-ctrl.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/clk/mvebu/clk-gating-ctrl.c +4 -1
@@ -97,8 +97,11 @@ static void __init mvebu_clk_gating_setup(* isn't taken by any driver, but should anyway be * kept enabled, so we mark it as IGNORE_UNUSED for * now. +* Do the same for the "runit" clock on Kirkwood; +* gating this clock causes an immediate lockup. */ -if (!strcmp (descr[n].name,"ddr" )) +if (!strcmp (descr[n].name,"ddr" ) + || !strcmp (descr[n].name,"runit" )) flags |=CLK_IGNORE_UNUSED ; ctrl->gates[n] = clk_register_gate(NULL, descr[n].name, parent,
Signature · 1 line
1.7.9.5
Sebastian Hesselbarth
maintainerPATCH v2 2/2 clk: mvebu: Do not gate runit clock on Kirkwood
Quoted 19 lines · Simon Baatz
Commit f479db "ARM: Kirkwood: Ensure runit clock always ticks." made sure that the runit clock always ticks on Kirkwood.
When moving the clock gating to
clk-gating-ctrl.cfor Kirkwood DT devices, this change was disabled. Set theCLK_IGNORE_UNUSEDflag for "runit" to ensure that it always ticks.Signed-off-by Simon Baatz<[email protected]>--- drivers/clk/mvebu/clk-gating-ctrl.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)drivers/clk/mvebu/clk-gating-ctrl.c +0
-0@@ -97,8 +97,11 @@ static void __init mvebu_clk_gating_setup(* isn't taken by any driver, but should anyway be * kept enabled, so we mark it as IGNORE_UNUSED for
drivers/clk/mvebu/clk-gating-ctrl.c static void __init mvebu_clk_gating_setup( · line 107
Quoted 10 lines
drivers/clk/mvebu/clk-gating-ctrl.c +4
-1* now. +* Do the same for the "runit" clock on Kirkwood; +* gating this clock causes an immediate lockup. */ -if (!strcmp (descr[n].name,"ddr" )) +if (!strcmp (descr[n].name,"ddr" ) + || !strcmp (descr[n].name,"runit" )) flags |=CLK_IGNORE_UNUSED ; ctrl->gates[n] = clk_register_gate(NULL, descr[n].name, parent,
Simon,
I'd rather have .flags passed by the SoC specific struct as you did in v1.
Sebastian