ARM: mvebu: pass the coherency availability information at init time

PATCH New gerg at uclinux.org 3 participants 10 messages Jun 11, 2015 – Jun 30, 2015

Subsystems (5)

Stable branch Arm (32-bit) Arm sub-architectures Arm/Marvell Kirkwood and Armada 370, 375, 38x, 39x, XP, 3700, 7K/8K, CN9130 SoC support +1 more subsystems

lore

Share & tools

lore

gerg at uclinux.org

author

2015-06-11 03:19 UTC #0 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

From: Greg Ungerer <[email protected]>

This patch is specifically meant to be applied to stable tree linux-3.10.y.

IO coherency should not be used on the Armada 370 SoC, due to all the necessary pre-requisites for reliable operation not being met (such as write allocate cache policy, shareable pages, SMP bit set).

Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was intended to disable IO coherency for the Armada 370. However it only disables the CPU side IO coherency. The mbus driver (drivers/bus/ mvebu-mbus.c) still passes the IO coherency attributes through the dram chip selects and onto driver memory window attributes. It does this based on looking directly into the device tree (looking for "marvell,coherency-fabric").

To fix we pass the coherency availability information (whether enabled or not) to the mbus driver at init time. This is done in the same way that it was done for mainline kernels in commit 5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the coherency availability information at init time").

Having the IO coherency enabled on the Armada 370 SoC causes rare unreliable system behavior. It is not easy to consistently reproduce problems caused by this. Best method I have seen is heavy network load resulting in kernel dumps due to corrupted memory.

Signed-off-by Greg Ungerer <[email protected]>

Acked-by Gregory CLEMENT <[email protected]>

---
 arch/arm/mach-dove/common.c         |  2 +-
 arch/arm/mach-kirkwood/common.c     |  2 +-
 arch/arm/mach-mv78xx0/common.c      |  4 ++--
 arch/arm/mach-mvebu/armada-370-xp.c |  3 ++-
 arch/arm/mach-mvebu/coherency.c     | 15 +++++++++++++++
 arch/arm/mach-mvebu/coherency.h     |  1 +
 arch/arm/mach-orion5x/common.c      |  2 +-
 drivers/bus/mvebu-mbus.c            |  5 ++---
 include/linux/mbus.h                |  2 +-
 9 files changed, 26 insertions(+), 10 deletions(-)

arch/arm/mach-dove/common.c +1 -1

index e2b5da0..8d4f5dc 100644
@@ -226,7 +226,7 @@ void __init dove_init_early(void)
 	orion_time_set_base(TIMER_VIRT_BASE);
 	mvebu_mbus_init("marvell,dove-mbus",
 			BRIDGE_WINS_BASE, BRIDGE_WINS_SZ,
-			DOVE_MC_WINS_BASE, DOVE_MC_WINS_SZ);
+			DOVE_MC_WINS_BASE, DOVE_MC_WINS_SZ, 0);
 }
 
 static int __init dove_find_tclk(void)

arch/arm/mach-kirkwood/common.c +1 -1

index f389228..4f6831e 100644
@@ -530,7 +530,7 @@ void __init kirkwood_init_early(void)
 
 	mvebu_mbus_init("marvell,kirkwood-mbus",
 			BRIDGE_WINS_BASE, BRIDGE_WINS_SZ,
-			DDR_WINDOW_CPU_BASE, DDR_WINDOW_CPU_SZ);
+			DDR_WINDOW_CPU_BASE, DDR_WINDOW_CPU_SZ, 0);
 }
 
 int kirkwood_tclk;

arch/arm/mach-mv78xx0/common.c +2 -2

index 749a7f8..4722c98 100644
@@ -337,11 +337,11 @@ void __init mv78xx0_init_early(void)
 	if (mv78xx0_core_index() == 0)
 		mvebu_mbus_init("marvell,mv78xx0-mbus",
 				BRIDGE_WINS_CPU0_BASE, BRIDGE_WINS_SZ,
-				DDR_WINDOW_CPU0_BASE, DDR_WINDOW_CPU_SZ);
+				DDR_WINDOW_CPU0_BASE, DDR_WINDOW_CPU_SZ, 0);
 	else
 		mvebu_mbus_init("marvell,mv78xx0-mbus",
 				BRIDGE_WINS_CPU1_BASE, BRIDGE_WINS_SZ,
-				DDR_WINDOW_CPU1_BASE, DDR_WINDOW_CPU_SZ);
+				DDR_WINDOW_CPU1_BASE, DDR_WINDOW_CPU_SZ, 0);
 }
 
 void __init_refok mv78xx0_timer_init(void)

arch/arm/mach-mvebu/armada-370-xp.c +2 -1

index 1c48890..4377c34 100644
@@ -66,7 +66,8 @@ void __init armada_370_xp_init_early(void)
 			ARMADA_370_XP_MBUS_WINS_BASE,
 			ARMADA_370_XP_MBUS_WINS_SIZE,
 			ARMADA_370_XP_SDRAM_WINS_BASE,
-			ARMADA_370_XP_SDRAM_WINS_SIZE);
+			ARMADA_370_XP_SDRAM_WINS_SIZE,
+			coherency_available());
 
 #ifdef CONFIG_CACHE_L2X0
 	l2x0_of_init(0, ~0UL);

arch/arm/mach-mvebu/coherency.c +15 -0

index 3ee701f..ea26ebb 100644
@@ -137,6 +137,20 @@ static struct notifier_block mvebu_hwcc_platform_nb = {
 	.notifier_call = mvebu_hwcc_platform_notifier,
 };
 
+/*
+ * Keep track of whether we have IO hardware coherency enabled or not.
+ * On Armada 370's we will not be using it for example. We need to make
+ * that available [through coherency_available()] so the mbus controller
+ * doesn't enable the IO coherency bit in the attribute bits of the
+ * chip selects.
+ */
+static int coherency_enabled;
+
+int coherency_available(void)
+{
+	return coherency_enabled;
+}
+
 int __init coherency_init(void)
 {
 	struct device_node *np;

@@ -170,6 +184,7 @@ int __init coherency_init(void)
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
 		set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
+		coherency_enabled = 1;
 		bus_register_notifier(&platform_bus_type,
 					&mvebu_hwcc_platform_nb);
 	}

arch/arm/mach-mvebu/coherency.h +1 -0

index 2f42813..1501a4e 100644
@@ -19,6 +19,7 @@ int coherency_get_cpu_count(void);
 #endif
 
 int set_cpu_coherent(int cpu_id, int smp_group_id);
+int coherency_available(void);
 int coherency_init(void);
 
 #endif	/* __MACH_370_XP_COHERENCY_H */

arch/arm/mach-orion5x/common.c +1 -1

index f8a6db9..04877392 100644
@@ -213,7 +213,7 @@ void __init orion5x_init_early(void)
 		mbus_soc_name = NULL;
 	mvebu_mbus_init(mbus_soc_name, ORION5X_BRIDGE_WINS_BASE,
 			ORION5X_BRIDGE_WINS_SZ,
-			ORION5X_DDR_WINS_BASE, ORION5X_DDR_WINS_SZ);
+			ORION5X_DDR_WINS_BASE, ORION5X_DDR_WINS_SZ, 0);
 }
 
 void orion5x_setup_wins(void)

drivers/bus/mvebu-mbus.c +2 -3

index 711dcf4..7c43782 100644
@@ -838,7 +838,7 @@ fs_initcall(mvebu_mbus_debugfs_init);
 int __init mvebu_mbus_init(const char *soc, phys_addr_t mbuswins_phys_base,
 			   size_t mbuswins_size,
 			   phys_addr_t sdramwins_phys_base,
-			   size_t sdramwins_size)
+			   size_t sdramwins_size, int is_coherent)
 {
 	struct mvebu_mbus_state *mbus = &mbus_state;
 	const struct of_device_id *of_id;

@@ -865,8 +865,7 @@ int __init mvebu_mbus_init(const char *soc, phys_addr_t mbuswins_phys_base,
 		return -ENOMEM;
 	}
 
-	if (of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
-		mbus->hw_io_coherency = 1;
+	mbus->hw_io_coherency = is_coherent;
 
 	for (win = 0; win < mbus->soc->num_wins; win++)
 		mvebu_mbus_disable_window(mbus, win);

include/linux/mbus.h +1 -1

index dba482e..e80b9c7 100644
@@ -67,6 +67,6 @@ int mvebu_mbus_add_window(const char *devname, phys_addr_t base,
 int mvebu_mbus_del_window(phys_addr_t base, size_t size);
 int mvebu_mbus_init(const char *soc, phys_addr_t mbus_phys_base,
 		    size_t mbus_size, phys_addr_t sdram_phys_base,
-		    size_t sdram_size);
+		    size_t sdram_size, int is_coherent);
 
 #endif /* __LINUX_MBUS_H */

Signature · 1 line

1.9.1

Greg KH

maintainer

2015-06-11 03:45 UTC #1 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 3 lines · gerg at uclinux.org

From: Greg Ungerer <[email protected]>
This patch is specifically meant to be applied to stable tree linux-3.10.y.

Why? What's wrong with taking the exact specific upstream patches instead?

Your descriptions here:

Quoted 19 lines

IO coherency should not be used on the Armada 370 SoC, due to all the necessary pre-requisites for reliable operation not being met (such as write allocate cache policy, shareable pages, SMP bit set).
Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was intended to disable IO coherency for the Armada 370. However it only disables the CPU side IO coherency. The mbus driver (drivers/bus/ mvebu-mbus.c) still passes the IO coherency attributes through the dram chip selects and onto driver memory window attributes. It does this based on looking directly into the device tree (looking for "marvell,coherency-fabric").
To fix we pass the coherency availability information (whether enabled or not) to the mbus driver at init time. This is done in the same way that it was done for mainline kernels in commit 5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the coherency availability information at init time").

Quoted 4 lines

Having the IO coherency enabled on the Armada 370 SoC causes rare unreliable system behavior. It is not easy to consistently reproduce problems caused by this. Best method I have seen is heavy network load resulting in kernel dumps due to corrupted memory.

I don't understand the issue here, I really don't want to not take patches that are not in Linus's tree, sorry.

greg k-h

Greg Ungerer

author

2015-06-11 04:04 UTC #2 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 7 lines · Greg KH

Quoted 3 lines · gerg at uclinux.org
From: Greg Ungerer <[email protected]>
This patch is specifically meant to be applied to stable tree linux-3.10.y.
Why? What's wrong with taking the exact specific upstream patches instead?

The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.

Quoted 21 lines

Your descriptions here:
Quoted 19 lines
IO coherency should not be used on the Armada 370 SoC, due to all the necessary pre-requisites for reliable operation not being met (such as write allocate cache policy, shareable pages, SMP bit set).
Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was intended to disable IO coherency for the Armada 370. However it only disables the CPU side IO coherency. The mbus driver (drivers/bus/ mvebu-mbus.c) still passes the IO coherency attributes through the dram chip selects and onto driver memory window attributes. It does this based on looking directly into the device tree (looking for "marvell,coherency-fabric").
To fix we pass the coherency availability information (whether enabled or not) to the mbus driver at init time. This is done in the same way that it was done for mainline kernels in commit 5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the coherency availability information at init time").

Quoted 7 lines

Quoted 4 lines
Having the IO coherency enabled on the Armada 370 SoC causes rare unreliable system behavior. It is not easy to consistently reproduce problems caused by this. Best method I have seen is heavy network load resulting in kernel dumps due to corrupted memory.
I don't understand the issue here, I really don't want to not take patches that are not in Linus's tree, sorry.

Sure.

Regards
Greg

Thomas Petazzoni

2015-06-11 07:25 UTC #3 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Greg, Greg,

Quoted 6 lines · Greg Ungerer

Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.

As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.

The mistake is that 5686a1e5aa4 should have been marked for stable really.

Best regards,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Greg KH

maintainer

2015-06-11 14:51 UTC #4 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 7 lines · Thomas Petazzoni

Greg, Greg,
Quoted 3 lines · Greg Ungerer
Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?

Quoted 8 lines

Quoted 3 lines
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.

What about 3.14-stable?

And if this is just a simple backport, that should have been stated here.

thanks,

greg k-h

Thomas Petazzoni

2015-06-11 15:14 UTC #5 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Greg,

Quoted 6 lines · Greg KH

Quoted 4 lines
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.
What about 3.14-stable?

5686a1e5aa4 was merged in 3.16, and it should be backported all the way up to when Armada 370 support was merged with I/O coherency support, i.e in 3.8.

Best regards,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Greg Ungerer

author

2015-06-12 01:18 UTC #6 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

In reply to Greg KH

Quoted 12 lines · Greg KH

Quoted 11 lines · Thomas Petazzoni
Greg, Greg,
Quoted 6 lines · Greg Ungerer
Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.

Quoted 6 lines

Quoted 4 lines
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.
What about 3.14-stable?

As Thomas pointed out, yes. Due to file movements and other changes neither this patch (for 3.10.y) or the original commit 5686a1e5aa4 apply cleanly to 3.14.y.

How do you want to handle that for 3.14.y?

Quoted 2 lines

And if this is just a simple backport, that should have been stated here.

The part that said "This is done in the same way that it was done for mainline kernels in commit" was meant to convey that meaning. Do you want an updated patch with those exact words, "simple backport", in the commit message?

Regards
Greg

Greg KH

maintainer

2015-06-30 00:31 UTC #7 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 18 lines · Greg Ungerer

Quoted 17 lines · Greg KH
Quoted 15 lines · Thomas Petazzoni
Greg, Greg,
Quoted 6 lines · Greg Ungerer
Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.

Quoted 7 lines

Quoted 1 line
What about 3.14-stable?
As Thomas pointed out, yes. Due to file movements and other changes neither this patch (for 3.10.y) or the original commit 5686a1e5aa4 apply cleanly to 3.14.y.
How do you want to handle that for 3.14.y?

I need a backport for 3.14.y as well.

And I need a signed-off-by: from the subsystem maintainers that this backport is acceptable, as it's so different from what is in Linus's tree, before I can take it.

thanks,

greg k-h

Greg Ungerer

author

2015-06-30 13:09 UTC #8 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 21 lines · Greg KH

Quoted 20 lines · Greg Ungerer
Quoted 7 lines · Greg KH
Quoted 6 lines · Thomas Petazzoni
Greg, Greg,
Quoted 2 lines · Greg Ungerer
Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.
What about 3.14-stable?
As Thomas pointed out, yes. Due to file movements and other changes neither this patch (for 3.10.y) or the original commit 5686a1e5aa4 apply cleanly to 3.14.y.

Quoted 6 lines

Quoted 1 line
How do you want to handle that for 3.14.y?
I need a backport for 3.14.y as well.
And I need a signed-off-by: from the subsystem maintainers that this backport is acceptable, as it's so different from what is in Linus's tree, before I can take it.

Ok, I will prepare a 3.14 port. I will send to all recipients of this mail, that should catch all those who need to sign off on it.

I have generated a version 2 of the original 3.10 patch. No change to the code diffs, but it changes the commit message to include all of the original commit followed by a brief description of the back port. Perhaps this is better?

Regards
Greg

Greg KH

maintainer

2015-06-30 16:48 UTC #9 lore raw

PATCH 1/1 ARM: mvebu: pass the coherency availability information at init time

Quoted 31 lines · Greg Ungerer

Quoted 21 lines · Greg KH
Quoted 20 lines · Greg Ungerer
Quoted 7 lines · Greg KH
Quoted 6 lines · Thomas Petazzoni
Greg, Greg,
Quoted 2 lines · Greg Ungerer
Quoted 2 lines
Why? What's wrong with taking the exact specific upstream patches instead?
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
The exact patch mentioned below ("5686a1e5aa4") will not apply. Too much of the code around it has changed. This does the same thing in the same away taking into account the changes around it.
As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency availability information at init time"), I can confirm that it will clearly not apply as is on 3.10. What Greg Ungerer is proposing here is a backport of 5686a1e5aa4 to 3.10.
What about 3.14-stable?
As Thomas pointed out, yes. Due to file movements and other changes neither this patch (for 3.10.y) or the original commit 5686a1e5aa4 apply cleanly to 3.14.y.
Quoted 6 lines
Quoted 1 line
How do you want to handle that for 3.14.y?
I need a backport for 3.14.y as well.
And I need a signed-off-by: from the subsystem maintainers that this backport is acceptable, as it's so different from what is in Linus's tree, before I can take it.

Quoted 7 lines

Ok, I will prepare a 3.14 port. I will send to all recipients of this mail, that should catch all those who need to sign off on it.
I have generated a version 2 of the original 3.10 patch. No change to the code diffs, but it changes the commit message to include all of the original commit followed by a brief description of the back port. Perhaps this is better?

Yes, much better, thanks, I'll go queue those up later today.

greg k-h