ARM: mvebu: ethernet packets corruption and I/O coherency

Hi Thomas and all,
I have seen your patches ("ARM: mvebu: no I/O coherency on non-SMP and related updates") regarding I/O coherency and I think it might be related to a problem I am experiencing.

I do have a custom board based on Marvell Armada 370 running kernel 3.13.11 and I see random outgoing ethernet packet corruption (about 1 packet every some million) using mvneta driver.

I have tried linux kernel 3.17 without any improvement.

What do you think? Can you backport the fix to 3.13 kernel?

Regards,
Francesco

Dear Francesco Dolcini,

It could indeed be related. I have marked the relevant patches in the "ARM: mvebu: no I/O coherency on non-SMP and related updates" series as to be backported to stable up to v3.8, so when they get accepted, I'll take care of backporting them.

Best regards,

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Hi Thomas and all

I prepared and tested this small patch to fix the problem on kernel 3.13.11 and it seems to fix my ethernet packet corruption problem. Do you think it is fine?

Do you think that this bug on I/O cache coherency could also trigger some sporadic random OOPS and kernel panic? I got an OOPS with a broken LR in skb_segment() and a kernel panic in put_page(), but I was never able to reproduce any of them.

Thanks,
Francesco

@@ -124,6 +124,12 @@
 {
 	struct device_node *np;
 
+	if (!is_smp())
+	{
+	    pr_info("Coherency fabric disabled\n");
+	    return 0;
+	}
+
 	np = of_find_matching_node(NULL, of_coherency_table);
 	if (np) {
 		struct resource res;

@@ -150,6 +156,9 @@
 {
 	struct device_node *np;
 
+	if (!is_smp())
+	    return 0;
+
 	np = of_find_matching_node(NULL, of_coherency_table);
 	if (np) {
 		bus_register_notifier(&platform_bus_type,

Dear Francesco Dolcini,

There's one missing thing: as of 3.13, the mvebu-mbus driver was directly looking at the DT to see if it had a coherency fabric node, and if that's the case, then it was enabling the per-MBus window bit telling that this window uses HW I/O coherency. I'm not sure it causes some problems in practice, since with your patch all the cache maintenance operations anyway properly re-enabled. But still, I'd suggest to modify the mvebu-mbus driver accordingly. See:

	np = of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric");
	if (np) {
		mbus->hw_io_coherency = 1;
		of_node_put(np);
	}

in drivers/bus/mvebu-mbus.c.

It's hard to say exactly what could happen with the wrong I/O cache coherency setup. I would expect only the buffers used for DMA to not be updated properly, but I might be wrong.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Interestingly I used to experience some random panics under high network loads on the mirabox and I never knew whether they were attributed to the power supply or to cache corruption. But since I have modified the driver and cache management to synchronize caches before the Rx loop, I haven't encountered them anymore. It could be a pure coincidence just like it could also be more or less related, maybe due to the fact that the cache is synchronized much earlier than the data are used and that this changes the access patterns.

Just my few cents,
Willy