Data corruption when using INIC-1623TA2 controller

I'm experiencing frequent data corruption on my raid1 ext4 fs. The error is not always the same.

I first thought it was due to a previous resize of the FS I've done.
I had multiple times some message about huge amount of multiply claimed blocks in inode xxxx.
fsck.ext4 was not working fully and was always ending with a message like "FS still have error".
I was unable to copy all the files to another FS to save it.
So I end up checking the badblocks (that's where I've been dumb, I choose a non data conservative way).
However, badblock was succesful without errors.
So in the end I lost some data, however, I don't know if it's due to the bug or the the badblocks check. So feel free to readjust severity.

Since theni, I have bought two brand new disks, created a completly new ext4 FS, and copied the files that I had succesfully recovered. Then I run fsck.ext4 on the FS... it seems it is almost working.
I'm remounting the /dev/md0... And each time I start using the system seriously, I have new errors, like the one I had today:
(I was just copying files on it)

[1436849.120036] EXT4-fs (md0): error count: 6
[1436849.120044] EXT4-fs (md0): initial error at 1371763084: htree_dirblock_to_tree:587: inode 20971803: block 83894316
[1436849.120054] EXT4-fs (md0): last error at 1371765809: htree_dirblock_to_tree:587: inode 41813096: block 167256110
[1446656.923648] EXT4-fs error (device md0): htree_dirblock_to_tree:587: inode #52698372: block 210773049: comm smbd: bad entry in directory: directory entry across blocks - offset=1052(9244), inode=1949184565, rec_len=29816, name_len=24

So as planned I unplugged the working non RAID1 disk from their controller, and connect the ext4 RAID1 and the ext3 RAID1 disk to it (yeah these are 2 powerful RAID1 with 1 device only ;) for testing purposes).
I also tried to re-plug each PCI card, and connect the video card fan that was not connected (yeah it was a bad idea to limit the noise level few years ago).

I did all the tests I could to try to overheat the system (same as yesterday):
* 4 running dd if=/dev/urandom | gzip >/dev/null for the cpu
* massive copy from one disk to the other
* delete of duplicates between two directories (with many duplicates)

All that in parallel. Everything seems to work fine. No corruption nor CPU overheating message (yesterday I still had some even after remove the overclock of the CPU).

Here's the lspci -vvvv for this card (if I'm not wrong):

02:09.0 SATA controller: Initio Corporation INI-1623 PCI SATA-II Controller (rev 02) (prog-if 00 [Vendor specific])
Subsystem: Initio Corporation Device 1626
Control: I/O+ Mem+ BusMaster+ SpecCycle-
MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr-
DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 17 Region 0: I/O ports at 9000 [size=256] Region 1: Memory at ef022000 (32-bit,
non-prefetchable) [size=4K]

[virtual] Expansion ROM at 80000000 [disabled]
[size=128K]
Capabilities: [dc] Power Management version 2
Flags: PMEClk+ DSI- D1+ D2+
AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0
DScale=0 PME-
Kernel driver in use: sata_inic162x

So this does seem to be a fault in either this card or the driver. Can you suggest any further tests that Martin could do?

Unfortunately, I don't have any idea. That driver never really matured enough. I couldn't find enough information and no one from initio responded, so....

From bb9696192826a7d9279caf872e95b41bc26c7eff Mon Sep 17 00:00:00 2001

Hello, guys.

I've committed the following patch to for-3.11-fixes. It's unfortunate but I can't see a good way out at this point. :(

Quoted 4 lines · Tejun Heo

Hello, guys.

I've committed the following patch to for-3.11-fixes. It's unfortunate but I can't see a good way out at this point. :(

[...]

This change is also appropriate for a stable update.

I will also remove the driver from Debian packages.

From bb9696192826a7d9279caf872e95b41bc26c7eff Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Mon, 22 Jul 2013 16:53:36 -0400

sata_inic162x never reached a state where it's reliable enough for production use and data corruption is a relatively common occurrence. Make the driver generate warning about the issues and mark the Kconfig option as experimental.

If the situation doesn't improve, we'd be better off making it depend on CONFIG_BROKEN. Let's wait for several cycles and see if the kernel message draws any attention.

Signed-off-by Tejun Heo <[email protected]>

Reported-by Martin Braure de Calignon <[email protected]>

Reported-by Ben Hutchings <[email protected]>

Reported-by [email protected]

---
 drivers/ata/Kconfig         |  2 +-
 drivers/ata/sata_inic162x.c | 14 ++++++++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

drivers/ata/Kconfig +1 -1

index 80dc988..5cddaf8 100644
@@ -107,7 +107,7 @@ config SATA_FSL
 	  If unsure, say N.
 
 config SATA_INIC162X
-	tristate "Initio 162x SATA support"
+	tristate "Initio 162x SATA support (Very Experimental)"
 	depends on PCI
 	help
 	  This option enables support for Initio 162x Serial ATA.

drivers/ata/sata_inic162x.c +14 -0

index e451317..5c54d95 100644
@@ -6,6 +6,18 @@
  *
  * This file is released under GPL v2.
  *
+ * **** WARNING ****
+ *
+ * This driver never worked properly and unfortunately data corruption is
+ * relatively common.  There isn't anyone working on the driver and there's
+ * no support from the vendor.  Do not use this driver in any production
+ * environment.
+ *
+ * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491
+ * https://bugzilla.kernel.org/show_bug.cgi?id=60565
+ *
+ * *****************
+ *
  * This controller is eccentric and easily locks up if something isn't
  * right.  Documentation is available at initio's website but it only
  * documents registers (not programming model).

@@ -807,6 +819,8 @@ static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	ata_print_version_once(&pdev->dev, DRV_VERSION);
 
+	dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact [email protected] if in production use\n");
+
 	/* alloc host */
 	host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS);
 	hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);

Signature · 1 line

1.8.3.1

Hello, Greg.

I think the following commit should go into -stable but forgot to cc stable. It's now in Linus' tree. Can you please include it in -stable?

Hello, Greg.

I think the following commit should go into -stable but forgot to cc stable. It's now in Linus' tree. Can you please include it in -stable?

Thanks!

Quoted 62 lines · Tejun Heo

From bb9696192826a7d9279caf872e95b41bc26c7eff Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Mon, 22 Jul 2013 16:53:36 -0400

sata_inic162x never reached a state where it's reliable enough for production use and data corruption is a relatively common occurrence. Make the driver generate warning about the issues and mark the Kconfig option as experimental.

If the situation doesn't improve, we'd be better off making it depend on CONFIG_BROKEN. Let's wait for several cycles and see if the kernel message draws any attention.

Signed-off-by Tejun Heo <[email protected]>

Reported-by Martin Braure de Calignon <[email protected]>

Reported-by Ben Hutchings <[email protected]>

Reported-by [email protected]

---
 drivers/ata/Kconfig         |  2 +-
 drivers/ata/sata_inic162x.c | 14 ++++++++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)

drivers/ata/Kconfig +1 -1

index 80dc988..5cddaf8 100644
@@ -107,7 +107,7 @@ config SATA_FSL
 	  If unsure, say N.
 
 config SATA_INIC162X
-	tristate "Initio 162x SATA support"
+	tristate "Initio 162x SATA support (Very Experimental)"
 	depends on PCI
 	help
 	  This option enables support for Initio 162x Serial ATA.

drivers/ata/sata_inic162x.c +12 -0

index e451317..5c54d95 100644
@@ -6,6 +6,18 @@
  *
  * This file is released under GPL v2.
  *
+ * **** WARNING ****
+ *
+ * This driver never worked properly and unfortunately data corruption is
+ * relatively common.  There isn't anyone working on the driver and there's
+ * no support from the vendor.  Do not use this driver in any production
+ * environment.
+ *
+ * http://thread.gmane.org/gmane.linux.debian.devel.bugs.rc/378525/focus=54491
+ * https://bugzilla.kernel.org/show_bug.cgi?id=60565
+ *
+ * *****************
+ *
  * This controller is eccentric and easily locks up if something isn't
  * right.  Documentation is available at initio's website but it only
  * documents registers (not programming model).

@@ -807,6 +819,8 @@ static int inic_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	ata_print_version_once(&pdev->dev, DRV_VERSION);
 

Quoted 8 lines

drivers/ata/sata_inic162x.c +2 -0

+	dev_alert(&pdev->dev, "inic162x support is broken with common data corruption issues and will be disabled by default, contact [email protected] if in production use\n");
+
 	/* alloc host */
 	host = ata_host_alloc_pinfo(&pdev->dev, ppi, NR_PORTS);
 	hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);

Signature · 1 line

1.8.3.1

Hello, Greg.

I think the following commit should go into -stable but forgot to cc stable. It's now in Linus' tree. Can you please include it in -stable?

Thanks!

Quoted 3 lines · Tejun Heo

From bb9696192826a7d9279caf872e95b41bc26c7eff Mon Sep 17 00:00:00 2001
From: Tejun Heo <[email protected]>
Date: Mon, 22 Jul 2013 16:53:36 -0400