sata_nv does not function in kernel > 2.6.20.21... possible ACPI or PCI involvement?

Hello All,

Jeff Garzik forwarded me here to get some assistance resolving a functionality regression in my sata_nv controller driver in recent Linux kernels. I have attached the same data here that I ended up sending to Jeff + others on the linux-ide list already. I assume you will probably need other different data from Jeff so please let me know if any other omitted information is required for diagnostics and I will respond ASAP.

I am using the Supermicro H8DCE motherboard. Some (not all) of the SATA channels quit working due to some kind of resource conflict when I upgrade to any kernel above 2.6.20.xx series, in my case I am running 2.6.20.21 SMP x86_64 presently.

The boot error which appears in dmesg for the missing SATA channels on 2.6.23.12 is pasted below.

Thanks,
Matthew Hall

ACPI: PCI Interrupt Link [LT3D] enabled at IRQ 46
ACPI: PCI Interrupt 0000:80:07.0[A] -> Link [LT3D] -> GSI 46 (level, low) -> IRQ 46
sata_nv 0000:80:07.0: Using ADMA mode
PCI: Unable to reserve mem region #6:1000@dfefe000 for device 0000:80:07.0
ACPI: PCI interrupt for device 0000:80:07.0 disabled
sata_nv: probe of 0000:80:07.0 failed with error -16
ACPI: PCI Interrupt Link [LT2E] enabled at IRQ 45
ACPI: PCI Interrupt 0000:80:08.0[A] -> Link [LT2E] -> GSI 45 (level, low) -> IRQ 45
sata_nv 0000:80:08.0: Using ADMA mode
PCI: Unable to reserve mem region #6:1000@dfefd000 for device 0000:80:08.0
ACPI: PCI interrupt for device 0000:80:08.0 disabled
sata_nv: probe of 0000:80:08.0 failed with error -16

This looks like the same problem reported here:

https://bugzilla.redhat.com/show_bug.cgi?id=313491
https://bugzilla.redhat.com/show_bug.cgi?id=280641

Your 2.6.23.12 /proc/iomem shows this:

dfefd000-dfefdfff : 0000:80:08.0

    dfefd000-dfefd3ff : pnp 00:07
  dfefe000-dfefefff : 0000:80:07.0
    dfefe000-dfefe3ff : pnp 00:07

That means the dfefd000-dfefd3ff range is mentioned both as a resource of an ACPI motherboard device (we call it "pnp 00:07" above) and as a BAR of the SATA PCI device 0000:80:08.0.

The PNP "system" driver reserves that range to keep us from putting another device on top of it, and the sata_nv reservation fails because it extends past the end of the PNP reservation.

I think this is a bug in the BIOS description of the motherboard device. In 2.6.20, the PNP system driver reserved ioport resources but ignored mmio resources. In 2.6.23, the system driver reserves both ioport and mmio resources, which I think is more correct, but exposes this bug.

I posted the attached test patch to the redhat bugzilla above, but nobody's tested it yet. Can you try it?

Bjorn

Index: w/drivers/pnp/quirks.c
===================================================================
@@ -108,6 +108,47 @@
 		       "pnp: SB audio device quirk - increasing port range\n");
 }
 
+static int overlaps(resource_size_t start1, resource_size_t end1,
+		    resource_size_t start2, resource_size_t end2)
+{
+	if (start2 <= start1 && start1 < end2)
+		return 1;
+	if (start2 <= end1 && end1 < end2)
+		return 1;
+	return 0;
+}
+
+
+#include <linux/pci.h>
+
+static void quirk_supermicro_motherboard(struct pnp_dev *dev)
+{
+	struct pci_dev *pdev = NULL;
+	int i, j;
+
+	for_each_pci_dev(pdev) {
+		for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
+			if (!(pci_resource_flags(pdev, i) & IORESOURCE_MEM) ||
+			    pci_resource_len(pdev, i) == 0)
+				continue;
+
+			for (j = 0; j < PNP_MAX_MEM; j++) {
+				if (!pnp_mem_valid(dev, j) ||
+				    pnp_mem_len(dev, j) == 0)
+					continue;
+
+				if (overlaps(pnp_mem_start(dev, j),
+					     pnp_mem_end(dev, j),
+					     pci_resource_start(pdev, i),
+					     pci_resource_end(pdev, i))) {
+					dev_warn(&dev->dev, "mem resource overlaps %s BAR %d, disabling\n", pci_name(pdev), i);
+					pnp_mem_flags(dev, j) = 0;
+				}
+			}
+		}
+	}
+}
+
 /*
  *  PnP Quirks
  *  Cards or devices that need some tweaking due to incomplete resource info

@@ -128,6 +169,8 @@
 	{"CTL0043", quirk_sb16audio_resources},
 	{"CTL0044", quirk_sb16audio_resources},
 	{"CTL0045", quirk_sb16audio_resources},
+	{"PNP0c01", quirk_supermicro_motherboard},
+	{"PNP0c02", quirk_supermicro_motherboard},
 	{""}
 };
 

Bjorn,

I complained to Supermicro and mentioned that the beta BIOS they provided to one of the customers in the Redhat Bugzilla threads did not solve the issue at all. I also included links to both Bugzilla threads as well as a link to the mail you sent back to me as snipped above which contained your excellent analysis of the problem.

The patch applied, booted, ran, and solved the problem perfectly. Do you want any output or configuration files to verify anything? Let me know if there is anything I can do to help. Other than that if you think it is safe to apply this, I am in favor of adding it to the mainline or other appropriate tree.

Thanks,
Matthew Hall

My patch is based on the assumption that a PCI resource (like the SATA BAR here) is completely discoverable and configurable through the standard PCI mechanisms, and it therefore should not also be described as a resource of an ACPI device.

If Supermicro went to the trouble of providing a beta BIOS, maybe they're interested enough to shed some light on why their BIOS does it this way. It's possible that my assumption is wrong, and Linux is just not doing the right thing. If you have a Supermicro contact, you might pass this by them.

I'll clean up the patch and post it. With luck, it might show up in the -mm tree soon and then in 2.6.25.

Bjorn

Matthew,

Can you please try the revised patch below? It's not generic like my first one, but I think it's a little safer because it doesn't rely on the discovery of PCI devices before the PNP quirk is run.

Thanks,
Bjorn

PNP: disable Supermicro H8DCE motherboard resources that overlap SATA BARs

Some Supermicro BIOSes apparently describe a SATA PCI BAR as a motherboard resource. The PNP system driver claims motherboard resources, and this prevents the sata_nv driver from requesting it later.

This patch disables the PNP0C01/PNP0C02 resources so they won't be claimed by the PNP system driver, so they'll available for sata_nv.

References:

    https://bugzilla.redhat.com/show_bug.cgi?id=280641
    https://bugzilla.redhat.com/show_bug.cgi?id=313491
    http://lkml.org/lkml/2008/1/9/449
    http://thread.gmane.org/gmane.linux.acpi.devel/27312

Index: work5/drivers/pnp/quirks.c
===================================================================
@@ -17,6 +17,7 @@
 #include <linux/slab.h>
 #include <linux/pnp.h>
 #include <linux/io.h>
+#include <linux/dmi.h>
 #include <linux/kallsyms.h>
 #include "base.h"
 

@@ -108,6 +109,46 @@
 		       "pnp: SB audio device quirk - increasing port range\n");
 }
 
+static void quirk_supermicro_h8dce_system(struct pnp_dev *dev)
+{
+	int i;
+	static struct dmi_system_id supermicro_h8dce[] = {
+		{
+			.ident = "Supermicro H8DCE",
+			.matches = {
+				DMI_MATCH(DMI_SYS_VENDOR, "Supermicro"),
+				DMI_MATCH(DMI_PRODUCT_NAME, "H8DCE"),
+			},
+		},
+		{ }
+	};
+
+	if (!dmi_check_system(supermicro_h8dce))
+		return;
+
+	/*
+	 * On the Supermicro H8DCE, there's a system device with resources
+	 * that overlap BAR 6 of the built-in SATA PCI adapter.  If the PNP
+	 * system device claims them, the sata_nv driver won't be able to.
+	 * More details at:
+	 *     https://bugzilla.redhat.com/show_bug.cgi?id=280641
+	 *     https://bugzilla.redhat.com/show_bug.cgi?id=313491
+	 *     http://lkml.org/lkml/2008/1/9/449
+	 *     http://thread.gmane.org/gmane.linux.acpi.devel/27312
+	 */
+	for (i = 0; i < PNP_MAX_MEM; i++) {
+		if (pnp_mem_valid(dev, i) && pnp_mem_len(dev, i) &&
+		    (pnp_mem_start(dev, i) & 0xdfef0000) == 0xdfef0000) {
+			dev_warn(&dev->dev, "disabling 0x%llx-0x%llx to prevent"
+				" conflict with sata_nv PCI device\n",
+				(unsigned long long) pnp_mem_start(dev, i),
+				(unsigned long long) (pnp_mem_start(dev, i) +
+					pnp_mem_len(dev, i) - 1));
+			pnp_mem_flags(dev, i) = 0;
+		}
+	}
+}
+
 /*
  *  PnP Quirks
  *  Cards or devices that need some tweaking due to incomplete resource info

@@ -128,6 +169,8 @@
 	{"CTL0043", quirk_sb16audio_resources},
 	{"CTL0044", quirk_sb16audio_resources},
 	{"CTL0045", quirk_sb16audio_resources},
+	{"PNP0c01", quirk_supermicro_h8dce_system},
+	{"PNP0c02", quirk_supermicro_h8dce_system},
 	{""}
 };
 

Bjorn,

It works on my machine! I just had to change the first hunk to get the includes to work because your file had a few more includes than mine since it was newer code.

Best Regards,
Matthew Hall

Hmm... The PNP quirk below is appealingly simple, and Matthew confirmed that it fixes the problem on his box.

But now I'm a little hesitant. It relies on the fact that we've discovered the SATA PCI device before running the quirks for the PNP system device. That happens to be the case now, but I don't think we should rely on it.

The ACPI root bridge device and the PNP system device are both discovered via the ACPI namespace. We currently treat them differently -- one is treated as an ACPI device, and the other is treated as a PNP device via the PNPACPI backend. But that distinction seems artificial to me, and I can imagine that the ACPI PCI root bridge driver could someday be integrated into PNP.

In that case, the PNP system device might be discovered and have its quirks run before the PCI root bridge is discovered and scanned.

As much as I'd like to have a generic quirk that would work around this defect and others like it, I think the safest thing would be to do something specific to the Supermicro H8DCE BIOS, since all the reports I've seen are on that box:

    https://bugzilla.redhat.com/show_bug.cgi?id=280641
    https://bugzilla.redhat.com/show_bug.cgi?id=313491
    http://lkml.org/lkml/2008/1/9/449

Matthew, could you collect the dmidecode output?

Bjorn