Skip to contentSkip to conversation

pci/quirks: Add quirk for Bayhub O2 SD controller

PATCH In review Rajat Jain 4 participants 5 messages Dec 8, 2021 – Dec 15, 2021

Subsystems (3)
lore
Share & tools
lore

Rajat Jain

author
#0 lore raw

PATCH 1/1 pci/quirks: Add quirk for Bayhub O2 SD controller

This particular SD controller from O2 / Bayhub only allows dword accesses to its LTR max latency registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

Thus add a quirk that saves and restores these registers manually using dword acesses:
LTR Max Snoop Latency Register
LTR Max No-Snoop Latency Register

---
 drivers/mmc/host/sdhci-pci.h |  1 -
 drivers/pci/quirks.c         | 39 ++++++++++++++++++++++++++++++++++++
 include/linux/pci_ids.h      |  1 +
 3 files changed, 40 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci-pci.h +0 -1
index 5e3193278ff9..d47cc0ba7ca4 100644
@@ -10,7 +10,6 @@
 #define PCI_DEVICE_ID_O2_SDS1		0x8421
 #define PCI_DEVICE_ID_O2_FUJIN2		0x8520
 #define PCI_DEVICE_ID_O2_SEABIRD0	0x8620
-#define PCI_DEVICE_ID_O2_SEABIRD1	0x8621
 
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO0	0x8809
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO1	0x880a
drivers/pci/quirks.c +39 -0
index 003950c738d2..b7bd19802744 100644
@@ -5857,3 +5857,42 @@ static void nvidia_ion_ahci_fixup(struct pci_dev *pdev)
 	pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING;
 }
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_NVIDIA, 0x0ab8, nvidia_ion_ahci_fixup);
+
+/*
+ * Bayhub OZ711LV2 SD controller has an errata that only allows DWORD accesses
+ * to the LTR max latency registers. Thus need to save and restore these
+ * registers manually.
+ */
+static void o2_seabird1_save_ltr(struct pci_dev *dev)
+{
+	struct pci_cap_saved_state *save_state;
+	u32 *reg32;
+
+	save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+	if (save_state) {
+		reg32 = &save_state->cap.data[0];
+		/* Preserve PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+		pci_read_config_dword(dev, 0x234, reg32);
+	} else {
+		pci_err(dev, "quirk can't save LTR snoop latency\n");
+	}
+}
+
+static void o2_seabird1_restore_ltr(struct pci_dev *dev)
+{
+	struct pci_cap_saved_state *save_state;
+	u32 *reg32;
+
+	save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+	if (save_state) {
+		reg32 = &save_state->cap.data[0];
+		/* Restore PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+		pci_write_config_dword(dev, 0x234, *reg32);
+	} else {
+		pci_err(dev, "quirk can't restore LTR snoop latency\n");
+	}
+}
+DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+			       o2_seabird1_save_ltr);
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+			       o2_seabird1_restore_ltr);
include/linux/pci_ids.h +1 -0
index 011f2f1ea5bb..6ed16aa38196 100644
@@ -1717,6 +1717,7 @@
 #define PCI_DEVICE_ID_O2_8221		0x8221
 #define PCI_DEVICE_ID_O2_8320		0x8320
 #define PCI_DEVICE_ID_O2_8321		0x8321
+#define PCI_DEVICE_ID_O2_SEABIRD1	0x8621
 
 #define PCI_VENDOR_ID_3DFX		0x121a
 #define PCI_DEVICE_ID_3DFX_VOODOO	0x0001
Signature · 1 line
2.34.1.400.ga245620fadb-goog
Acked-by on v1 1/1
#1 lore raw
Quoted 11 lines · Rajat Jain

This particular SD controller from O2 / Bayhub only allows dword accesses to its LTR max latency registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

Thus add a quirk that saves and restores these registers manually using dword acesses:
LTR Max Snoop Latency Register
LTR Max No-Snoop Latency Register

Kind regards
Uffe

Quoted 80 lines
---
 drivers/mmc/host/sdhci-pci.h |  1 -
 drivers/pci/quirks.c         | 39 ++++++++++++++++++++++++++++++++++++
 include/linux/pci_ids.h      |  1 +
 3 files changed, 40 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci-pci.h +0 -1
index 5e3193278ff9..d47cc0ba7ca4 100644
@@ -10,7 +10,6 @@
 #define PCI_DEVICE_ID_O2_SDS1          0x8421
 #define PCI_DEVICE_ID_O2_FUJIN2                0x8520
 #define PCI_DEVICE_ID_O2_SEABIRD0      0x8620
-#define PCI_DEVICE_ID_O2_SEABIRD1      0x8621
 
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO0  0x8809
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO1  0x880a
drivers/pci/quirks.c +39 -0
index 003950c738d2..b7bd19802744 100644
@@ -5857,3 +5857,42 @@ static void nvidia_ion_ahci_fixup(struct pci_dev *pdev)
        pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING;
 }
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_NVIDIA, 0x0ab8, nvidia_ion_ahci_fixup);
+
+/*
+ * Bayhub OZ711LV2 SD controller has an errata that only allows DWORD accesses
+ * to the LTR max latency registers. Thus need to save and restore these
+ * registers manually.
+ */
+static void o2_seabird1_save_ltr(struct pci_dev *dev)
+{
+       struct pci_cap_saved_state *save_state;
+       u32 *reg32;
+
+       save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+       if (save_state) {
+               reg32 = &save_state->cap.data[0];
+               /* Preserve PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+               pci_read_config_dword(dev, 0x234, reg32);
+       } else {
+               pci_err(dev, "quirk can't save LTR snoop latency\n");
+       }
+}
+
+static void o2_seabird1_restore_ltr(struct pci_dev *dev)
+{
+       struct pci_cap_saved_state *save_state;
+       u32 *reg32;
+
+       save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+       if (save_state) {
+               reg32 = &save_state->cap.data[0];
+               /* Restore PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+               pci_write_config_dword(dev, 0x234, *reg32);
+       } else {
+               pci_err(dev, "quirk can't restore LTR snoop latency\n");
+       }
+}
+DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+                              o2_seabird1_save_ltr);
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+                              o2_seabird1_restore_ltr);
include/linux/pci_ids.h +1 -1
index 011f2f1ea5bb..6ed16aa38196 100644
@@ -1717,6 +1717,7 @@
 #define PCI_DEVICE_ID_O2_8221          0x8221
 #define PCI_DEVICE_ID_O2_8320          0x8320
 #define PCI_DEVICE_ID_O2_8321          0x8321
+#define PCI_DEVICE_ID_O2_SEABIRD1      0x8621
 
 #define PCI_VENDOR_ID_3DFX             0x121a
 #define PCI_DEVICE_ID_3DFX_VOODOO      0x0001
--

2.34.1.400.ga245620fadb-goog

#2 lore raw
In reply to Rajat Jain
Quoted 3 lines · Rajat Jain

This particular SD controller from O2 / Bayhub only allows dword accesses to its LTR max latency registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

What happens if we use a non-dword access? Unsupported Request? Invalid data returned? Writes ignored?

I guess word accesses must not cause PCIe errors, since we still do them in pci_save_ltr_state() and pci_restore_ltr_state() even with this patch.

The app note says 0x234 (Max Latency registers), 0x248 (L1 PM Substates Control 1), and 0x24c (L1 PM Substates Control 2) are all broken, but the patch only mentions 0x234.

I guess for 0x248 and 0x24c (the L1 PM Substates Control registers), we're just lucky because those are dword registers, and all current users already do dword accesses.

What if we instead changed pci_save_ltr_state() and pci_restore_ltr_state() to do a single dword access instead of two word accesses? That kind of sweeps it under the rug, but we're already doing that for 0x248 and 0x24c.

If we did that, we shouldn't need a quirk at all, but the hardware bug is still lurking, and we should add a comment about it somewhere.

I guess setpci (and maybe lspci) could still do smaller accesses and see whatever the bad behavior is. Hmmm. Maybe we just have to live with that.

The app note doesn't actually say how to identify the part -- no "affected Device ID", for instance. Are we confident that the other O2_* devices are unaffected?

Quoted 86 lines

Thus add a quirk that saves and restores these registers manually using dword acesses:
LTR Max Snoop Latency Register
LTR Max No-Snoop Latency Register

---
 drivers/mmc/host/sdhci-pci.h |  1 -
 drivers/pci/quirks.c         | 39 ++++++++++++++++++++++++++++++++++++
 include/linux/pci_ids.h      |  1 +
 3 files changed, 40 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci-pci.h +0 -1
index 5e3193278ff9..d47cc0ba7ca4 100644
@@ -10,7 +10,6 @@
 #define PCI_DEVICE_ID_O2_SDS1		0x8421
 #define PCI_DEVICE_ID_O2_FUJIN2		0x8520
 #define PCI_DEVICE_ID_O2_SEABIRD0	0x8620
-#define PCI_DEVICE_ID_O2_SEABIRD1	0x8621
 
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO0	0x8809
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO1	0x880a
drivers/pci/quirks.c +39 -0
index 003950c738d2..b7bd19802744 100644
@@ -5857,3 +5857,42 @@ static void nvidia_ion_ahci_fixup(struct pci_dev *pdev)
 	pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING;
 }
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_NVIDIA, 0x0ab8, nvidia_ion_ahci_fixup);
+
+/*
+ * Bayhub OZ711LV2 SD controller has an errata that only allows DWORD accesses
+ * to the LTR max latency registers. Thus need to save and restore these
+ * registers manually.
+ */
+static void o2_seabird1_save_ltr(struct pci_dev *dev)
+{
+	struct pci_cap_saved_state *save_state;
+	u32 *reg32;
+
+	save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+	if (save_state) {
+		reg32 = &save_state->cap.data[0];
+		/* Preserve PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+		pci_read_config_dword(dev, 0x234, reg32);
+	} else {
+		pci_err(dev, "quirk can't save LTR snoop latency\n");
+	}
+}
+
+static void o2_seabird1_restore_ltr(struct pci_dev *dev)
+{
+	struct pci_cap_saved_state *save_state;
+	u32 *reg32;
+
+	save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+	if (save_state) {
+		reg32 = &save_state->cap.data[0];
+		/* Restore PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+		pci_write_config_dword(dev, 0x234, *reg32);
+	} else {
+		pci_err(dev, "quirk can't restore LTR snoop latency\n");
+	}
+}
+DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+			       o2_seabird1_save_ltr);
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+			       o2_seabird1_restore_ltr);
include/linux/pci_ids.h +1 -0
index 011f2f1ea5bb..6ed16aa38196 100644
@@ -1717,6 +1717,7 @@
 #define PCI_DEVICE_ID_O2_8221		0x8221
 #define PCI_DEVICE_ID_O2_8320		0x8320
 #define PCI_DEVICE_ID_O2_8321		0x8321
+#define PCI_DEVICE_ID_O2_SEABIRD1	0x8621
 
 #define PCI_VENDOR_ID_3DFX		0x121a
 #define PCI_DEVICE_ID_3DFX_VOODOO	0x0001
Signature · 1 line
2.34.1.400.ga245620fadb-goog
#3 lore raw

Hi Bjorn,

Thanks for taking a look.

Quoted 8 lines · Bjorn Helgaas
Quoted 3 lines · Rajat Jain

This particular SD controller from O2 / Bayhub only allows dword accesses to its LTR max latency registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

What happens if we use a non-dword access? Unsupported Request? Invalid data returned? Writes ignored?

Invalid data values are read / written.

Quoted 4 lines

I guess word accesses must not cause PCIe errors, since we still do them in pci_save_ltr_state() and pci_restore_ltr_state() even with this patch.

Yes, that is correct.

Quoted 8 lines

The app note says 0x234 (Max Latency registers), 0x248 (L1 PM Substates Control 1), and 0x24c (L1 PM Substates Control 2) are all broken, but the patch only mentions 0x234.

I guess for 0x248 and 0x24c (the L1 PM Substates Control registers), we're just lucky because those are dword registers, and all current users already do dword accesses.

Yes, that is right.

Quoted 5 lines

What if we instead changed pci_save_ltr_state() and pci_restore_ltr_state() to do a single dword access instead of two word accesses? That kind of sweeps it under the rug, but we're already doing that for 0x248 and 0x24c.

Yes, that is what I had in mind originally, and actually I'd prefer that too. I was afraid you might disagree :-). It sounds like we're on the same page though, so should I send a patch with that approach?

Quoted 3 lines

If we did that, we shouldn't need a quirk at all, but the hardware bug is still lurking, and we should add a comment about it somewhere.

I can add a comment in pci_save_ltr_state() / pci_restore_ltr_state().

Quoted 8 lines

I guess setpci (and maybe lspci) could still do smaller accesses and see whatever the bad behavior is. Hmmm. Maybe we just have to live with that.

The app note doesn't actually say how to identify the part -- no "affected Device ID", for instance. Are we confident that the other O2_* devices are unaffected?

Yes, I noticed that. I confirmed with them that no other parts are affected (in an internal bug unfortunately). I can ask them to update their appnote also.

Thanks & Best Regards,

Rajat

Quoted 87 lines
Quoted 87 lines

Thus add a quirk that saves and restores these registers manually using dword acesses:
LTR Max Snoop Latency Register
LTR Max No-Snoop Latency Register

---
 drivers/mmc/host/sdhci-pci.h |  1 -
 drivers/pci/quirks.c         | 39 ++++++++++++++++++++++++++++++++++++
 include/linux/pci_ids.h      |  1 +
 3 files changed, 40 insertions(+), 1 deletion(-)
drivers/mmc/host/sdhci-pci.h +0 -1
index 5e3193278ff9..d47cc0ba7ca4 100644
@@ -10,7 +10,6 @@
 #define PCI_DEVICE_ID_O2_SDS1                0x8421
 #define PCI_DEVICE_ID_O2_FUJIN2              0x8520
 #define PCI_DEVICE_ID_O2_SEABIRD0    0x8620
-#define PCI_DEVICE_ID_O2_SEABIRD1    0x8621
 
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO0        0x8809
 #define PCI_DEVICE_ID_INTEL_PCH_SDIO1        0x880a
drivers/pci/quirks.c +39 -0
index 003950c738d2..b7bd19802744 100644
@@ -5857,3 +5857,42 @@ static void nvidia_ion_ahci_fixup(struct pci_dev *pdev)
      pdev->dev_flags |= PCI_DEV_FLAGS_HAS_MSI_MASKING;
 }
 DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_NVIDIA, 0x0ab8, nvidia_ion_ahci_fixup);
+
+/*
+ * Bayhub OZ711LV2 SD controller has an errata that only allows DWORD accesses
+ * to the LTR max latency registers. Thus need to save and restore these
+ * registers manually.
+ */
+static void o2_seabird1_save_ltr(struct pci_dev *dev)
+{
+     struct pci_cap_saved_state *save_state;
+     u32 *reg32;
+
+     save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+     if (save_state) {
+             reg32 = &save_state->cap.data[0];
+             /* Preserve PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+             pci_read_config_dword(dev, 0x234, reg32);
+     } else {
+             pci_err(dev, "quirk can't save LTR snoop latency\n");
+     }
+}
+
+static void o2_seabird1_restore_ltr(struct pci_dev *dev)
+{
+     struct pci_cap_saved_state *save_state;
+     u32 *reg32;
+
+     save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_LTR);
+     if (save_state) {
+             reg32 = &save_state->cap.data[0];
+             /* Restore PCI_LTR_MAX_SNOOP_LAT & PCI_LTR_MAX_NOSNOOP_LAT */
+             pci_write_config_dword(dev, 0x234, *reg32);
+     } else {
+             pci_err(dev, "quirk can't restore LTR snoop latency\n");
+     }
+}
+DECLARE_PCI_FIXUP_SUSPEND_LATE(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+                            o2_seabird1_save_ltr);
+DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_O2, PCI_DEVICE_ID_O2_SEABIRD1,
+                            o2_seabird1_restore_ltr);
include/linux/pci_ids.h +1 -1
index 011f2f1ea5bb..6ed16aa38196 100644
@@ -1717,6 +1717,7 @@
 #define PCI_DEVICE_ID_O2_8221                0x8221
 #define PCI_DEVICE_ID_O2_8320                0x8320
 #define PCI_DEVICE_ID_O2_8321                0x8321
+#define PCI_DEVICE_ID_O2_SEABIRD1    0x8621
 
 #define PCI_VENDOR_ID_3DFX           0x121a
 #define PCI_DEVICE_ID_3DFX_VOODOO    0x0001
--

2.34.1.400.ga245620fadb-goog

#4 lore raw
Quoted 27 lines · Rajat Jain
Quoted 7 lines · Bjorn Helgaas
Quoted 3 lines · Rajat Jain

This particular SD controller from O2 / Bayhub only allows dword accesses to its LTR max latency registers:
https://github.com/rajatxjain/public_shared/blob/main/OZ711LV2_appnote.pdf

What happens if we use a non-dword access? Unsupported Request? Invalid data returned? Writes ignored?

Invalid data values are read / written.

Quoted 3 lines

I guess word accesses must not cause PCIe errors, since we still do them in pci_save_ltr_state() and pci_restore_ltr_state() even with this patch.

Yes, that is correct.

Quoted 7 lines

The app note says 0x234 (Max Latency registers), 0x248 (L1 PM Substates Control 1), and 0x24c (L1 PM Substates Control 2) are all broken, but the patch only mentions 0x234.

I guess for 0x248 and 0x24c (the L1 PM Substates Control registers), we're just lucky because those are dword registers, and all current users already do dword accesses.

Yes, that is right.

Quoted 8 lines
Quoted 4 lines

What if we instead changed pci_save_ltr_state() and pci_restore_ltr_state() to do a single dword access instead of two word accesses? That kind of sweeps it under the rug, but we're already doing that for 0x248 and 0x24c.

Yes, that is what I had in mind originally, and actually I'd prefer that too. I was afraid you might disagree :-). It sounds like we're on the same page though, so should I send a patch with that approach?

I think so. I don't *like* papering over it, but the quirk only compensates for one situation (pci_save_ltr_state() and pci_restore_ltr_state()). Any other places where we might read/write the LTR values will still fail. We don't have any other places yet, but if/when we get ASPM L1.2 figured out, I think we will.

If we had a quirk mechanism for filtering config accesses to certain devices, that would be ideal, but I don't think we have that. If you squint hard enough, aer_inject.c has something like that, but it's not general purpose.

Quoted 4 lines
Quoted 2 lines

If we did that, we shouldn't need a quirk at all, but the hardware bug is still lurking, and we should add a comment about it somewhere.

I can add a comment in pci_save_ltr_state() / pci_restore_ltr_state().

Maybe also in pcie_aspm_cap_init() for the L1 PM part. Just a one-liner should be enough. All the details will be in the commit log and the app note.

Bjorn