Skip to contentSkip to conversation

Add Intel Atom S1200 seris ioatdma support

PATCH In review Dave Jiang 3 participants 21 messages Mar 26, 2013 – Apr 10, 2013 1 message not in the archive dmaengine

lore
Share & tools
lore

Dave Jiang

author
#0 lore raw

cover Add Intel Atom S1200 seris ioatdma support

The following series adds support for the Intel Atom S1200 product family ioatdma. This ioatdma also implements a set of version v3.3 features such as 16 sources PQ, descriptor write back error status, and does not have many of the silicon bugs that the 3.2 line of hardware has due to a brand new implemention.

The series is dependent on the haswell update patch sent prior.

---

Dave Jiang (10):
ioatdma: Adding PCI IDs for Intel Atom S1200 product family ioatdma devices
ioatdma: Add 64bit chansts register read for ioat v3.3.
ioatdma: channel reset scheme fixup on Intel Atom S1200 platforms
ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
ioatdma: skip legacy reset bits since v3.3 plattform doesn't need it
ioatdma: Removing PQ val disable for cb3.3
ioatdma: skip silicon bug workaround for pq_align for cb3.3
ioatdma: Adding support for 16 src PQ ops and super extended descriptors
ioatdma: Adding write back descriptor error status support for ioatdma 3.3
ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap

 drivers/dma/Kconfig          |    2 
 drivers/dma/ioat/dma.c       |   14 +
 drivers/dma/ioat/dma.h       |   59 +++
 drivers/dma/ioat/dma_v2.h    |    2 
 drivers/dma/ioat/dma_v3.c    |  930 ++++++++++++++++++++++++++++++++++++------
 drivers/dma/ioat/hw.h        |   66 +++
 drivers/dma/ioat/pci.c       |    9 
 drivers/dma/ioat/registers.h |    4 
 8 files changed, 941 insertions(+), 145 deletions(-)
Signature · 1 line
Signature
author
#1 lore raw

PATCH 1/10 ioatdma: Adding PCI IDs for Intel Atom S1200 product family ioatdma devices

These should be good for the IOAT DMA devices on the Intel Atom S1269, S1279, and S1289 platforms. We are also adding IOAT v3.3 definition for the new DMA engine.

---
 drivers/dma/ioat/hw.h  |    6 ++++++
 drivers/dma/ioat/pci.c |    6 ++++++
 2 files changed, 12 insertions(+)
drivers/dma/ioat/hw.h +6 -0
index 8cfa077..ce431f5 100644
@@ -52,10 +52,16 @@
 #define PCI_DEVICE_ID_INTEL_IOAT_HSW8	0x2f2e
 #define PCI_DEVICE_ID_INTEL_IOAT_HSW9	0x2f2f
 
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD0	0x0C50
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD1	0x0C51
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD2	0x0C52
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD3	0x0C53
+
 #define IOAT_VER_1_2            0x12    /* Version 1.2 */
 #define IOAT_VER_2_0            0x20    /* Version 2.0 */
 #define IOAT_VER_3_0            0x30    /* Version 3.0 */
 #define IOAT_VER_3_2            0x32    /* Version 3.2 */
+#define IOAT_VER_3_3            0x33    /* Version 3.3 */
 
 
 int system_has_dca_enabled(struct pci_dev *pdev);
drivers/dma/ioat/pci.c +6 -0
index 67c8e83..1f63296 100644
@@ -105,6 +105,12 @@ static struct pci_device_id ioat_pci_tbl[] = {
 	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW8) },
 	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_HSW9) },
 
+	/* I/OAT v3.3 platforms */
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD0) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD1) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD2) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IOAT_BWD3) },
+
 	{ 0, }
 };
 MODULE_DEVICE_TABLE(pci, ioat_pci_tbl);
author
#2 lore raw

PATCH 2/10 ioatdma: Add 64bit chansts register read for ioat v3.3.

In reply to Dave Jiang

The channel status register for v3.3 is now 64bit. Use readq if available on v3.3 platforms.

---
 drivers/dma/ioat/dma.h |   22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
drivers/dma/ioat/dma.h +21 -1
index 95ae7b3..9285caa 100644
@@ -201,7 +201,7 @@ ioat_chan_by_index(struct ioatdma_device *device, int index)
 	return device->idx[index];
 }
 
-static inline u64 ioat_chansts(struct ioat_chan_common *chan)
+static inline u64 ioat_chansts_32(struct ioat_chan_common *chan)
 {
 	u8 ver = chan->device->version;
 	u64 status;
@@ -218,6 +218,26 @@ static inline u64 ioat_chansts(struct ioat_chan_common *chan)
 	return status;
 }
 
+#if BITS_PER_LONG == 64
+
+static inline u64 ioat_chansts(struct ioat_chan_common *chan)
+{
+	u8 ver = chan->device->version;
+	u64 status;
+
+	 /* With IOAT v3.3 the status register is 64bit.  */
+	if (ver >= IOAT_VER_3_3)
+		status = readq(chan->reg_base + IOAT_CHANSTS_OFFSET(ver));
+	else
+		status = ioat_chansts_32(chan);
+
+	return status;
+}
+
+#else
+#define ioat_chansts ioat_chansts_32
+#endif
+
 static inline void ioat_start(struct ioat_chan_common *chan)
 {
 	u8 ver = chan->device->version;
author
#3 lore raw

PATCH 3/10 ioatdma: channel reset scheme fixup on Intel Atom S1200 platforms

In reply to Dave Jiang

The Intel Atom S1200 family ioatdma changed the channel reset behavior. It does a reset similar to PCI FLR by resetting all the MSIX registers. We have to re-init msix interrupts because of this. This workaround is only specific to this platform and is not expected to carry over to the later generations.

---
 drivers/dma/ioat/dma.c    |    8 +-
 drivers/dma/ioat/dma.h    |   10 ++
 drivers/dma/ioat/dma_v3.c |  236 +++++++++++++++++++++++++++++----------------
 3 files changed, 171 insertions(+), 83 deletions(-)
drivers/dma/ioat/dma.c +7 -1
index 1879a59..17a2393 100644
@@ -892,7 +892,7 @@ MODULE_PARM_DESC(ioat_interrupt_style,
  * ioat_dma_setup_interrupts - setup interrupt handler
  * @device: ioat device
  */
-static int ioat_dma_setup_interrupts(struct ioatdma_device *device)
+int ioat_dma_setup_interrupts(struct ioatdma_device *device)
 {
 	struct ioat_chan_common *chan;
 	struct pci_dev *pdev = device->pdev;
@@ -941,6 +941,7 @@ msix:
 		}
 	}
 	intrctrl |= IOAT_INTRCTRL_MSIX_VECTOR_CONTROL;
+	device->irq_mode = IOAT_MSIX;
 	goto done;
 
 msix_single_vector:
@@ -956,6 +957,7 @@ msix_single_vector:
 		pci_disable_msix(pdev);
 		goto msi;
 	}
+	device->irq_mode = IOAT_MSIX_SINGLE;
 	goto done;
 
 msi:
@@ -969,6 +971,7 @@ msi:
 		pci_disable_msi(pdev);
 		goto intx;
 	}
+	device->irq_mode = IOAT_MSIX;
 	goto done;
 
 intx:
@@ -977,6 +980,7 @@ intx:
 	if (err)
 		goto err_no_irq;
 
+	device->irq_mode = IOAT_INTX;
 done:
 	if (device->intr_quirk)
 		device->intr_quirk(device);
@@ -987,9 +991,11 @@ done:
 err_no_irq:
 	/* Disable all interrupt generation */
 	writeb(0, device->reg_base + IOAT_INTRCTRL_OFFSET);
+	device->irq_mode = IOAT_NOIRQ;
 	dev_err(dev, "no usable interrupts\n");
 	return err;
 }
+EXPORT_SYMBOL(ioat_dma_setup_interrupts);
 
 static void ioat_disable_interrupts(struct ioatdma_device *device)
 {
drivers/dma/ioat/dma.h +10 -0
index 9285caa..b16902c 100644
@@ -48,6 +48,14 @@
  */
 #define NULL_DESC_BUFFER_SIZE 1
 
+enum ioat_irq_mode {
+	IOAT_NOIRQ = 0,
+	IOAT_MSIX,
+	IOAT_MSIX_SINGLE,
+	IOAT_MSI,
+	IOAT_INTX
+};
+
 /**
  * struct ioatdma_device - internal representation of a IOAT device
  * @pdev: PCI-Express device
@@ -77,6 +85,7 @@ struct ioatdma_device {
 	struct msix_entry msix_entries[4];
 	struct ioat_chan_common *idx[4];
 	struct dca_provider *dca;
+	enum ioat_irq_mode irq_mode;
 	void (*intr_quirk)(struct ioatdma_device *device);
 	int (*enumerate_channels)(struct ioatdma_device *device);
 	int (*reset_hw)(struct ioat_chan_common *chan);
@@ -341,6 +350,7 @@ bool ioat_cleanup_preamble(struct ioat_chan_common *chan,
 			   dma_addr_t *phys_complete);
 void ioat_kobject_add(struct ioatdma_device *device, struct kobj_type *type);
 void ioat_kobject_del(struct ioatdma_device *device);
+int ioat_dma_setup_interrupts(struct ioatdma_device *device);
 extern const struct sysfs_ops ioat_sysfs_ops;
 extern struct ioat_sysfs_entry ioat_version_attr;
 extern struct ioat_sysfs_entry ioat_cap_attr;
drivers/dma/ioat/dma_v3.c +154 -82

257 lines not on this page · raw message

author
#4 lore raw

PATCH 4/10 ioatdma: Removing hw bug workaround for CB3.x .2 and earlier

In reply to Dave Jiang

CB3.2 and earlier hardware has silicon bugs that are no longer needed with the new hardware. We don't have to use a NULL op to signal interrupt for RAID ops any longer. This code make sure the legacy workarounds only happen on legacy hardware.

---
 drivers/dma/ioat/dma.c    |    6 +++++
 drivers/dma/ioat/dma.h    |    8 +++++++
 drivers/dma/ioat/dma_v3.c |   50 +++++++++++++++++++++++++++++++++++----------
 3 files changed, 53 insertions(+), 11 deletions(-)
drivers/dma/ioat/dma.c +6 -0
index 17a2393..e2bf3fa 100644
@@ -1042,6 +1042,12 @@ int ioat_probe(struct ioatdma_device *device)
 	if (err)
 		goto err_setup_interrupts;
 
+	if (device->init_device) {
+		err = device->init_device(device);
+		if (err)
+			goto err_self_test;
+	}
+
 	err = device->self_test(device);
 	if (err)
 		goto err_self_test;
drivers/dma/ioat/dma.h +8 -0
index b16902c..12eab37 100644
@@ -92,8 +92,14 @@ struct ioatdma_device {
 	void (*cleanup_fn)(unsigned long data);
 	void (*timer_fn)(unsigned long data);
 	int (*self_test)(struct ioatdma_device *device);
+	int (*init_device)(struct ioatdma_device *device);
 };
 
+enum ioat_hwbugs {
+	IOAT_LEGACY_COMPLETION_REQUIRED = (1 << 0),
+};
+
+
 struct ioat_chan_common {
 	struct dma_chan common;
 	void __iomem *reg_base;
@@ -116,6 +122,8 @@ struct ioat_chan_common {
 	u64 *completion;
 	struct tasklet_struct cleanup_task;
 	struct kobject kobj;
+
+	u32 hwbug_flags;
 };
 
 struct ioat_sysfs_entry {
drivers/dma/ioat/dma_v3.c +39 -11

74 lines not on this page · raw message

author
#5 lore raw

PATCH 5/10 ioatdma: skip legacy reset bits since v3.3 plattform doesn't need it

In reply to Dave Jiang

Make it so only 3.2 and earlier platform need the PCI config register clearings since this implementation does not have the registers.

---
 drivers/dma/ioat/dma_v3.c |   34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)
drivers/dma/ioat/dma_v3.c +21 -13
index e66fead..c53204e 100644
@@ -1339,20 +1339,28 @@ static int ioat3_reset_hw(struct ioat_chan_common *chan)
 	chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 	writel(chanerr, chan->reg_base + IOAT_CHANERR_OFFSET);
 
-	/* clear any pending errors */
-	err = pci_read_config_dword(pdev, IOAT_PCI_CHANERR_INT_OFFSET, &chanerr);
-	if (err) {
-		dev_err(&pdev->dev, "channel error register unreachable\n");
-		return err;
-	}
-	pci_write_config_dword(pdev, IOAT_PCI_CHANERR_INT_OFFSET, chanerr);
+	if (device->version < IOAT_VER_3_3) {
+		/* clear any pending errors */
+		err = pci_read_config_dword(pdev,
+				IOAT_PCI_CHANERR_INT_OFFSET, &chanerr);
+		if (err) {
+			dev_err(&pdev->dev,
+				"channel error register unreachable\n");
+			return err;
+		}
+		pci_write_config_dword(pdev,
+				IOAT_PCI_CHANERR_INT_OFFSET, chanerr);
 
-	/* Clear DMAUNCERRSTS Cfg-Reg Parity Error status bit
-	 * (workaround for spurious config parity error after restart)
-	 */
-	pci_read_config_word(pdev, IOAT_PCI_DEVICE_ID_OFFSET, &dev_id);
-	if (dev_id == PCI_DEVICE_ID_INTEL_IOAT_TBG0)
-		pci_write_config_dword(pdev, IOAT_PCI_DMAUNCERRSTS_OFFSET, 0x10);
+		/* Clear DMAUNCERRSTS Cfg-Reg Parity Error status bit
+		 * (workaround for spurious config parity error after restart)
+		 */
+		pci_read_config_word(pdev, IOAT_PCI_DEVICE_ID_OFFSET, &dev_id);
+		if (dev_id == PCI_DEVICE_ID_INTEL_IOAT_TBG0) {
+			pci_write_config_dword(pdev,
+					       IOAT_PCI_DMAUNCERRSTS_OFFSET,
+					       0x10);
+		}
+	}
 
 	err = ioat2_reset_sync(chan, msecs_to_jiffies(200));
 	if (err) {
author
#6 lore raw

PATCH 6/10 ioatdma: Removing PQ val disable for cb3.3

In reply to Dave Jiang

The PQ Val ops work on the newer hardware so we should actually provide support for it and remove the disabling bits.

---
 drivers/dma/Kconfig          |    2 -
 drivers/dma/ioat/dma.h       |    1 
 drivers/dma/ioat/dma_v3.c    |  134 ++++++++++++++++++++++++++++++++++++++----
 drivers/dma/ioat/registers.h |    2 +
 4 files changed, 125 insertions(+), 14 deletions(-)
drivers/dma/Kconfig +0 -2
index 80b6997..dec088d 100644
@@ -63,8 +63,6 @@ config INTEL_IOATDMA
 	depends on PCI && X86
 	select DMA_ENGINE
 	select DCA
-	select ASYNC_TX_DISABLE_PQ_VAL_DMA
-	select ASYNC_TX_DISABLE_XOR_VAL_DMA
 	help
 	  Enable support for the Intel(R) I/OAT DMA engine present
 	  in recent Intel Xeon chipsets.
drivers/dma/ioat/dma.h +1 -0
index 12eab37..4fbf5c7 100644
@@ -39,6 +39,7 @@
 #define to_ioat_desc(lh) container_of(lh, struct ioat_desc_sw, node)
 #define tx_to_ioat_desc(tx) container_of(tx, struct ioat_desc_sw, txd)
 #define to_dev(ioat_chan) (&(ioat_chan)->device->pdev->dev)
+#define to_pdev(ioat_chan) ((ioat_chan)->device->pdev)
 
 #define chan_num(ch) ((int)((ch)->reg_base - (ch)->device->reg_base) / 0x80)
 
drivers/dma/ioat/dma_v3.c +122 -12

181 lines not on this page · raw message

drivers/dma/ioat/registers.h +2 -0
index 1391798..c1ad194 100644
@@ -93,6 +93,8 @@
 #define IOAT_CHANCTRL_ERR_COMPLETION_EN		0x0004
 #define IOAT_CHANCTRL_INT_REARM			0x0001
 #define IOAT_CHANCTRL_RUN			(IOAT_CHANCTRL_INT_REARM |\
+						 IOAT_CHANCTRL_ERR_INT_EN |\
+						 IOAT_CHANCTRL_ERR_COMPLETION_EN |\
 						 IOAT_CHANCTRL_ANY_ERR_ABORT_EN)
 
 #define IOAT_DMA_COMP_OFFSET			0x02	/* 16-bit DMA channel compatibility */
#7 lore raw

Re: [PATCH 06/10] ioatdma: Removing PQ val disable for cb3.3

Quoted 24 lines · Dave Jiang

The PQ Val ops work on the newer hardware so we should actually provide support for it and remove the disabling bits.

---
 drivers/dma/Kconfig          |    2 -
 drivers/dma/ioat/dma.h       |    1
 drivers/dma/ioat/dma_v3.c    |  134 ++++++++++++++++++++++++++++++++++++++----
 drivers/dma/ioat/registers.h |    2 +
 4 files changed, 125 insertions(+), 14 deletions(-)
drivers/dma/Kconfig +0 -2
index 80b6997..dec088d 100644
@@ -63,8 +63,6 @@ config INTEL_IOATDMA
        depends on PCI && X86
        select DMA_ENGINE
        select DCA
-       select ASYNC_TX_DISABLE_PQ_VAL_DMA
-       select ASYNC_TX_DISABLE_XOR_VAL_DMA
        help
          Enable support for the Intel(R) I/OAT DMA engine present
          in recent Intel Xeon chipsets.

[..]

Quoted 9 lines
drivers/dma/Kconfig +1 -4
@@ -1469,15 +1581,13 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
        device->cleanup_fn = ioat3_cleanup_event;
        device->timer_fn = ioat3_timer_event;
 
-       #ifdef CONFIG_ASYNC_TX_DISABLE_PQ_VAL_DMA
-       dma_cap_clear(DMA_PQ_VAL, dma->cap_mask);
-       dma->device_prep_dma_pq_val = NULL;
-       #endif
+       if (is_xeon_cb32(pdev)) {

drivers/dma/Kconfig int ioat3_dma_probe(struct ioatdma_device *device, int dca) · line 1590

Quoted 10 lines
drivers/dma/Kconfig +5 -4
+               dma_cap_clear(DMA_XOR_VAL, dma->cap_mask);
+               dma->device_prep_dma_xor_val = NULL;
 
-       #ifdef CONFIG_ASYNC_TX_DISABLE_XOR_VAL_DMA
-       dma_cap_clear(DMA_XOR_VAL, dma->cap_mask);
-       dma->device_prep_dma_xor_val = NULL;
-       #endif
+               dma_cap_clear(DMA_PQ_VAL, dma->cap_mask);
+               dma->device_prep_dma_pq_val = NULL;
+       }

Note that this effectively disables raid offload for is_xeon_cb32() platforms since DMA_ASYNC_TX will not be set.

author
#8 lore raw

Re: [PATCH 06/10] ioatdma: Removing PQ val disable for cb3.3

Quoted 38 lines · Dan Williams
Quoted 24 lines · Dave Jiang

The PQ Val ops work on the newer hardware so we should actually provide support for it and remove the disabling bits.

---
  drivers/dma/Kconfig          |    2 -
  drivers/dma/ioat/dma.h       |    1
  drivers/dma/ioat/dma_v3.c    |  134 ++++++++++++++++++++++++++++++++++++++----
  drivers/dma/ioat/registers.h |    2 +
  4 files changed, 125 insertions(+), 14 deletions(-)
drivers/dma/Kconfig +0 -2
index 80b6997..dec088d 100644
@@ -63,8 +63,6 @@ config INTEL_IOATDMA
         depends on PCI && X86
         select DMA_ENGINE
         select DCA
-       select ASYNC_TX_DISABLE_PQ_VAL_DMA
-       select ASYNC_TX_DISABLE_XOR_VAL_DMA
         help
           Enable support for the Intel(R) I/OAT DMA engine present
           in recent Intel Xeon chipsets.

[..]

Quoted 12 lines
drivers/dma/Kconfig +3 -4
@@ -1469,15 +1581,13 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
         device->cleanup_fn = ioat3_cleanup_event;
         device->timer_fn = ioat3_timer_event;
 
-       #ifdef CONFIG_ASYNC_TX_DISABLE_PQ_VAL_DMA
-       dma_cap_clear(DMA_PQ_VAL, dma->cap_mask);
-       dma->device_prep_dma_pq_val = NULL;
-       #endif
+       if (is_xeon_cb32(pdev)) {
+               dma_cap_clear(DMA_XOR_VAL, dma->cap_mask);
+               dma->device_prep_dma_xor_val = NULL;
 
Quoted 9 lines

drivers/dma/Kconfig int ioat3_dma_probe(struct ioatdma_device *device, int dca) · line 1590

Quoted 7 lines
drivers/dma/Kconfig +3 -4
-       #ifdef CONFIG_ASYNC_TX_DISABLE_XOR_VAL_DMA
-       dma_cap_clear(DMA_XOR_VAL, dma->cap_mask);
-       dma->device_prep_dma_xor_val = NULL;
-       #endif
+               dma_cap_clear(DMA_PQ_VAL, dma->cap_mask);
+               dma->device_prep_dma_pq_val = NULL;
+       }

Note that this effectively disables raid offload for is_xeon_cb32() platforms since DMA_ASYNC_TX will not be set.

We are ok with that until a workaround can be figured out.

author
#9 lore raw

PATCH 7/10 ioatdma: skip silicon bug workaround for pq_align for cb3.3

In reply to Dave Jiang

The alignment workaround is only necessary for cb3.2 or earlier platforms.

---
 drivers/dma/ioat/dma_v3.c |   12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
drivers/dma/ioat/dma_v3.c +10 -2
index a6d4175..a55c346 100644
@@ -1549,10 +1549,14 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 		dma_cap_set(DMA_XOR_VAL, dma->cap_mask);
 		dma->device_prep_dma_xor_val = ioat3_prep_xor_val;
 	}
+
 	if (cap & IOAT_CAP_PQ) {
 		is_raid_device = true;
 		dma_set_maxpq(dma, 8, 0);
-		dma->pq_align = 6;
+		if (is_xeon_cb32(pdev))
+			dma->pq_align = 6;
+		else
+			dma->pq_align = 0;
 
 		dma_cap_set(DMA_PQ, dma->cap_mask);
 		dma->device_prep_dma_pq = ioat3_prep_pq;
@@ -1562,7 +1566,10 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 
 		if (!(cap & IOAT_CAP_XOR)) {
 			dma->max_xor = 8;
-			dma->xor_align = 6;
+			if (is_xeon_cb32(pdev))
+				dma->xor_align = 6;
+			else
+				dma->xor_align = 0;
 
 			dma_cap_set(DMA_XOR, dma->cap_mask);
 			dma->device_prep_dma_xor = ioat3_prep_pqxor;
@@ -1571,6 +1578,7 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 			dma->device_prep_dma_xor_val = ioat3_prep_pqxor_val;
 		}
 	}
+
 	if (is_raid_device && (cap & IOAT_CAP_FILL_BLOCK)) {
 		dma_cap_set(DMA_MEMSET, dma->cap_mask);
 		dma->device_prep_dma_memset = ioat3_prep_memset_lock;
author
#10 lore raw

PATCH 8/10 ioatdma: Adding support for 16 src PQ ops and super extended descriptors

In reply to Dave Jiang

v3.3 introduced 16 sources PQ operations. This also introduced super extended descriptors to support the 16 srcs operations. This patch adds support for the 16 sources ops and in turn adds the super extended descriptors for those ops.

5 SED pools are created depending on the descriptor sizes. An SED can be a 64 bytes sized descriptor or larger and must be physically contiguous. A kmem cache pool is created for allocating the software descriptor that manages the hardware descriptor. The super extended descriptor will take place of extended descriptor under certain operations and be "attached" to the op descriptor during operation. This is a new feature for ioatdma v3.3.

---
 drivers/dma/ioat/dma.h       |   18 ++
 drivers/dma/ioat/dma_v2.h    |    2 
 drivers/dma/ioat/dma_v3.c    |  394 ++++++++++++++++++++++++++++++++++++++++--
 drivers/dma/ioat/hw.h        |   43 ++++-
 drivers/dma/ioat/pci.c       |    3 
 drivers/dma/ioat/registers.h |    1 
 6 files changed, 439 insertions(+), 22 deletions(-)
drivers/dma/ioat/dma.h +18 -0
index 4fbf5c7..8c88724 100644
@@ -81,6 +81,9 @@ struct ioatdma_device {
 	void __iomem *reg_base;
 	struct pci_pool *dma_pool;
 	struct pci_pool *completion_pool;
+#define MAX_SED_POOLS	5
+	struct dma_pool *sed_hw_pool[MAX_SED_POOLS];
+	struct kmem_cache *sed_pool;
 	struct dma_device common;
 	u8 version;
 	struct msix_entry msix_entries[4];
@@ -96,6 +99,7 @@ struct ioatdma_device {
 	int (*init_device)(struct ioatdma_device *device);
 };
 
+
 enum ioat_hwbugs {
 	IOAT_LEGACY_COMPLETION_REQUIRED = (1 << 0),
 };
@@ -149,6 +153,20 @@ struct ioat_dma_chan {
 	u16 active;
 };
 
+/**
+ * struct ioat_sed_ent - wrapper around super extended hardware descriptor
+ * @hw: hardware SED
+ * @sed_dma: dma address for the SED
+ * @list: list member
+ * @parent: point to the dma descriptor that's the parent
+ */
+struct ioat_sed_ent {
+	struct ioat_sed_raw_descriptor *hw;
+	dma_addr_t dma;
+	struct ioat_ring_ent *parent;
+	unsigned int hw_pool;
+};
+
 static inline struct ioat_chan_common *to_chan_common(struct dma_chan *c)
 {
 	return container_of(c, struct ioat_chan_common, common);
drivers/dma/ioat/dma_v2.h +2 -0
index e100f64..29bf944 100644
@@ -137,6 +137,7 @@ struct ioat_ring_ent {
 	#ifdef DEBUG
 	int id;
 	#endif
+	struct ioat_sed_ent *sed;
 };
 
 static inline struct ioat_ring_ent *
@@ -157,6 +158,7 @@ static inline void ioat2_set_chainaddr(struct ioat2_dma_chan *ioat, u64 addr)
 
 int ioat2_dma_probe(struct ioatdma_device *dev, int dca);
 int ioat3_dma_probe(struct ioatdma_device *dev, int dca);
+void ioat3_dma_remove(struct ioatdma_device *dev);
 struct dca_provider *ioat2_dca_init(struct pci_dev *pdev, void __iomem *iobase);
 struct dca_provider *ioat3_dca_init(struct pci_dev *pdev, void __iomem *iobase);
 int ioat2_check_space_lock(struct ioat2_dma_chan *ioat, int num_descs);
drivers/dma/ioat/dma_v3.c +373 -21

543 lines not on this page · raw message

drivers/dma/ioat/hw.h +42 -1

59 lines not on this page · raw message

drivers/dma/ioat/pci.c +3 -0
index 1f63296..2c8d560 100644
@@ -207,6 +207,9 @@ static void ioat_remove(struct pci_dev *pdev)
 	if (!device)
 		return;
 
+	if (device->version >= IOAT_VER_3_0)
+		ioat3_dma_remove(device);
+
 	dev_err(&pdev->dev, "Removing dma and dca services\n");
 	if (device->dca) {
 		unregister_dca_provider(device->dca, &pdev->dev);
drivers/dma/ioat/registers.h +1 -0
index c1ad194..efdd47e 100644
@@ -79,6 +79,7 @@
 #define IOAT_CAP_APIC				0x00000080
 #define IOAT_CAP_XOR				0x00000100
 #define IOAT_CAP_PQ				0x00000200
+#define IOAT_CAP_RAID16SS			0x00020000
 
 #define IOAT_CHANNEL_MMIO_SIZE			0x80	/* Each Channel MMIO space is this size */
 
author
#11 lore raw

PATCH 9/10 ioatdma: Adding write back descriptor error status support for ioatdma 3.3

In reply to Dave Jiang

v3.3 provides support for write back descriptor error status. This allows reporting of errors in a descriptor field. In supporting this, certain errors such as P/Q validation errors no longer halts the channel. The DMA engine can continue to execute until the end of the chain and allow software to report the "errors" up the stack. We are also going to mask those error interrupts and handle them when the "chain" has completed at the end.

---
 drivers/dma/ioat/dma_v3.c    |   87 ++++++++++++++++++++++++++++++++++++------
 drivers/dma/ioat/hw.h        |   17 +++++++-
 drivers/dma/ioat/registers.h |    1 
 3 files changed, 90 insertions(+), 15 deletions(-)
drivers/dma/ioat/dma_v3.c +74 -13
index 230a8bc..83d44f3 100644
@@ -498,6 +498,32 @@ static bool ioat3_cleanup_preamble(struct ioat_chan_common *chan,
 	return true;
 }
 
+static void desc_get_errstat(struct ioat_ring_ent *desc)
+{
+	struct ioat_dma_descriptor *hw = desc->hw;
+
+	switch (hw->ctl_f.op) {
+	case IOAT_OP_PQ_VAL:
+	case IOAT_OP_PQ_VAL_16S:
+	{
+		struct ioat_pq_descriptor *pq = desc->pq;
+
+		/* check if there's error written */
+		if (!pq->dwbes_f.wbes)
+			return;
+
+		if (pq->dwbes_f.p_val_err)
+			*desc->result |= SUM_CHECK_P_RESULT;
+
+		if (pq->dwbes_f.q_val_err)
+			*desc->result |= SUM_CHECK_Q_RESULT;
+		return;
+	}
+	default:
+		return;
+	}
+}
+
 /**
  * __cleanup - reclaim used descriptors
  * @ioat: channel (ring) to clean
@@ -535,6 +561,10 @@ static void __cleanup(struct ioat2_dma_chan *ioat, dma_addr_t phys_complete)
 		prefetch(ioat2_get_ring_ent(ioat, idx + i + 1));
 		desc = ioat2_get_ring_ent(ioat, idx + i);
 		dump_desc_dbg(ioat, desc);
+
+		/* set err stat if we are using dwbes */
+		desc_get_errstat(desc);
+
 		tx = &desc->txd;
 		if (tx->cookie) {
 			dma_cookie_complete(tx);
@@ -580,14 +610,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan *ioat)
 {
 	struct ioat_chan_common *chan = &ioat->base;
 	u64 phys_complete;
+	u32 chanerr;
 
 	spin_lock_bh(&chan->cleanup_lock);
 
 	if (ioat3_cleanup_preamble(chan, &phys_complete))
 		__cleanup(ioat, phys_complete);
 
+	chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 	if (is_ioat_halted(*chan->completion)) {
-		u32 chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 
 		if (chanerr & IOAT_CHANERR_HANDLE_MASK) {
 			mod_timer(&chan->timer, jiffies + IDLE_TIMEOUT);
@@ -595,6 +626,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan *ioat)
 		}
 	}
 
+	/*
+	 * with DWBES we must clear the chanerr register at the end of the
+	 * chain in order to be able to issue the next command.
+	 */
+	if (chanerr) {
+		writel(chanerr & IOAT_CHANERR_HANDLE_MASK,
+		       chan->reg_base + IOAT_CHANERR_OFFSET);
+	}
+
 	spin_unlock_bh(&chan->cleanup_lock);
 }
 
@@ -1077,6 +1117,8 @@ __ioat3_prep_pq_lock(struct dma_chan *c, enum sum_check_flags *result,
 		pq->q_addr = dst[1] + offset;
 		pq->ctl = 0;
 		pq->ctl_f.op = op;
+		/* we turn on descriptor write back error status */
+		pq->ctl_f.wb_en = result ? 1 : 0;
 		pq->ctl_f.src_cnt = src_cnt_to_hw(s);
 		pq->ctl_f.p_disable = !!(flags & DMA_PREP_PQ_DISABLE_P);
 		pq->ctl_f.q_disable = !!(flags & DMA_PREP_PQ_DISABLE_Q);
@@ -1193,6 +1235,8 @@ __ioat3_prep_pq16_lock(struct dma_chan *c, enum sum_check_flags *result,
 		pq->ctl = 0;
 		pq->ctl_f.op = op;
 		pq->ctl_f.src_cnt = src16_cnt_to_hw(s);
+		/* we turn on descriptor write back error status */
+		pq->ctl_f.wb_en = result ? 1 : 0;
 		pq->ctl_f.p_disable = !!(flags & DMA_PREP_PQ_DISABLE_P);
 		pq->ctl_f.q_disable = !!(flags & DMA_PREP_PQ_DISABLE_Q);
 
@@ -1785,9 +1829,28 @@ static int ioat3_init_device(struct ioatdma_device *device)
 	struct dma_device *dma;
 	struct dma_chan *c;
 	struct ioat_chan_common *chan;
+	u32 cap, errmask;
 
 	dma = &device->common;
 
+	cap = readl(device->reg_base + IOAT_DMA_CAP_OFFSET);
+
+	/*
+	 * if we have descriptor write back error status, we mask the
+	 * error interrupts
+	 */
+	if (cap & (IOAT_CAP_DWBES | IOAT_CAP_RAID16SS)) {
+		list_for_each_entry(c, &dma->channels, device_node) {
+			chan = to_chan_common(c);
+			errmask = readl(chan->reg_base +
+					IOAT_CHANERR_MASK_OFFSET);
+			errmask |= IOAT_CHANERR_XOR_P_OR_CRC_ERR |
+				   IOAT_CHANERR_XOR_Q_ERR;
+			writel(errmask, chan->reg_base +
+					IOAT_CHANERR_MASK_OFFSET);
+		}
+	}
+
 	list_for_each_entry(c, &dma->channels, device_node) {
 		if (is_xeon_cb32(pdev)) {
 			chan = to_chan_common(c);
@@ -1846,6 +1909,11 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 	if (cap & IOAT_CAP_PQ) {
 		is_raid_device = true;
 
+		dma->device_prep_dma_pq = ioat3_prep_pq;
+		dma->device_prep_dma_pq_val = ioat3_prep_pq_val;
+		dma_cap_set(DMA_PQ, dma->cap_mask);
+		dma_cap_set(DMA_PQ_VAL, dma->cap_mask);
+
 		if (cap & IOAT_CAP_RAID16SS) {
 			dma_set_maxpq(dma, 16, 0);
 			dma->pq_align = 0;
@@ -1857,13 +1925,12 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 				dma->pq_align = 0;
 		}
 
-		dma_cap_set(DMA_PQ, dma->cap_mask);
-		dma->device_prep_dma_pq = ioat3_prep_pq;
-
-		dma_cap_set(DMA_PQ_VAL, dma->cap_mask);
-		dma->device_prep_dma_pq_val = ioat3_prep_pq_val;
-
 		if (!(cap & IOAT_CAP_XOR)) {
+			dma->device_prep_dma_xor = ioat3_prep_pqxor;
+			dma->device_prep_dma_xor_val = ioat3_prep_pqxor_val;
+			dma_cap_set(DMA_XOR, dma->cap_mask);
+			dma_cap_set(DMA_XOR_VAL, dma->cap_mask);
+
 			if (cap & IOAT_CAP_RAID16SS) {
 				dma->max_xor = 16;
 				dma->xor_align = 0;
@@ -1874,12 +1941,6 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 				else
 					dma->xor_align = 0;
 			}
-
-			dma_cap_set(DMA_XOR, dma->cap_mask);
-			dma->device_prep_dma_xor = ioat3_prep_pqxor;
-
-			dma_cap_set(DMA_XOR_VAL, dma->cap_mask);
-			dma->device_prep_dma_xor_val = ioat3_prep_pqxor_val;
 		}
 	}
 
drivers/dma/ioat/hw.h +15 -2
index d10570d..5ee57d4 100644
@@ -165,7 +165,17 @@ struct ioat_xor_ext_descriptor {
 };
 
 struct ioat_pq_descriptor {
-	uint32_t	size;
+	union {
+		uint32_t	size;
+		uint32_t	dwbes;
+		struct {
+			unsigned int rsvd:25;
+			unsigned int p_val_err:1;
+			unsigned int q_val_err:1;
+			unsigned int rsvd1:4;
+			unsigned int wbes:1;
+		} dwbes_f;
+	};
 	union {
 		uint32_t ctl;
 		struct {
@@ -180,7 +190,10 @@ struct ioat_pq_descriptor {
 			unsigned int hint:1;
 			unsigned int p_disable:1;
 			unsigned int q_disable:1;
-			unsigned int rsvd:11;
+			unsigned int rsvd2:2;
+			unsigned int wb_en:1;
+			unsigned int prl_en:1;
+			unsigned int rsvd3:7;
 			#define IOAT_OP_PQ 0x89
 			#define IOAT_OP_PQ_VAL 0x8a
 			#define IOAT_OP_PQ_16S 0xa0
drivers/dma/ioat/registers.h +1 -0
index efdd47e..2f1cfa0 100644
@@ -79,6 +79,7 @@
 #define IOAT_CAP_APIC				0x00000080
 #define IOAT_CAP_XOR				0x00000100
 #define IOAT_CAP_PQ				0x00000200
+#define IOAT_CAP_DWBES				0x00002000
 #define IOAT_CAP_RAID16SS			0x00020000
 
 #define IOAT_CHANNEL_MMIO_SIZE			0x80	/* Each Channel MMIO space is this size */
#12 lore raw

Re: [PATCH 09/10] ioatdma: Adding write back descriptor error status support for ioatdma 3.3

Quoted 83 lines · Dave Jiang

v3.3 provides support for write back descriptor error status. This allows reporting of errors in a descriptor field. In supporting this, certain errors such as P/Q validation errors no longer halts the channel. The DMA engine can continue to execute until the end of the chain and allow software
to report the "errors" up the stack. We are also going to mask those error interrupts and handle them when the "chain" has completed at the end.

---
drivers/dma/ioat/dma_v3.c | 87
++++++++++++++++++++++++++++++++++++------
drivers/dma/ioat/hw.h | 17 +++++++-
drivers/dma/ioat/registers.h | 1

3 files changed, 90 insertions(+), 15 deletions(-)
drivers/dma/ioat/dma_v3.c +26 -0
index 230a8bc..83d44f3 100644
@@ -498,6 +498,32 @@ static bool ioat3_cleanup_preamble(struct
 ioat_chan_common *chan,
 	return true;
 }
 
+static void desc_get_errstat(struct ioat_ring_ent *desc)
+{
+	struct ioat_dma_descriptor *hw = desc->hw;
+
+	switch (hw->ctl_f.op) {
+	case IOAT_OP_PQ_VAL:
+	case IOAT_OP_PQ_VAL_16S:
+	{
+		struct ioat_pq_descriptor *pq = desc->pq;
+
+		/* check if there's error written */
+		if (!pq->dwbes_f.wbes)
+			return;
+
+		if (pq->dwbes_f.p_val_err)
+			*desc->result |= SUM_CHECK_P_RESULT;
+
+		if (pq->dwbes_f.q_val_err)
+			*desc->result |= SUM_CHECK_Q_RESULT;
+		return;
+	}
+	default:
+		return;
+	}
+}
+
 /**
 * __cleanup - reclaim used descriptors

* @ioat: channel (ring) to clean

drivers/dma/ioat/dma_v3.c +4 -0
@@ -535,6 +561,10 @@ static void __cleanup(struct ioat2_dma_chan *ioat,
 dma_addr_t phys_complete)
 		prefetch(ioat2_get_ring_ent(ioat, idx + i + 1));
 		desc = ioat2_get_ring_ent(ioat, idx + i);
 		dump_desc_dbg(ioat, desc);
+
+		/* set err stat if we are using dwbes */
+		desc_get_errstat(desc);
+
 		tx = &desc->txd;
 		if (tx->cookie) {
			dma_cookie_complete(tx);
drivers/dma/ioat/dma_v3.c +2 -1
@@ -580,14 +610,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan
 *ioat)
 {
 	struct ioat_chan_common *chan = &ioat->base;
 	u64 phys_complete;
+	u32 chanerr;
 
 	spin_lock_bh(&chan->cleanup_lock);
 
 	if (ioat3_cleanup_preamble(chan, &phys_complete))
 		__cleanup(ioat, phys_complete);
 
+	chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 	if (is_ioat_halted(*chan->completion)) {
-		u32 chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 
 		if (chanerr & IOAT_CHANERR_HANDLE_MASK) {
Quoted 1 line
			mod_timer(&chan->timer, jiffies + IDLE_TIMEOUT);

This is now incurring a mmio read for every cleanup which somewhat defeats the point of posting the completion status to memory. Should be able to get away with just writel(IOAT_CHANERR_HANDLE_MASK) iff cleanup found came across an error-writeback.

Quoted 4 lines
drivers/dma/ioat/dma_v3.c +0 -0
@@ -595,6 +626,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan
 *ioat)
 		}
 	}

drivers/dma/ioat/dma_v3.c static void ioat3_cleanup(struct ioat2_dma_chan · line 638

Quoted 10 lines
drivers/dma/ioat/dma_v3.c +9 -0
 
+	/*
+	 * with DWBES we must clear the chanerr register at the end of the
+	 * chain in order to be able to issue the next command.
+	 */
+	if (chanerr) {
+		writel(chanerr & IOAT_CHANERR_HANDLE_MASK,
+		       chan->reg_base + IOAT_CHANERR_OFFSET);
+	}
+

Does this also mean we need to re-write dmacount? I.e. are writes to
dmacount ignored while chanerr is non-zero?

author
#13 lore raw

Re: [PATCH 09/10] ioatdma: Adding write back descriptor error status support for ioatdma 3.3

Quoted 85 lines · Dan Williams
Quoted 82 lines · Dave Jiang

v3.3 provides support for write back descriptor error status. This allows reporting of errors in a descriptor field. In supporting this, certain errors such as P/Q validation errors no longer halts the channel. The DMA engine can continue to execute until the end of the chain and allow software
to report the "errors" up the stack. We are also going to mask those error interrupts and handle them when the "chain" has completed at the end.

---
drivers/dma/ioat/dma_v3.c | 87
++++++++++++++++++++++++++++++++++++------
drivers/dma/ioat/hw.h | 17 +++++++-
drivers/dma/ioat/registers.h | 1

3 files changed, 90 insertions(+), 15 deletions(-)
drivers/dma/ioat/dma_v3.c +26 -0
index 230a8bc..83d44f3 100644
@@ -498,6 +498,32 @@ static bool ioat3_cleanup_preamble(struct
 ioat_chan_common *chan,
 	return true;
 }
 
+static void desc_get_errstat(struct ioat_ring_ent *desc)
+{
+	struct ioat_dma_descriptor *hw = desc->hw;
+
+	switch (hw->ctl_f.op) {
+	case IOAT_OP_PQ_VAL:
+	case IOAT_OP_PQ_VAL_16S:
+	{
+		struct ioat_pq_descriptor *pq = desc->pq;
+
+		/* check if there's error written */
+		if (!pq->dwbes_f.wbes)
+			return;
+
+		if (pq->dwbes_f.p_val_err)
+			*desc->result |= SUM_CHECK_P_RESULT;
+
+		if (pq->dwbes_f.q_val_err)
+			*desc->result |= SUM_CHECK_Q_RESULT;
+		return;
+	}
+	default:
+		return;
+	}
+}
+
 /**
  * __cleanup - reclaim used descriptors

* @ioat: channel (ring) to clean

drivers/dma/ioat/dma_v3.c +4 -0
@@ -535,6 +561,10 @@ static void __cleanup(struct ioat2_dma_chan *ioat,
 dma_addr_t phys_complete)
 		prefetch(ioat2_get_ring_ent(ioat, idx + i + 1));
 		desc = ioat2_get_ring_ent(ioat, idx + i);
 		dump_desc_dbg(ioat, desc);
+
+		/* set err stat if we are using dwbes */
+		desc_get_errstat(desc);
+
 		tx = &desc->txd;
 		if (tx->cookie) {
			dma_cookie_complete(tx);
drivers/dma/ioat/dma_v3.c +2 -1
@@ -580,14 +610,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan
 *ioat)
 {
 	struct ioat_chan_common *chan = &ioat->base;
 	u64 phys_complete;
+	u32 chanerr;
 
 	spin_lock_bh(&chan->cleanup_lock);
 
 	if (ioat3_cleanup_preamble(chan, &phys_complete))
 		__cleanup(ioat, phys_complete);
 
+	chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 	if (is_ioat_halted(*chan->completion)) {
-		u32 chanerr = readl(chan->reg_base + IOAT_CHANERR_OFFSET);
 
Quoted 6 lines
Quoted 2 lines
drivers/dma/ioat/dma_v3.c +0 -0
 		if (chanerr & IOAT_CHANERR_HANDLE_MASK) {
			mod_timer(&chan->timer, jiffies + IDLE_TIMEOUT);

This is now incurring a mmio read for every cleanup which somewhat defeats the point of posting the completion status to memory. Should be able to get away with just writel(IOAT_CHANERR_HANDLE_MASK) iff cleanup found came across an error-writeback.

Ok I'll fix that.

Quoted 7 lines
Quoted 7 lines
drivers/dma/ioat/dma_v3.c +0 -0
@@ -595,6 +626,15 @@ static void ioat3_cleanup(struct ioat2_dma_chan
 *ioat)
 		}
 	}
 
Quoted 12 lines

drivers/dma/ioat/dma_v3.c static void ioat3_cleanup(struct ioat2_dma_chan *ioat) · line 637

Quoted 9 lines
drivers/dma/ioat/dma_v3.c +9 -0
+	/*
+	 * with DWBES we must clear the chanerr register at the end of the
+	 * chain in order to be able to issue the next command.
+	 */
+	if (chanerr) {
+		writel(chanerr & IOAT_CHANERR_HANDLE_MASK,
+		       chan->reg_base + IOAT_CHANERR_OFFSET);
+	}
+

Does this also mean we need to re-write dmacount? I.e. are writes to
dmacount ignored while chanerr is non-zero?

I don't believe so. At least that hasn't been an issue with testing. I have asked them to remove that "feature" with future silicon.

#14 lore raw

PATCH v2 9/10 ioatdma: Adding write back descriptor error status support for ioatdma 3.3

In reply to Dan Williams

v3.3 provides support for write back descriptor error status. This allows reporting of errors in a descriptor field. In supporting this, certain errors such as P/Q validation errors no longer halts the channel. The DMA engine can continue to execute until the end of the chain and allow software to report the "errors" up the stack. We are also going to mask those error interrupts and handle them when the "chain" has completed at the end.

---

Dan, I removed the issue workaround since I can no longer reproduce the issue on the latest stepping and the design engineers say it's not needed.

 drivers/dma/ioat/dma.h       |   1 +
 drivers/dma/ioat/dma_v3.c    | 102 +++++++++++++++++++++++++++++++++----------
 drivers/dma/ioat/hw.h        |  17 +++++++-
 drivers/dma/ioat/registers.h |   1 +
 4 files changed, 96 insertions(+), 25 deletions(-)
drivers/dma/ioat/dma.h +1 -0
index 8c88724..53c15d2 100644
@@ -90,6 +90,7 @@ struct ioatdma_device {
 	struct ioat_chan_common *idx[4];
 	struct dca_provider *dca;
 	enum ioat_irq_mode irq_mode;
+	u32 cap;
 	void (*intr_quirk)(struct ioatdma_device *device);
 	int (*enumerate_channels)(struct ioatdma_device *device);
 	int (*reset_hw)(struct ioat_chan_common *chan);
drivers/dma/ioat/dma_v3.c +79 -23

188 lines not on this page · raw message

drivers/dma/ioat/hw.h +15 -2
index d10570d..5ee57d4 100644
@@ -165,7 +165,17 @@ struct ioat_xor_ext_descriptor {
 };
 
 struct ioat_pq_descriptor {
-	uint32_t	size;
+	union {
+		uint32_t	size;
+		uint32_t	dwbes;
+		struct {
+			unsigned int rsvd:25;
+			unsigned int p_val_err:1;
+			unsigned int q_val_err:1;
+			unsigned int rsvd1:4;
+			unsigned int wbes:1;
+		} dwbes_f;
+	};
 	union {
 		uint32_t ctl;
 		struct {
@@ -180,7 +190,10 @@ struct ioat_pq_descriptor {
 			unsigned int hint:1;
 			unsigned int p_disable:1;
 			unsigned int q_disable:1;
-			unsigned int rsvd:11;
+			unsigned int rsvd2:2;
+			unsigned int wb_en:1;
+			unsigned int prl_en:1;
+			unsigned int rsvd3:7;
 			#define IOAT_OP_PQ 0x89
 			#define IOAT_OP_PQ_VAL 0x8a
 			#define IOAT_OP_PQ_16S 0xa0
drivers/dma/ioat/registers.h +1 -0
index efdd47e..2f1cfa0 100644
@@ -79,6 +79,7 @@
 #define IOAT_CAP_APIC				0x00000080
 #define IOAT_CAP_XOR				0x00000100
 #define IOAT_CAP_PQ				0x00000200
+#define IOAT_CAP_DWBES				0x00002000
 #define IOAT_CAP_RAID16SS			0x00020000
 
 #define IOAT_CHANNEL_MMIO_SIZE			0x80	/* Each Channel MMIO space is this size */
Signature · 1 line
1.8.1.4
author
#15 lore raw

PATCH 10/10 ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap

In reply to Dave Jiang

This workaround checks for channel 2&3 and remove RAID cap.

---
 drivers/dma/ioat/dma_v3.c |   15 +++++++++++++++
 1 file changed, 15 insertions(+)
drivers/dma/ioat/dma_v3.c +15 -0
index 83d44f3..6f6d2ec 100644
@@ -238,6 +238,18 @@ static bool is_bwd_ioat(struct pci_dev *pdev)
 	}
 }
 
+static bool is_bwd_noraid(struct pci_dev *pdev)
+{
+	switch (pdev->device) {
+	case PCI_DEVICE_ID_INTEL_IOAT_BWD2:
+	case PCI_DEVICE_ID_INTEL_IOAT_BWD3:
+		return true;
+	default:
+		return false;
+	}
+
+}
+
 static void pq16_set_src(struct ioat_raw_descriptor *desc[3],
 			dma_addr_t addr, u32 offset, u8 coef, int idx)
 {
@@ -1890,6 +1902,9 @@ int ioat3_dma_probe(struct ioatdma_device *device, int dca)
 
 	cap = readl(device->reg_base + IOAT_DMA_CAP_OFFSET);
 
+	if (is_bwd_noraid(pdev))
+		cap &= ~(IOAT_CAP_XOR | IOAT_CAP_PQ | IOAT_CAP_RAID16SS);
+
 	/* dca is incompatible with raid operations */
 	if (dca_en && (cap & (IOAT_CAP_XOR|IOAT_CAP_PQ)))
 		cap &= ~(IOAT_CAP_XOR|IOAT_CAP_PQ);
#16 lore raw
In reply to Dave Jiang
Quoted 6 lines · Dave Jiang

The following series adds support for the Intel Atom S1200 product family ioatdma. This ioatdma also implements a set of version v3.3 features such as 16 sources PQ, descriptor write back error status, and does not have many of the silicon bugs that the 3.2 line of hardware has due to a brand new implemention.

The series is dependent on the haswell update patch sent prior.

Hey Dan,

Are you okay with the series. Merge window is very close...

Quoted 28 lines

---

Dave Jiang (10):
ioatdma: Adding PCI IDs for Intel Atom S1200 product family ioatdma devices
ioatdma: Add 64bit chansts register read for ioat v3.3.
ioatdma: channel reset scheme fixup on Intel Atom S1200 platforms
ioatdma: Removing hw bug workaround for CB3.x .2 and earlier
ioatdma: skip legacy reset bits since v3.3 plattform doesn't need it
ioatdma: Removing PQ val disable for cb3.3
ioatdma: skip silicon bug workaround for pq_align for cb3.3
ioatdma: Adding support for 16 src PQ ops and super extended descriptors
ioatdma: Adding write back descriptor error status support for ioatdma 3.3
ioatdma: S1200 platforms ioatdma channel 2 and 3 falsely advertise RAID cap

 drivers/dma/Kconfig          |    2 
 drivers/dma/ioat/dma.c       |   14 +
 drivers/dma/ioat/dma.h       |   59 +++
 drivers/dma/ioat/dma_v2.h    |    2 
 drivers/dma/ioat/dma_v3.c    |  930 ++++++++++++++++++++++++++++++++++++------
 drivers/dma/ioat/hw.h        |   66 +++
 drivers/dma/ioat/pci.c       |    9 
 drivers/dma/ioat/registers.h |    4 
 8 files changed, 941 insertions(+), 145 deletions(-)
Signature · 1 line
Signature
#17 lore raw
Quoted 11 lines · Vinod Koul
Quoted 6 lines · Dave Jiang

The following series adds support for the Intel Atom S1200 product family ioatdma. This ioatdma also implements a set of version v3.3 features such as 16 sources PQ, descriptor write back error status, and does not have many of the silicon bugs that the 3.2 line of hardware has due to a brand new implemention.

The series is dependent on the haswell update patch sent prior.

Hey Dan,

Are you okay with the series. Merge window is very close...

[resend, last one got html formatted/rejected]

Hi Vinod, thanks for the ping.

Chatted with Dave patches 1-3 and 5-8, and 10 are reviewed/acked. For patch 4 and 9 we think we have a slightly cleaner way to organize the quirk handling and an update will be coming.

--
Dan

A message that is not in the archive (sent off-list or not yet received): CAA9_cmftcvYN_g0Sj12tMa4T8LxbPdk6wDxwkbTrgQPUUfc0FA@mail.gmail.com

#19 lore raw
Quoted 9 lines · Dan Williams
Quoted 3 lines · Vinod Koul

Are you okay with the series. Merge window is very close...

Hi Vinod, thanks for the ping.

Chatted with Dave patches 1-3 and 5-8, and 10 are reviewed/acked. For patch 4 and 9 we think we have a slightly cleaner way to organize the quirk handling and an update will be coming.

Dave,

I tried these, but they fail to apply on for-linus branch of my tree. Cna you pls rebase these 8 patches and resend.

--
~Vinod

#20 lore raw
Quoted 10 lines · Vinod
Quoted 9 lines · Dan Williams
Quoted 3 lines · Vinod Koul

Are you okay with the series. Merge window is very close...

Hi Vinod, thanks for the ping.

Chatted with Dave patches 1-3 and 5-8, and 10 are reviewed/acked. For patch 4 and 9 we think we have a slightly cleaner way to organize the quirk handling and an update will be coming.

Quoted 7 lines

Dave,

I tried these, but they fail to apply on for-linus branch of my tree. Cna you pls rebase these 8 patches and resend.

--
~Vinod

Yes I will rebase and send them with the other two changed patches. It depends on the haswell patch. Could it be that?

#21 lore raw
In reply to Vinod Koul
Quoted 7 lines · Vinod Koul
Quoted 6 lines · Dan Williams
Quoted 3 lines · Vinod Koul

Are you okay with the series. Merge window is very close...

Hi Vinod, thanks for the ping.

Quoted 7 lines
Quoted 3 lines

Chatted with Dave patches 1-3 and 5-8, and 10 are reviewed/acked. For patch 4 and 9 we think we have a slightly cleaner way to organize the quirk handling and an update will be coming.

Dave,

I tried these, but they fail to apply on for-linus branch of my tree. Cna you pls rebase these 8 patches and resend.

Okay i forgot about the HSW ID patch. Applied that and then these worked till last two, you would need to rebase those two only

Have applied rest
--
~Vinod