Skip to contentSkip to conversation

Enable dma driver for MIC X100 Coprocessors.

PATCH In review Siva Yerramreddy 4 participants 14 messages Apr 14, 2014 – May 28, 2014 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Enable dma driver for MIC X100 Coprocessors.

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Description:
============
This set of patches add support for MIC X100 dma driver.
MIC PCIe card has a dma controller with 8 channels. These channels are shared between the host s/w and the card s/w. 0 to 3 are used by host and 4 to 7 by card. As the dma device doesn't show up as PCIe device, a virtual bus called mic bus is created and virtual dma devices are created on it by the host/card drivers. On host the channels are private and used only by the host driver to transfer data for the virtio devices.

Here is a higher level block diagram.

				      |
	       +----------+           |             +----------+
	       | Card OS  |           |             | Host OS  |
	       +----------+           |             +----------+
				      |
	+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
	| Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
	| Net   | |Console | |Block | | |Net      |  |Console | |Block   |
	| Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
	+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
	    |         |         |     |      |            |         |
	    |         |         |     |User  |            |         |
	    |         |         |     |------|------------|---------|-------
	    +-------------------+     |Kernel +--------------------------+
		      |               |       | Virtio over PCIe IOCTLs  |
		      |               |       +--------------------------+
+-----------+	      |               |	    		  |  +-----------+
| MIC DMA   |	      |               | 		  |  | MIC DMA	 |
| Driver    |         |               |	  		  |  | Driver	 |
+-----------+         |               |			  |  +-----------+
      |		      |	   	      |			  |	   |
+---------------+     |		      |			  |  +----------------+
|MIC virtual Bus|     |	 	      |			  |  |MIC virtual Bus |
+---------------+     |		      |			  |  +----------------+
      |               |               |                   |		 |
      |   +--------------+	      |            +---------------+     |
      |   |Intel MIC     |	      |            |Intel MIC      |     |
      +---|Card Driver   |            |            |Host Driver    |     |
	  +--------------+            |            +---------------+-----+
		      |               |                   |
	     +-------------------------------------------------------------+
	     |                                                             |
	     |                    PCIe Bus                                 |
	     +-------------------------------------------------------------+

The following series of patches are partitioned as follows:

Patch 1: Add mic bus and dma driver documentation.
Author: Siva Yerramreddy
Patch 2: Add a bus driver for virtual MIC devices.
Authors: Siva Yerramreddy, Sudeep Dutt
Patch 3: MIC X100 DMA Driver.
Author: Siva Yerramreddy
Patch 4: Add threaded irq support in host driver.
This is needed as the dma driver uses threaded irq.
Author: Siva Yerramreddy
Patch 5: Add dma support in host driver.
Authors: Siva Yerramreddy, Ashutosh Dixit, Sudeep Dutt
Patch 6: Add threaded irq support in card driver.
This is needed as the dma driver uses threaded irq.
Author: Siva Yerramreddy
Patch 7: Add dma support in card driver.
Author: Siva Yerramreddy
Patch 8: Add support for loading/unloading dma driver.
Author: Siva Yerramreddy

The patches have been compiled/validated against v3.14. Tested using dmatest module with module parameter "threads_per_chan=60".

Thanks to Dan Williams, Vinod Koul, Jon Mason, Dave Jiang for the initial review.

Siva Yerramreddy (8):
misc: mic: Add mic bus and dma driver documentation
misc: mic: add a bus driver for virtual MIC devices
dma: MIC X100 DMA Driver
misc: mic: add threaded irq support in host driver
misc: mic: add dma support in host driver
misc: mic: add threaded irq support in card driver
misc: mic: add dma support in card driver
misc: mic: add support for loading/unloading dma driver

 Documentation/mic/mic_overview.txt |  67 ++--
 Documentation/mic/mpssd/mpss       |  14 +-
 drivers/dma/Kconfig                |  19 +
 drivers/dma/Makefile               |   1 +
 drivers/dma/mic_x100_dma.c         | 774 +++++++++++++++++++++++++++++++++++++
 drivers/dma/mic_x100_dma.h         | 286 ++++++++++++++
 drivers/misc/mic/Kconfig           |  21 +-
 drivers/misc/mic/Makefile          |   1 +
 drivers/misc/mic/bus/Makefile      |   5 +
 drivers/misc/mic/bus/mic_bus.c     | 188 +++++++++
 drivers/misc/mic/card/mic_device.c |  21 +-
 drivers/misc/mic/card/mic_device.h |   8 +-
 drivers/misc/mic/card/mic_virtio.c |   4 +-
 drivers/misc/mic/card/mic_x100.c   |  52 ++-
 drivers/misc/mic/host/mic_boot.c   |  78 +++-
 drivers/misc/mic/host/mic_device.h |  24 ++
 drivers/misc/mic/host/mic_intr.c   | 116 +++---
 drivers/misc/mic/host/mic_intr.h   |  21 +-
 drivers/misc/mic/host/mic_main.c   |   5 +-
 drivers/misc/mic/host/mic_virtio.c | 185 +++++++--
 drivers/misc/mic/host/mic_virtio.h |  21 +-
 drivers/misc/mic/host/mic_x100.c   |   8 +
 include/linux/mic_bus.h            | 148 +++++++
 23 files changed, 1922 insertions(+), 145 deletions(-)
 create mode 100644 drivers/dma/mic_x100_dma.c
 create mode 100644 drivers/dma/mic_x100_dma.h
 create mode 100644 drivers/misc/mic/bus/Makefile
 create mode 100644 drivers/misc/mic/bus/mic_bus.c
 create mode 100644 include/linux/mic_bus.h
Signature · 1 line
1.8.2.2
#1 lore raw

PATCH 1/8 misc: mic: Add mic bus and dma driver documentation

Added an overview of mic bus and dma driver.

---
 Documentation/mic/mic_overview.txt | 67 +++++++++++++++++++++++---------------
 1 file changed, 41 insertions(+), 26 deletions(-)
Documentation/mic/mic_overview.txt +41 -26
index b419292..77c5418 100644
@@ -17,35 +17,50 @@ for applications. A key benefit of our solution is that it leverages
 the standard virtio framework for network, disk and console devices,
 though in our case the virtio framework is used across a PCIe bus.
 
+MIC PCIe card has a dma controller with 8 channels. These channels are
+shared between the host s/w and the card s/w. 0 to 3 are used by host
+and 4 to 7 by card. As the dma device doesn't show up as PCIe device,
+a virtual bus called mic bus is created and virtual dma devices are
+created on it by the host/card drivers. On host the channels are private
+and used only by the host driver to transfer data for the virtio devices.
+
 Here is a block diagram of the various components described above. The
 virtio backends are situated on the host rather than the card given better
 single threaded performance for the host compared to MIC, the ability of
 the host to initiate DMA's to/from the card using the MIC DMA engine and
 the fact that the virtio block storage backend can only be on the host.
 
-                              |
-       +----------+           |             +----------+
-       | Card OS  |           |             | Host OS  |
-       +----------+           |             +----------+
-                              |
-+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
-| Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
-| Net   | |Console | |Block | | |Net      |  |Console | |Block   |
-| Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
-+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
-    |         |         |     |      |            |         |
-    |         |         |     |User  |            |         |
-    |         |         |     |------|------------|---------|-------
-    +-------------------+     |Kernel +--------------------------+
-              |               |       | Virtio over PCIe IOCTLs  |
-              |               |       +--------------------------+
-      +--------------+        |                   |
-      |Intel MIC     |        |            +---------------+
-      |Card Driver   |        |            |Intel MIC      |
-      +--------------+        |            |Host Driver    |
-              |               |            +---------------+
-              |               |                   |
-     +-------------------------------------------------------------+
-     |                                                             |
-     |                    PCIe Bus                                 |
-     +-------------------------------------------------------------+
+                                      |
+               +----------+           |             +----------+
+               | Card OS  |           |             | Host OS  |
+               +----------+           |             +----------+
+                                      |
+        +-------+ +--------+ +------+ | +---------+  +--------+ +--------+
+        | Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
+        | Net   | |Console | |Block | | |Net      |  |Console | |Block   |
+        | Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
+        +-------+ +--------+ +------+ | +---------+  +--------+ +--------+
+            |         |         |     |      |            |         |
+            |         |         |     |User  |            |         |
+            |         |         |     |------|------------|---------|-------
+            +-------------------+     |Kernel +--------------------------+
+                      |               |       | Virtio over PCIe IOCTLs  |
+                      |               |       +--------------------------+
++-----------+         |               |                   |  +-----------+
+| MIC DMA   |         |               |                   |  | MIC DMA   |
+| Driver    |         |               |                   |  | Driver    |
++-----------+         |               |                   |  +-----------+
+      |               |               |                   |        |
++---------------+     |               |                   |  +----------------+
+|MIC virtual Bus|     |               |                   |  |MIC virtual Bus |
++---------------+     |               |                   |  +----------------+
+      |               |               |                   |              |
+      |   +--------------+            |            +---------------+     |
+      |   |Intel MIC     |            |            |Intel MIC      |     |
+      +---|Card Driver   |            |            |Host Driver    |     |
+          +--------------+            |            +---------------+-----+
+                      |               |                   |
+             +-------------------------------------------------------------+
+             |                                                             |
+             |                    PCIe Bus                                 |
+             +-------------------------------------------------------------+
Signature · 1 line
1.8.2.2
#2 lore raw

PATCH 2/8 misc: mic: add a bus driver for virtual MIC devices

In reply to Siva Yerramreddy

This MIC virtual bus driver takes the responsibility of creating all the virtual devices connected to the PCIe device on the host and the platform device on the card. The MIC bus hardware operations provide a way to abstract certain hardware details from the base physical devices. Examples of devices added on the MIC virtual bus include host DMA and card DMA. This abstraction enables using a common DMA driver on host and card.

---
 drivers/misc/mic/Kconfig       |  17 ++++
 drivers/misc/mic/Makefile      |   1 +
 drivers/misc/mic/bus/Makefile  |   5 ++
 drivers/misc/mic/bus/mic_bus.c | 188 +++++++++++++++++++++++++++++++++++++++++
 include/linux/mic_bus.h        | 148 ++++++++++++++++++++++++++++++++
 5 files changed, 359 insertions(+)
 create mode 100644 drivers/misc/mic/bus/Makefile
 create mode 100644 drivers/misc/mic/bus/mic_bus.c
 create mode 100644 include/linux/mic_bus.h
drivers/misc/mic/Kconfig +17 -0
index e42b331..cfe185f 100644
@@ -1,3 +1,20 @@
+comment "Intel MIC Bus Driver"
+
+config INTEL_MIC_BUS
+	tristate "Intel MIC Bus Driver"
+	depends on 64BIT && PCI && X86 && X86_DEV_DMA_OPS
+	help
+	  This option is selected by any driver which registers a
+	  device or driver on the MIC Bus, such as CONFIG_INTEL_MIC_HOST,
+	  CONFIG_INTEL_MIC_CARD, CONFIG_INTEL_MIC_X100_DMA etc.
+
+	  If you are building a host/card kernel with an Intel MIC device
+	  then say M (recommended) or Y, else say N. If unsure say N.
+
+	  More information about the Intel MIC family as well as the Linux
+	  OS and tools for MIC to use with this driver are available from
+	  <http://software.intel.com/en-us/mic-developer>.
+
 comment "Intel MIC Host Driver"
 
 config INTEL_MIC_HOST
drivers/misc/mic/Makefile +1 -0
index 05b34d6..e9bf148 100644
@@ -4,3 +4,4 @@
 #
 obj-$(CONFIG_INTEL_MIC_HOST) += host/
 obj-$(CONFIG_INTEL_MIC_CARD) += card/
+obj-$(CONFIG_INTEL_MIC_BUS) += bus/
drivers/misc/mic/bus/Makefile new file +5 -0
new file mode 100644
index 0000000..d85c7f2
@@ -0,0 +1,5 @@
+#
+# Makefile - Intel MIC Linux driver.
+# Copyright(c) 2014, Intel Corporation.
+#
+obj-$(CONFIG_INTEL_MIC_BUS) += mic_bus.o
drivers/misc/mic/bus/mic_bus.c new file +188 -0

188 lines not on this page · raw message

include/linux/mic_bus.h new file +148 -0

148 lines not on this page · raw message

Signature · 1 line
1.8.2.2
#3 lore raw

PATCH 3/8 dma: MIC X100 DMA Driver

In reply to Siva Yerramreddy

This patch implements DMA Engine API for DMA controller on MIC X100 Coprocessors. DMA h/w is shared between host and card s/w.
Channels 0 to 3 are used by host and 4 to 7 are used by card.
Since the DMA device doesn't show up as PCIe device, a virtual bus called mic bus is created and virtual devices are added on that bus to follow device model. Allowed dma transfer directions are host to card, card to host and card to card.

---
 drivers/dma/Kconfig        |  19 ++
 drivers/dma/Makefile       |   1 +
 drivers/dma/mic_x100_dma.c | 774 +++++++++++++++++++++++++++++++++++++++++++++
 drivers/dma/mic_x100_dma.h | 286 +++++++++++++++++
 4 files changed, 1080 insertions(+)
 create mode 100644 drivers/dma/mic_x100_dma.c
 create mode 100644 drivers/dma/mic_x100_dma.h
drivers/dma/Kconfig +19 -0
index 605b016..f9747ef 100644
@@ -33,6 +33,25 @@ if DMADEVICES
 
 comment "DMA Devices"
 
+config INTEL_MIC_X100_DMA
+	tristate "Intel MIC X100 DMA Driver"
+	depends on 64BIT && X86 && INTEL_MIC_BUS
+	select DMAENGINE
+	default N
+	help
+	  This enables DMA support for the Intel Many Integrated Core
+	  (MIC) family of PCIe form factor coprocessor X100 devices that
+	  run a 64 bit Linux OS. This driver will be used by both MIC
+	  host and card drivers.
+
+	  If you are building host kernel with a MIC device or a card
+	  kernel for a MIC device, then say M (recommended) or Y, else
+	  say N. If unsure say N.
+
+	  More information about the Intel MIC family as well as the Linux
+	  OS and tools for MIC to use with this driver are available from
+	  <http://software.intel.com/en-us/mic-developer>.
+
 config INTEL_MID_DMAC
 	tristate "Intel MID DMA support for Peripheral DMA controllers"
 	depends on PCI && X86
drivers/dma/Makefile +1 -0
index a029d0f4..4994cb2 100644
@@ -44,3 +44,4 @@ obj-$(CONFIG_DMA_JZ4740) += dma-jz4740.o
 obj-$(CONFIG_TI_CPPI41) += cppi41.o
 obj-$(CONFIG_K3_DMA) += k3dma.o
 obj-$(CONFIG_MOXART_DMA) += moxart-dma.o
+obj-$(CONFIG_INTEL_MIC_X100_DMA) += mic_x100_dma.o
drivers/dma/mic_x100_dma.c new file +774 -0

774 lines not on this page · raw message

drivers/dma/mic_x100_dma.h new file +286 -0

286 lines not on this page · raw message

Signature · 1 line
1.8.2.2
#4 lore raw

PATCH 4/8 misc: mic: add threaded irq support in host driver

In reply to Siva Yerramreddy

Convert mic_request_irq to mic_request_threaded_irq to support threaded irq for virtual devices on mic bus.

---
 drivers/misc/mic/host/mic_intr.c   | 116 ++++++++++++++++++++++---------------
 drivers/misc/mic/host/mic_intr.h   |  18 ++++--
 drivers/misc/mic/host/mic_main.c   |   5 +-
 drivers/misc/mic/host/mic_virtio.c |   6 +-
 4 files changed, 90 insertions(+), 55 deletions(-)
drivers/misc/mic/host/mic_intr.c +69 -47
index dbc5afd..e53b150 100644
@@ -24,28 +24,29 @@
 #include "../common/mic_dev.h"
 #include "mic_device.h"
 
-/*
- * mic_invoke_callback - Invoke callback functions registered for
- * the corresponding source id.
- *
- * @mdev: pointer to the mic_device instance
- * @idx: The interrupt source id.
- *
- * Returns none.
- */
-static inline void mic_invoke_callback(struct mic_device *mdev, int idx)
+static irqreturn_t mic_thread_fn(int irq, void *dev)
 {
+	struct mic_device *mdev = dev;
+	struct mic_intr_info *intr_info = mdev->intr_info;
+	struct mic_irq_info *irq_info = &mdev->irq_info;
 	struct mic_intr_cb *intr_cb;
 	struct pci_dev *pdev = container_of(mdev->sdev->parent,
-		struct pci_dev, dev);
+					    struct pci_dev, dev);
+	int i;
 
-	spin_lock(&mdev->irq_info.mic_intr_lock);
-	list_for_each_entry(intr_cb, &mdev->irq_info.cb_list[idx], list)
-		if (intr_cb->func)
-			intr_cb->func(pdev->irq, intr_cb->data);
-	spin_unlock(&mdev->irq_info.mic_intr_lock);
+	spin_lock(&irq_info->mic_thread_lock);
+	for (i = intr_info->intr_start_idx[MIC_INTR_DB];
+			i < intr_info->intr_len[MIC_INTR_DB]; i++)
+		if (test_and_clear_bit(i, &irq_info->mask)) {
+			list_for_each_entry(intr_cb, &irq_info->cb_list[i],
+					    list)
+				if (intr_cb->thread_fn)
+					intr_cb->thread_fn(pdev->irq,
+							 intr_cb->data);
+		}
+	spin_unlock(&irq_info->mic_thread_lock);
+	return IRQ_HANDLED;
 }
-
 /**
  * mic_interrupt - Generic interrupt handler for
  * MSI and INTx based interrupts.
@@ -53,7 +54,11 @@ static inline void mic_invoke_callback(struct mic_device *mdev, int idx)
 static irqreturn_t mic_interrupt(int irq, void *dev)
 {
 	struct mic_device *mdev = dev;
-	struct mic_intr_info *info = mdev->intr_info;
+	struct mic_intr_info *intr_info = mdev->intr_info;
+	struct mic_irq_info *irq_info = &mdev->irq_info;
+	struct mic_intr_cb *intr_cb;
+	struct pci_dev *pdev = container_of(mdev->sdev->parent,
+					    struct pci_dev, dev);
 	u32 mask;
 	int i;
 
@@ -61,12 +66,19 @@ static irqreturn_t mic_interrupt(int irq, void *dev)
 	if (!mask)
 		return IRQ_NONE;
 
-	for (i = info->intr_start_idx[MIC_INTR_DB];
-			i < info->intr_len[MIC_INTR_DB]; i++)
-		if (mask & BIT(i))
-			mic_invoke_callback(mdev, i);
-
-	return IRQ_HANDLED;
+	spin_lock(&irq_info->mic_intr_lock);
+	for (i = intr_info->intr_start_idx[MIC_INTR_DB];
+			i < intr_info->intr_len[MIC_INTR_DB]; i++)
+		if (mask & BIT(i)) {
+			list_for_each_entry(intr_cb, &irq_info->cb_list[i],
+					    list)
+				if (intr_cb->handler)
+					intr_cb->handler(pdev->irq,
+							 intr_cb->data);
+			set_bit(i, &irq_info->mask);
+		}
+	spin_unlock(&irq_info->mic_intr_lock);
+	return IRQ_WAKE_THREAD;
 }
 
 /* Return the interrupt offset from the index. Index is 0 based. */
@@ -99,14 +111,15 @@ static struct msix_entry *mic_get_available_vector(struct mic_device *mdev)
  *
  * @mdev: pointer to the mic_device instance
  * @idx: The source id to be registered.
- * @func: The function to be called when the source id receives
+ * @handler: The function to be called when the source id receives
  * the interrupt.
+ * @thread_fn: thread fn. corresponding to the handler
  * @data: Private data of the requester.
  * Return the callback structure that was registered or an
  * appropriate error on failure.
  */
 static struct mic_intr_cb *mic_register_intr_callback(struct mic_device *mdev,
-			u8 idx, irqreturn_t (*func) (int irq, void *dev),
+			u8 idx, irq_handler_t handler, irq_handler_t thread_fn,
 			void *data)
 {
 	struct mic_intr_cb *intr_cb;
@@ -117,7 +130,8 @@ static struct mic_intr_cb *mic_register_intr_callback(struct mic_device *mdev,
 	if (!intr_cb)
 		return ERR_PTR(-ENOMEM);
 
-	intr_cb->func = func;
+	intr_cb->handler = handler;
+	intr_cb->thread_fn = thread_fn;
 	intr_cb->data = data;
 	intr_cb->cb_id = ida_simple_get(&mdev->irq_info.cb_ida,
 		0, 0, GFP_KERNEL);
@@ -126,9 +140,11 @@ static struct mic_intr_cb *mic_register_intr_callback(struct mic_device *mdev,
 		goto ida_fail;
 	}
 
+	spin_lock(&mdev->irq_info.mic_thread_lock);
 	spin_lock_irqsave(&mdev->irq_info.mic_intr_lock, flags);
 	list_add_tail(&intr_cb->list, &mdev->irq_info.cb_list[idx]);
 	spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
+	spin_unlock(&mdev->irq_info.mic_thread_lock);
 
 	return intr_cb;
 ida_fail:
@@ -152,8 +168,9 @@ static u8 mic_unregister_intr_callback(struct mic_device *mdev, u32 idx)
 	unsigned long flags;
 	int i;
 
+	spin_lock(&mdev->irq_info.mic_thread_lock);
+	spin_lock_irqsave(&mdev->irq_info.mic_intr_lock, flags);
 	for (i = 0;  i < MIC_NUM_OFFSETS; i++) {
-		spin_lock_irqsave(&mdev->irq_info.mic_intr_lock, flags);
 		list_for_each_safe(pos, tmp, &mdev->irq_info.cb_list[i]) {
 			intr_cb = list_entry(pos, struct mic_intr_cb, list);
 			if (intr_cb->cb_id == idx) {
@@ -163,11 +180,13 @@ static u8 mic_unregister_intr_callback(struct mic_device *mdev, u32 idx)
 				kfree(intr_cb);
 				spin_unlock_irqrestore(
 					&mdev->irq_info.mic_intr_lock, flags);
+				spin_unlock(&mdev->irq_info.mic_thread_lock);
 				return i;
 			}
 		}
-		spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
 	}
+	spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
+	spin_unlock(&mdev->irq_info.mic_thread_lock);
 	return MIC_NUM_OFFSETS;
 }
 
@@ -242,6 +261,7 @@ static int mic_setup_callbacks(struct mic_device *mdev)
 		INIT_LIST_HEAD(&mdev->irq_info.cb_list[i]);
 	ida_init(&mdev->irq_info.cb_ida);
 	spin_lock_init(&mdev->irq_info.mic_intr_lock);
+	spin_lock_init(&mdev->irq_info.mic_thread_lock);
 	return 0;
 }
 
@@ -258,14 +278,12 @@ static void mic_release_callbacks(struct mic_device *mdev)
 	struct mic_intr_cb *intr_cb;
 	int i;
 
+	spin_lock(&mdev->irq_info.mic_thread_lock);
+	spin_lock_irqsave(&mdev->irq_info.mic_intr_lock, flags);
 	for (i = 0; i < MIC_NUM_OFFSETS; i++) {
-		spin_lock_irqsave(&mdev->irq_info.mic_intr_lock, flags);
 
-		if (list_empty(&mdev->irq_info.cb_list[i])) {
-			spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock,
-					       flags);
+		if (list_empty(&mdev->irq_info.cb_list[i]))
 			break;
-		}
 
 		list_for_each_safe(pos, tmp, &mdev->irq_info.cb_list[i]) {
 			intr_cb = list_entry(pos, struct mic_intr_cb, list);
@@ -274,8 +292,9 @@ static void mic_release_callbacks(struct mic_device *mdev)
 					  intr_cb->cb_id);
 			kfree(intr_cb);
 		}
-		spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
 	}
+	spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
+	spin_unlock(&mdev->irq_info.mic_thread_lock);
 	ida_destroy(&mdev->irq_info.cb_ida);
 	kfree(mdev->irq_info.cb_list);
 }
@@ -313,7 +332,8 @@ static int mic_setup_msi(struct mic_device *mdev, struct pci_dev *pdev)
 		goto err_nomem2;
 	}
 
-	rc = request_irq(pdev->irq, mic_interrupt, 0 , "mic-msi", mdev);
+	rc = request_threaded_irq(pdev->irq, mic_interrupt, mic_thread_fn,
+				  0, "mic-msi", mdev);
 	if (rc) {
 		dev_err(&pdev->dev, "Error allocating MSI interrupt\n");
 		goto err_irq_req_fail;
@@ -353,8 +373,8 @@ static int mic_setup_intx(struct mic_device *mdev, struct pci_dev *pdev)
 		goto err_nomem;
 	}
 
-	rc = request_irq(pdev->irq, mic_interrupt,
-		IRQF_SHARED, "mic-intx", mdev);
+	rc = request_threaded_irq(pdev->irq, mic_interrupt, mic_thread_fn,
+				  IRQF_SHARED, "mic-intx", mdev);
 	if (rc)
 		goto err;
 
@@ -391,13 +411,14 @@ int mic_next_db(struct mic_device *mdev)
 #define MK_COOKIE(x, y) ((x) | (y) << COOKIE_ID_SHIFT)
 
 /**
- * mic_request_irq - request an irq. mic_mutex needs
+ * mic_request_threaded_irq - request an irq. mic_mutex needs
  * to be held before calling this function.
  *
  * @mdev: pointer to mic_device instance
- * @func: The callback function that handles the interrupt.
+ * @handler: The callback function that handles the interrupt.
  * The function needs to call ack_interrupts
  * (mdev->ops->ack_interrupt(mdev)) when handling the interrupts.
+ * @thread_fn: thread fn required by request_threaded_irq.
  * @name: The ASCII name of the callee requesting the irq.
  * @data: private data that is returned back when calling the
  * function handler.
@@ -412,8 +433,8 @@ int mic_next_db(struct mic_device *mdev)
  * error code.
  *
  */
-struct mic_irq *mic_request_irq(struct mic_device *mdev,
-	irqreturn_t (*func)(int irq, void *dev),
+struct mic_irq *mic_request_threaded_irq(struct mic_device *mdev,
+	irq_handler_t handler, irq_handler_t thread_fn,
 	const char *name, void *data, int intr_src,
 	enum mic_intr_type type)
 {
@@ -444,7 +465,8 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 			goto err;
 		}
 
-		rc = request_irq(msix->vector, func, 0, name, data);
+		rc = request_threaded_irq(msix->vector, handler, thread_fn,
+					  0, name, data);
 		if (rc) {
 			dev_dbg(mdev->sdev->parent,
 				"request irq failed rc = %d\n", rc);
@@ -458,8 +480,8 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 		dev_dbg(mdev->sdev->parent, "irq: %d assigned for src: %d\n",
 			msix->vector, intr_src);
 	} else {
-		intr_cb = mic_register_intr_callback(mdev,
-				offset, func, data);
+		intr_cb = mic_register_intr_callback(mdev, offset, handler,
+						     thread_fn, data);
 		if (IS_ERR(intr_cb)) {
 			dev_err(mdev->sdev->parent,
 				"No available callback entries for use\n");
@@ -487,9 +509,9 @@ err:
  *  needs to be held before calling this function.
  *
  * @mdev: pointer to mic_device instance
- * @cookie: cookie obtained during a successful call to mic_request_irq
+ * @cookie: cookie obtained during a successful call to mic_request_threaded_irq
  * @data: private data specified by the calling function during the
- * mic_request_irq
+ * mic_request_threaded_irq
  *
  * returns: none.
  */
drivers/misc/mic/host/mic_intr.h +14 -4
index 6091aa9..6b03f84 100644
@@ -21,6 +21,8 @@
 #ifndef _MIC_INTR_H_
 #define _MIC_INTR_H_
 
+#include <linux/bitops.h>
+#include <linux/interrupt.h>
 /*
  * The minimum number of msix vectors required for normal operation.
  * 3 for virtio network, console and block devices.
@@ -68,7 +70,11 @@ struct mic_intr_info {
  * @num_vectors: The number of MSI/MSI-x vectors that have been allocated.
  * @cb_ida: callback ID allocator to track the callbacks registered.
  * @mic_intr_lock: spinlock to protect the interrupt callback list.
+ * @mic_thread_lock: spinlock to protect the thread callback list.
+ *		   This lock is used to protect against thread_fn while
+ *		   mic_intr_lock is used to protect against interrupt handler.
  * @cb_list: Array of callback lists one for each source.
+ * @mask: Mask used by the main thread fn to call the underlying thread fns.
  */
 struct mic_irq_info {
 	int next_avail_src;
@@ -77,19 +83,23 @@ struct mic_irq_info {
 	u16 num_vectors;
 	struct ida cb_ida;
 	spinlock_t mic_intr_lock;
+	spinlock_t mic_thread_lock;
 	struct list_head *cb_list;
+	unsigned long mask;
 };
 
 /**
  * struct mic_intr_cb - Interrupt callback structure.
  *
- * @func: The callback function
+ * @handler: The callback function
+ * @thread_fn: The thread_fn.
  * @data: Private data of the requester.
  * @cb_id: The callback id. Identifies this callback.
  * @list: list head pointing to the next callback structure.
  */
 struct mic_intr_cb {
-	irqreturn_t (*func) (int irq, void *data);
+	irq_handler_t handler;
+	irq_handler_t thread_fn;
 	void *data;
 	int cb_id;
 	struct list_head list;
@@ -124,8 +134,8 @@ struct mic_hw_intr_ops {
 };
 
 int mic_next_db(struct mic_device *mdev);
-struct mic_irq *mic_request_irq(struct mic_device *mdev,
-	irqreturn_t (*func)(int irq, void *data),
+struct mic_irq *mic_request_threaded_irq(struct mic_device *mdev,
+	irq_handler_t handler, irq_handler_t thread_fn,
 	const char *name, void *data, int intr_src,
 	enum mic_intr_type type);
 
drivers/misc/mic/host/mic_main.c +3 -2
index c04a021..fdc9c13 100644
@@ -389,8 +389,9 @@ static int mic_probe(struct pci_dev *pdev,
 	mutex_lock(&mdev->mic_mutex);
 
 	mdev->shutdown_db = mic_next_db(mdev);
-	mdev->shutdown_cookie = mic_request_irq(mdev, mic_shutdown_db,
-		"shutdown-interrupt", mdev, mdev->shutdown_db, MIC_INTR_DB);
+	mdev->shutdown_cookie = mic_request_threaded_irq(mdev, mic_shutdown_db,
+					NULL, "shutdown-interrupt", mdev,
+					mdev->shutdown_db, MIC_INTR_DB);
 	if (IS_ERR(mdev->shutdown_cookie)) {
 		rc = PTR_ERR(mdev->shutdown_cookie);
 		mutex_unlock(&mdev->mic_mutex);
drivers/misc/mic/host/mic_virtio.c +4 -2
index 7e1ef0e..aba3e83 100644
@@ -594,8 +594,10 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 	snprintf(irqname, sizeof(irqname), "mic%dvirtio%d", mdev->id,
 		 mvdev->virtio_id);
 	mvdev->virtio_db = mic_next_db(mdev);
-	mvdev->virtio_cookie = mic_request_irq(mdev, mic_virtio_intr_handler,
-			irqname, mvdev, mvdev->virtio_db, MIC_INTR_DB);
+	mvdev->virtio_cookie = mic_request_threaded_irq(mdev,
+					       mic_virtio_intr_handler,
+					       NULL, irqname, mvdev,
+					       mvdev->virtio_db, MIC_INTR_DB);
 	if (IS_ERR(mvdev->virtio_cookie)) {
 		ret = PTR_ERR(mvdev->virtio_cookie);
 		dev_dbg(mdev->sdev->parent, "request irq failed\n");
Signature · 1 line
1.8.2.2
#5 lore raw

PATCH 5/8 misc: mic: add dma support in host driver

In reply to Siva Yerramreddy

This patch adds a dma device on the mic virtual bus and uses this dmaengine to transfer data for virtio devices

---
 drivers/misc/mic/Kconfig           |   2 +-
 drivers/misc/mic/host/mic_boot.c   |  78 +++++++++++++++-
 drivers/misc/mic/host/mic_device.h |  24 +++++
 drivers/misc/mic/host/mic_intr.h   |   3 +-
 drivers/misc/mic/host/mic_virtio.c | 179 +++++++++++++++++++++++++++++--------
 drivers/misc/mic/host/mic_virtio.h |  21 ++++-
 drivers/misc/mic/host/mic_x100.c   |   8 ++
 7 files changed, 274 insertions(+), 41 deletions(-)
drivers/misc/mic/Kconfig +1 -1
index cfe185f..9409fe5 100644
@@ -19,7 +19,7 @@ comment "Intel MIC Host Driver"
 
 config INTEL_MIC_HOST
 	tristate "Intel MIC Host Driver"
-	depends on 64BIT && PCI && X86
+	depends on 64BIT && PCI && X86 && INTEL_MIC_BUS
 	select VHOST_RING
 	default N
 	help
drivers/misc/mic/host/mic_boot.c +77 -1

109 lines not on this page · raw message

drivers/misc/mic/host/mic_device.h +24 -0
index 0398c69..7583a80 100644
@@ -25,6 +25,8 @@
 #include <linux/idr.h>
 #include <linux/notifier.h>
 #include <linux/irqreturn.h>
+#include <linux/dmaengine.h>
+#include <linux/mic_bus.h>
 
 #include "mic_intr.h"
 
@@ -87,6 +89,8 @@ enum mic_stepping {
  * @cdev: Character device for MIC.
  * @vdev_list: list of virtio devices.
  * @pm_notifier: Handles PM notifications from the OS.
+ * @dma_mbdev: MIC BUS DMA device.
+ * @dma_ch: DMA channel reserved by this driver for use by virtio devices.
  */
 struct mic_device {
 	struct mic_mw mmio;
@@ -124,6 +128,8 @@ struct mic_device {
 	struct cdev cdev;
 	struct list_head vdev_list;
 	struct notifier_block pm_notifier;
+	struct mbus_device dma_mbdev;
+	struct dma_chan *dma_ch;
 };
 
 /**
@@ -144,6 +150,7 @@ struct mic_device {
  * @load_mic_fw: Load firmware segments required to boot the card
  * into card memory. This includes the kernel, command line, ramdisk etc.
  * @get_postcode: Get post code status from firmware.
+ * @dma_filter: DMA filter function to be used.
  */
 struct mic_hw_ops {
 	u8 aper_bar;
@@ -159,6 +166,7 @@ struct mic_hw_ops {
 	void (*send_firmware_intr)(struct mic_device *mdev);
 	int (*load_mic_fw)(struct mic_device *mdev, const char *buf);
 	u32 (*get_postcode)(struct mic_device *mdev);
+	bool (*dma_filter)(struct dma_chan *chan, void *param);
 };
 
 /**
@@ -187,6 +195,22 @@ mic_mmio_write(struct mic_mw *mw, u32 val, u32 offset)
 	iowrite32(val, mw->va + offset);
 }
 
+static inline struct dma_chan *mic_request_dma_chan(struct mic_device *mdev)
+{
+	dma_cap_mask_t mask;
+	struct dma_chan *chan;
+
+	dma_cap_zero(mask);
+	dma_cap_set(DMA_MEMCPY, mask);
+	chan = dma_request_channel(mask, mdev->ops->dma_filter,
+				   mdev->sdev->parent);
+	if (chan)
+		return chan;
+	dev_err(mdev->sdev->parent, "%s %d unable to acquire channel\n",
+		__func__, __LINE__);
+	return NULL;
+}
+
 void mic_sysfs_init(struct mic_device *mdev);
 int mic_start(struct mic_device *mdev, const char *buf);
 void mic_stop(struct mic_device *mdev, bool force);
drivers/misc/mic/host/mic_intr.h +2 -1
index 6b03f84..dd354ff 100644
@@ -27,8 +27,9 @@
  * The minimum number of msix vectors required for normal operation.
  * 3 for virtio network, console and block devices.
  * 1 for card shutdown notifications.
+ * 4 for host owned DMA channels.
  */
-#define MIC_MIN_MSIX 4
+#define MIC_MIN_MSIX 8
 #define MIC_NUM_OFFSETS 32
 
 /**
drivers/misc/mic/host/mic_virtio.c +143 -36

251 lines not on this page · raw message

drivers/misc/mic/host/mic_virtio.h +19 -2
index 184f3c8..d574efb 100644
@@ -46,18 +46,23 @@
  * @vrh: The host VRINGH used for accessing the card vrings.
  * @riov: The VRINGH read kernel IOV.
  * @wiov: The VRINGH write kernel IOV.
- * @head: The VRINGH head index address passed to vringh_getdesc_kern(..).
  * @vr_mutex: Mutex for synchronizing access to the VRING.
+ * @buf: Temporary kernel buffer used to copy in/out data
+ * from/to the card via DMA.
+ * @buf_da: dma address of buf.
  * @mvdev: Back pointer to MIC virtio device for vringh_notify(..).
+ * @head: The VRINGH head index address passed to vringh_getdesc_kern(..).
  */
 struct mic_vringh {
 	struct mic_vring vring;
 	struct vringh vrh;
 	struct vringh_kiov riov;
 	struct vringh_kiov wiov;
-	u16 head;
 	struct mutex vr_mutex;
+	void *buf;
+	dma_addr_t buf_da;
 	struct mic_vdev *mvdev;
+	u16 head;
 };
 
 /**
@@ -69,6 +74,14 @@ struct mic_vringh {
  * @poll_wake - Used for waking up threads blocked in poll.
  * @out_bytes - Debug stats for number of bytes copied from host to card.
  * @in_bytes - Debug stats for number of bytes copied from card to host.
+ * @out_bytes_dma - Debug stats for number of bytes copied from host to card
+ * using DMA.
+ * @in_bytes_dma - Debug stats for number of bytes copied from card to host
+ * using DMA.
+ * @tx_len_unaligned - Debug stats for number of bytes copied to the card where
+ * the transfer length did not have the required DMA alignment.
+ * @tx_dst_unaligned - Debug stats for number of bytes copied where the
+ * destination address on the card did not have the required DMA alignment.
  * @mvr - Store per VRING data structures.
  * @virtio_bh_work - Work struct used to schedule virtio bottom half handling.
  * @dd - Virtio device descriptor.
@@ -84,6 +97,10 @@ struct mic_vdev {
 	int poll_wake;
 	unsigned long out_bytes;
 	unsigned long in_bytes;
+	unsigned long out_bytes_dma;
+	unsigned long in_bytes_dma;
+	unsigned long tx_len_unaligned;
+	unsigned long tx_dst_unaligned;
 	struct mic_vringh mvr[MIC_MAX_VRINGS];
 	struct work_struct virtio_bh_work;
 	struct mic_device_desc *dd;
drivers/misc/mic/host/mic_x100.c +8 -0
index 5562fdd..b7a21e1 100644
@@ -549,6 +549,13 @@ struct mic_smpt_ops mic_x100_smpt_ops = {
 	.set = mic_x100_smpt_set,
 };
 
+static bool mic_x100_dma_filter(struct dma_chan *chan, void *param)
+{
+	if (chan->device->dev->parent == (struct device *)param)
+		return true;
+	return false;
+}
+
 struct mic_hw_ops mic_x100_ops = {
 	.aper_bar = MIC_X100_APER_BAR,
 	.mmio_bar = MIC_X100_MMIO_BAR,
@@ -563,6 +570,7 @@ struct mic_hw_ops mic_x100_ops = {
 	.send_firmware_intr = mic_x100_send_firmware_intr,
 	.load_mic_fw = mic_x100_load_firmware,
 	.get_postcode = mic_x100_get_postcode,
+	.dma_filter = mic_x100_dma_filter,
 };
 
 struct mic_hw_intr_ops mic_x100_intr_ops = {
Signature · 1 line
1.8.2.2
#6 lore raw

PATCH 6/8 misc: mic: add threaded irq support in card driver

In reply to Siva Yerramreddy

Add threaded irq support in mic_request_card_irq which will be used for virtual devices added on mic bus.

---
 drivers/misc/mic/card/mic_device.c | 21 +++++++++++----------
 drivers/misc/mic/card/mic_device.h |  5 +++--
 drivers/misc/mic/card/mic_virtio.c |  4 ++--
 3 files changed, 16 insertions(+), 14 deletions(-)
drivers/misc/mic/card/mic_device.c +11 -10
index d0980ff..ff485b7 100644
@@ -83,8 +83,8 @@ static int mic_shutdown_init(void)
 	int shutdown_db;
 
 	shutdown_db = mic_next_card_db();
-	shutdown_cookie = mic_request_card_irq(mic_shutdown_isr,
-			"Shutdown", mdrv, shutdown_db);
+	shutdown_cookie = mic_request_card_irq(mic_shutdown_isr, NULL,
+					       "Shutdown", mdrv, shutdown_db);
 	if (IS_ERR(shutdown_cookie))
 		rc = PTR_ERR(shutdown_cookie);
 	else
@@ -136,7 +136,8 @@ static void mic_dp_uninit(void)
 /**
  * mic_request_card_irq - request an irq.
  *
- * @func: The callback function that handles the interrupt.
+ * @handler: interrupt handler passed to request_threaded_irq.
+ * @thread_fn: thread fn. passed to request_threaded_irq.
  * @name: The ASCII name of the callee requesting the irq.
  * @data: private data that is returned back when calling the
  * function handler.
@@ -149,17 +150,17 @@ static void mic_dp_uninit(void)
  * error code.
  *
  */
-struct mic_irq *mic_request_card_irq(irqreturn_t (*func)(int irq, void *data),
-	const char *name, void *data, int index)
+struct mic_irq *mic_request_card_irq(irq_handler_t handler,
+	irq_handler_t thread_fn, const char *name, void *data, int index)
 {
 	int rc = 0;
 	unsigned long cookie;
 	struct mic_driver *mdrv = g_drv;
 
-	rc  = request_irq(mic_db_to_irq(mdrv, index), func,
-		0, name, data);
+	rc  = request_threaded_irq(mic_db_to_irq(mdrv, index), handler,
+				   thread_fn, 0, name, data);
 	if (rc) {
-		dev_err(mdrv->dev, "request_irq failed rc = %d\n", rc);
+		dev_err(mdrv->dev, "request_threaded_irq failed rc = %d\n", rc);
 		goto err;
 	}
 	mdrv->irq_info.irq_usage_count[index]++;
@@ -172,9 +173,9 @@ err:
 /**
  * mic_free_card_irq - free irq.
  *
- * @cookie: cookie obtained during a successful call to mic_request_irq
+ * @cookie: cookie obtained during a successful call to mic_request_threaded_irq
  * @data: private data specified by the calling function during the
- * mic_request_irq
+ * mic_request_threaded_irq
  *
  * returns: none.
  */
drivers/misc/mic/card/mic_device.h +3 -2
index 306f502..e12a0c2 100644
@@ -30,6 +30,7 @@
 #include <linux/workqueue.h>
 #include <linux/io.h>
 #include <linux/irqreturn.h>
+#include <linux/interrupt.h>
 
 /**
  * struct mic_intr_info - Contains h/w specific interrupt sources info
@@ -116,8 +117,8 @@ mic_mmio_write(struct mic_mw *mw, u32 val, u32 offset)
 int mic_driver_init(struct mic_driver *mdrv);
 void mic_driver_uninit(struct mic_driver *mdrv);
 int mic_next_card_db(void);
-struct mic_irq *mic_request_card_irq(irqreturn_t (*func)(int irq, void *data),
-	const char *name, void *data, int intr_src);
+struct mic_irq *mic_request_card_irq(irq_handler_t handler,
+	irq_handler_t thread_fn, const char *name, void *data, int intr_src);
 void mic_free_card_irq(struct mic_irq *cookie, void *data);
 u32 mic_read_spad(struct mic_device *mdev, unsigned int idx);
 void mic_send_intr(struct mic_device *mdev, int doorbell);
drivers/misc/mic/card/mic_virtio.c +2 -2
index 653799b..8cdbc68 100644
@@ -417,7 +417,7 @@ static int mic_add_device(struct mic_device_desc __iomem *d,
 
 	virtio_db = mic_next_card_db();
 	mvdev->virtio_cookie = mic_request_card_irq(mic_virtio_intr_handler,
-			"virtio intr", mvdev, virtio_db);
+			NULL, "virtio intr", mvdev, virtio_db);
 	if (IS_ERR(mvdev->virtio_cookie)) {
 		ret = PTR_ERR(mvdev->virtio_cookie);
 		goto kfree;
@@ -606,7 +606,7 @@ int mic_devices_init(struct mic_driver *mdrv)
 	mic_scan_devices(mdrv, !REMOVE_DEVICES);
 
 	config_db = mic_next_card_db();
-	virtio_config_cookie = mic_request_card_irq(mic_extint_handler,
+	virtio_config_cookie = mic_request_card_irq(mic_extint_handler, NULL,
 			"virtio_config_intr", mdrv, config_db);
 	if (IS_ERR(virtio_config_cookie)) {
 		rc = PTR_ERR(virtio_config_cookie);
Signature · 1 line
1.8.2.2
#7 lore raw

PATCH 7/8 misc: mic: add dma support in card driver

In reply to Siva Yerramreddy

This patch adds a dma device on the mic virtual bus

---
 drivers/misc/mic/Kconfig           |  2 +-
 drivers/misc/mic/card/mic_device.h |  3 +++
 drivers/misc/mic/card/mic_x100.c   | 52 +++++++++++++++++++++++++++++++++++++-
 3 files changed, 55 insertions(+), 2 deletions(-)
drivers/misc/mic/Kconfig +1 -1
index 9409fe5..439282b 100644
@@ -40,7 +40,7 @@ comment "Intel MIC Card Driver"
 
 config INTEL_MIC_CARD
 	tristate "Intel MIC Card Driver"
-	depends on 64BIT && X86
+	depends on 64BIT && X86 && INTEL_MIC_BUS
 	select VIRTIO
 	default N
 	help
drivers/misc/mic/card/mic_device.h +3 -0
index e12a0c2..8d735ba 100644
@@ -31,6 +31,7 @@
 #include <linux/io.h>
 #include <linux/irqreturn.h>
 #include <linux/interrupt.h>
+#include <linux/mic_bus.h>
 
 /**
  * struct mic_intr_info - Contains h/w specific interrupt sources info
@@ -71,6 +72,7 @@ struct mic_device {
  * @hotplug_work: Hot plug work for adding/removing virtio devices.
  * @irq_info: The OS specific irq information
  * @intr_info: H/W specific interrupt information.
+ * @dma_mbdev: dma device on the MIC virtual bus.
  */
 struct mic_driver {
 	char name[20];
@@ -81,6 +83,7 @@ struct mic_driver {
 	struct work_struct hotplug_work;
 	struct mic_irq_info irq_info;
 	struct mic_intr_info intr_info;
+	struct mbus_device dma_mbdev;
 };
 
 /**
drivers/misc/mic/card/mic_x100.c +51 -1

76 lines not on this page · raw message

Signature · 1 line
1.8.2.2
#8 lore raw

PATCH 8/8 misc: mic: add support for loading/unloading dma driver

In reply to Siva Yerramreddy

modprobe dma driver upon start and remove it upon unload.

---
 Documentation/mic/mpssd/mpss | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
Documentation/mic/mpssd/mpss +7 -7
index 3136c68..cacbdb0 100755
@@ -48,18 +48,18 @@ start()
 	fi
 
 	echo -e $"Starting MPSS Stack"
-	echo -e $"Loading MIC_HOST Module"
+	echo -e $"Loading MIC_X100_DMA & MIC_HOST Modules"
 
-	# Ensure the driver is loaded
-	if [ ! -d "$sysfs" ]; then
-		modprobe mic_host
+	for f in "mic_host" "mic_x100_dma"
+	do
+		modprobe $f
 		RETVAL=$?
 		if [ $RETVAL -ne 0 ]; then
 			failure
 			echo
 			return $RETVAL
 		fi
-	fi
+	done
 
 	# Start the daemon
 	echo -n $"Starting MPSSD "
@@ -170,8 +170,8 @@ unload()
 	stop
 
 	sleep 5
-	echo -n $"Removing MIC_HOST Module: "
-	modprobe -r mic_host
+	echo -n $"Removing MIC_HOST & MIC_X100_DMA Modules: "
+	modprobe -r mic_host mic_x100_dma
 	RETVAL=$?
 	[ $RETVAL -ne 0 ] && failure || success
 	echo
Signature · 1 line
1.8.2.2

A message that is not in the archive (sent off-list or not yet received): 1398363008.6542.1.camel@[email protected]

#10 lore raw
Quoted 5 lines · Siva Krishna Yerramreddy
Quoted 3 lines · Siva Yerramreddy

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Greg, any feedback on the patches?

Hi Greg,
The primary author of this patch series Siva is no longer with Intel so we will be taking ownership of addressing review feedback.

The patches have been applied to the MIC GITHUB tree which is registered with Fengguang Wu's 0-day infrastructure and no issues have been reported.

We have not received any feedback on the patches yet and were wondering if you had a chance to review them?

Thanks,
Sudeep Dutt

Quoted 115 lines
Quoted 113 lines

Description:
============
This set of patches add support for MIC X100 dma driver.
MIC PCIe card has a dma controller with 8 channels. These channels are shared between the host s/w and the card s/w. 0 to 3 are used by host and 4 to 7 by card. As the dma device doesn't show up as PCIe device, a virtual bus called mic bus is created and virtual dma devices are created on it by the host/card drivers. On host the channels are private and used only by the host driver to transfer data for the virtio devices.

Here is a higher level block diagram.

				      |
	       +----------+           |             +----------+
	       | Card OS  |           |             | Host OS  |
	       +----------+           |             +----------+
				      |
	+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
	| Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
	| Net   | |Console | |Block | | |Net      |  |Console | |Block   |
	| Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
	+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
	    |         |         |     |      |            |         |
	    |         |         |     |User  |            |         |
	    |         |         |     |------|------------|---------|-------
	    +-------------------+     |Kernel +--------------------------+
		      |               |       | Virtio over PCIe IOCTLs  |
		      |               |       +--------------------------+
+-----------+	      |               |	    		  |  +-----------+
| MIC DMA   |	      |               | 		  |  | MIC DMA	 |
| Driver    |         |               |	  		  |  | Driver	 |
+-----------+         |               |			  |  +-----------+
      |		      |	   	      |			  |	   |
+---------------+     |		      |			  |  +----------------+
|MIC virtual Bus|     |	 	      |			  |  |MIC virtual Bus |
+---------------+     |		      |			  |  +----------------+
      |               |               |                   |		 |
      |   +--------------+	      |            +---------------+     |
      |   |Intel MIC     |	      |            |Intel MIC      |     |
      +---|Card Driver   |            |            |Host Driver    |     |
	  +--------------+            |            +---------------+-----+
		      |               |                   |
	     +-------------------------------------------------------------+
	     |                                                             |
	     |                    PCIe Bus                                 |
	     +-------------------------------------------------------------+

The following series of patches are partitioned as follows:

Patch 1: Add mic bus and dma driver documentation.
Author: Siva Yerramreddy
Patch 2: Add a bus driver for virtual MIC devices.
Authors: Siva Yerramreddy, Sudeep Dutt
Patch 3: MIC X100 DMA Driver.
Author: Siva Yerramreddy
Patch 4: Add threaded irq support in host driver.
This is needed as the dma driver uses threaded irq.
Author: Siva Yerramreddy
Patch 5: Add dma support in host driver.
Authors: Siva Yerramreddy, Ashutosh Dixit, Sudeep Dutt
Patch 6: Add threaded irq support in card driver.
This is needed as the dma driver uses threaded irq.
Author: Siva Yerramreddy
Patch 7: Add dma support in card driver.
Author: Siva Yerramreddy
Patch 8: Add support for loading/unloading dma driver.
Author: Siva Yerramreddy

The patches have been compiled/validated against v3.14. Tested using dmatest module with module parameter "threads_per_chan=60".

Thanks to Dan Williams, Vinod Koul, Jon Mason, Dave Jiang for the initial review.

Siva Yerramreddy (8):
misc: mic: Add mic bus and dma driver documentation
misc: mic: add a bus driver for virtual MIC devices
dma: MIC X100 DMA Driver
misc: mic: add threaded irq support in host driver
misc: mic: add dma support in host driver
misc: mic: add threaded irq support in card driver
misc: mic: add dma support in card driver
misc: mic: add support for loading/unloading dma driver

 Documentation/mic/mic_overview.txt |  67 ++--
 Documentation/mic/mpssd/mpss       |  14 +-
 drivers/dma/Kconfig                |  19 +
 drivers/dma/Makefile               |   1 +
 drivers/dma/mic_x100_dma.c         | 774 +++++++++++++++++++++++++++++++++++++
 drivers/dma/mic_x100_dma.h         | 286 ++++++++++++++
 drivers/misc/mic/Kconfig           |  21 +-
 drivers/misc/mic/Makefile          |   1 +
 drivers/misc/mic/bus/Makefile      |   5 +
 drivers/misc/mic/bus/mic_bus.c     | 188 +++++++++
 drivers/misc/mic/card/mic_device.c |  21 +-
 drivers/misc/mic/card/mic_device.h |   8 +-
 drivers/misc/mic/card/mic_virtio.c |   4 +-
 drivers/misc/mic/card/mic_x100.c   |  52 ++-
 drivers/misc/mic/host/mic_boot.c   |  78 +++-
 drivers/misc/mic/host/mic_device.h |  24 ++
 drivers/misc/mic/host/mic_intr.c   | 116 +++---
 drivers/misc/mic/host/mic_intr.h   |  21 +-
 drivers/misc/mic/host/mic_main.c   |   5 +-
 drivers/misc/mic/host/mic_virtio.c | 185 +++++++--
 drivers/misc/mic/host/mic_virtio.h |  21 +-
 drivers/misc/mic/host/mic_x100.c   |   8 +
 include/linux/mic_bus.h            | 148 +++++++
 23 files changed, 1922 insertions(+), 145 deletions(-)
 create mode 100644 drivers/dma/mic_x100_dma.c
 create mode 100644 drivers/dma/mic_x100_dma.h
 create mode 100644 drivers/misc/mic/bus/Makefile
 create mode 100644 drivers/misc/mic/bus/mic_bus.c
 create mode 100644 include/linux/mic_bus.h
#11 lore raw
Quoted 11 lines · Sudeep Dutt
Quoted 5 lines · Siva Krishna Yerramreddy
Quoted 3 lines · Siva Yerramreddy

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Greg, any feedback on the patches?

Hi Greg,
The primary author of this patch series Siva is no longer with Intel so we will be taking ownership of addressing review feedback.

Quoted 6 lines

The patches have been applied to the MIC GITHUB tree which is registered with Fengguang Wu's 0-day infrastructure and no issues have been reported.

We have not received any feedback on the patches yet and were wondering if you had a chance to review them?

Fwiw, I'm still planning on reviewing these.

#12 lore raw
Quoted 12 lines · Dan Williams
Quoted 11 lines · Sudeep Dutt
Quoted 5 lines · Siva Krishna Yerramreddy
Quoted 3 lines · Siva Yerramreddy

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Greg, any feedback on the patches?

Hi Greg,
The primary author of this patch series Siva is no longer with Intel so we will be taking ownership of addressing review feedback.

Quoted 8 lines
Quoted 6 lines

The patches have been applied to the MIC GITHUB tree which is registered with Fengguang Wu's 0-day infrastructure and no issues have been reported.

We have not received any feedback on the patches yet and were wondering if you had a chance to review them?

Fwiw, I'm still planning on reviewing these.

They are in my queue, sorry, it's really big at the moment due to travel...

greg k-h

#13 lore raw
In reply to Sudeep Dutt
Quoted 10 lines · Sudeep Dutt
Quoted 5 lines · Siva Krishna Yerramreddy
Quoted 3 lines · Siva Yerramreddy

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Greg, any feedback on the patches?

Hi Greg,
The primary author of this patch series Siva is no longer with Intel so we will be taking ownership of addressing review feedback.

Care to resend these with an author email address that will not bounce? I don't like taking code from people with invalid email addresses...

thanks,

greg k-h

#14 lore raw
Quoted 6 lines · Greg Kroah-Hartman
Quoted 5 lines · Sudeep Dutt
Quoted 4 lines · Siva Krishna Yerramreddy
Quoted 3 lines · Siva Yerramreddy

I am sending all these patches to char-misc because there is a dependency between the patches for dma driver and other drivers.

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

Greg, any feedback on the patches?

Hi Greg,
The primary author of this patch series Siva is no longer with Intel so we will be taking ownership of addressing review feedback.

Care to resend these with an author email address that will not bounce? I don't like taking code from people with invalid email addresses...

Sure, I have resent the patch series. Please take a look.

Thanks,
Sudeep Dutt