Skip to contentSkip to conversation

Enable Drivers for Intel MIC X100 Coprocessors.

PATCH RESEND In review Sudeep Dutt 3 participants 32 messages Sep 5, 2013 – Sep 26, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover Enable Drivers for Intel MIC X100 Coprocessors.

ChangeLog:
=========

v2 => v3:
a) Patch 1 data structure cleanups, header file include cleanups, IDA interface reuse and switching to device_create_with_groups(..) as per feedback from Greg Kroah-Hartman.
b) Patch 7 signal documentation, sleep workaround removal and sysfs access API cleanups as per feedback from Michael S. Tsirkin.

v1 => v2: @ http://lwn.net/Articles/563131/
a) License wording cleanup, sysfs ABI documentation, patch 1 refactoring into 3 smaller patches and function renames, as per feedback from Greg Kroah-Hartman.
b) Use VRINGH infrastructure for accessing virtio rings from the host in patch 5, as per feedback from Michael S. Tsirkin.

v1: Initial post @ https://lwn.net/Articles/561314/

Description:
============

An Intel MIC X100 device is a PCIe form factor add-in coprocessor card based on the Intel Many Integrated Core (MIC) architecture that runs a Linux OS. It is a PCIe endpoint in a platform and therefore implements the three required standard address spaces i.e. configuration, memory and I/O. The host OS loads a device driver as is typical for PCIe devices. The card itself runs a bootstrap after reset that transfers control to the card OS downloaded from the host driver. The card OS as shipped by Intel is a Linux kernel with modifications for the X100 devices.

Since it is a PCIe card, it does not have the ability to host hardware devices for networking, storage and console. We provide these devices on X100 coprocessors thus enabling a self-bootable equivalent environment for applications. A key benefit of our solution is that it leverages the standard virtio framework for network, disk and console devices, though in our case the virtio framework is used across a PCIe bus.

Here is a block diagram of the various components described above. The virtio backends are situated on the host rather than the card given better single threaded performance for the host compared to MIC, the ability of the host to initiate DMA's to/from the card using the MIC DMA engine and the fact that the virtio block storage backend can only be on the host.

       +----------+           |             +----------+
       | Card OS  |           |             | Host OS  |
       +----------+           |             +----------+
                              |
+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
| Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
| Net   | |Console | |Block | | |Net      |  |Console | |Block   |
| Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
+-------+ +--------+ +------+ | +---------+  +--------+ +--------+
    |         |         |     |      |            |         |
    |         |         |     |User  |            |         |
    |         |         |     |------|------------|---------|-------
    +-------------------+     |Kernel +--------------------------+
              |               |       | Virtio over PCIe IOCTLs  |
              |               |       +--------------------------+
      +--------------+        |                   |
      |Intel MIC     |        |            +---------------+
      |Card Driver   |        |            |Intel MIC      |
      +--------------+        |            |Host Driver    |
              |               |            +---------------+
              |               |                   |
     +-------------------------------------------------------------+
     |                                                             |
     |                    PCIe Bus                                 |
     +-------------------------------------------------------------+

The following series of patches are partitioned as follows:

Patch 1: This patch introduces the "Intel MIC Host Driver" in the block diagram which does the following:
a) Initializes the Intel MIC X100 PCIe devices.
b) Provides sysfs entries for family and stepping information.

Patch 2: This patch enables the following features in the "Intel MIC Host Driver" in the block diagram:
a) MSIx, MSI and legacy interrupt support.
b) System Memory Page Table(SMPT) support. SMPT enables system memory access from the card. On X100 devices the host can program 32 SMPT registers each capable of accessing 16GB of system memory address space from X100 devices. The registers can thereby be used to access a cumulative 512GB of system memory address space from X100 devices at any point in time.

Patch 3: This patch enables the following features in the "Intel MIC Host Driver" in the block diagram:
a) Boots and shuts down the card via sysfs entries.
b) Allocates and maps a device page for communication with the card driver and updates the device page address via scratchpad registers.
c) Provides sysfs entries for shutdown status, kernel command line, firmware, ramdisk, bootmode and log buffer information.

Patch 4: This patch introduces the "Intel MIC Card Driver" in the block diagram which does the following:
a) Initializes the Intel MIC X100 platform device and driver.
b) Sets up support to handle shutdown requests from the host.
c) Maps the device page after obtaining the device page address from the scratchpad registers updated by the host.
d) Informs the host upon a card crash by registering a panic notifier.
e) Informs the host upon a poweroff/halt event.

Patch 5: This patch introduces the host "Virtio over PCIe" interface for Intel MIC. It allows creating user space backends on the host and instantiating virtio devices for them on the Intel MIC card. It uses the existing VRINGH infrastructure in the kernel to access virtio rings from the host. A character device per MIC is exposed with IOCTL, mmap and poll callbacks. This allows the user space backend to:
(a) add/remove a virtio device via a device page.
(b) map (R/O) virtio rings and device page to user space.
(c) poll for availability of data.
(d) copy a descriptor or entire descriptor chain to/from the card.
(e) modify virtio configuration.
(f) handle virtio device reset.
The buffers are copied over using CPU copies for this initial patch and host initiated MIC DMA support is planned for future patches. The avail and desc virtio rings are in host memory and the used ring is in card memory to maximize writes across PCIe for performance.

Patch 6: This patch introduces the card "Virtio over PCIe" interface for Intel MIC. It allows virtio drivers on the card to communicate with their user space backends on the host via a device page. Ring 3 apps on the host can add, remove and configure virtio devices. A thin MIC specific virtio_config_ops is implemented which is borrowed heavily from previous similar implementations in lguest and s390 @ drivers/lguest/lguest_device.c
drivers/s390/kvm/kvm_virtio.c

Patch 7: This patch introduces a sample user space daemon which implements the virtio device backends on the host. The daemon creates/removes/configures virtio device backends by communicating with the Intel MIC Host Driver. The virtio devices currently supported are virtio net, virtio console and virtio block. Virtio net supports TSO/GSO. The daemon also monitors card shutdown status and takes appropriate actions like killing the virtio backends and resetting the card upon card shutdown and crashes.

The patches have been compiled/validated against v3.11.

Ashutosh Dixit (2):
Intel MIC Host Driver Changes for Virtio Devices. Intel MIC Card Driver Changes for Virtio Devices.

Caz Yokoyama (1):
Sample Implementation of Intel MIC User Space Daemon.

Dasaratharaman Chandramouli (1):
Intel MIC Host Driver Interrupt/SMPT support.

Sudeep Dutt (3):
Intel MIC Host Driver for X100 family. Intel MIC Host Driver, card OS state management. Intel MIC Card Driver for X100 family.

 Documentation/ABI/testing/sysfs-class-mic.txt |  147 +++
 Documentation/mic/mic_overview.txt            |   49 +
 Documentation/mic/mpssd/.gitignore            |    1 +
 Documentation/mic/mpssd/Makefile              |   19 +
 Documentation/mic/mpssd/micctrl               |  173 +++
 Documentation/mic/mpssd/mpss                  |  202 +++
 Documentation/mic/mpssd/mpssd.c               | 1701 +++++++++++++++++++++++++
 Documentation/mic/mpssd/mpssd.h               |  100 ++
 Documentation/mic/mpssd/sysfs.c               |  102 ++
 drivers/misc/Kconfig                          |    1 +
 drivers/misc/Makefile                         |    1 +
 drivers/misc/mic/Kconfig                      |   39 +
 drivers/misc/mic/Makefile                     |    6 +
 drivers/misc/mic/card/Makefile                |   11 +
 drivers/misc/mic/card/mic_debugfs.c           |  130 ++
 drivers/misc/mic/card/mic_device.c            |  306 +++++
 drivers/misc/mic/card/mic_device.h            |  133 ++
 drivers/misc/mic/card/mic_virtio.c            |  631 +++++++++
 drivers/misc/mic/card/mic_virtio.h            |   77 ++
 drivers/misc/mic/card/mic_x100.c              |  256 ++++
 drivers/misc/mic/card/mic_x100.h              |   48 +
 drivers/misc/mic/common/mic_device.h          |   51 +
 drivers/misc/mic/host/Makefile                |   14 +
 drivers/misc/mic/host/mic_boot.c              |  185 +++
 drivers/misc/mic/host/mic_debugfs.c           |  495 +++++++
 drivers/misc/mic/host/mic_device.h            |  195 +++
 drivers/misc/mic/host/mic_fops.c              |  221 ++++
 drivers/misc/mic/host/mic_fops.h              |   32 +
 drivers/misc/mic/host/mic_intr.c              |  628 +++++++++
 drivers/misc/mic/host/mic_intr.h              |  137 ++
 drivers/misc/mic/host/mic_main.c              |  478 +++++++
 drivers/misc/mic/host/mic_smpt.c              |  442 +++++++
 drivers/misc/mic/host/mic_smpt.h              |   98 ++
 drivers/misc/mic/host/mic_sysfs.c             |  466 +++++++
 drivers/misc/mic/host/mic_virtio.c            |  703 ++++++++++
 drivers/misc/mic/host/mic_virtio.h            |  138 ++
 drivers/misc/mic/host/mic_x100.c              |  573 +++++++++
 drivers/misc/mic/host/mic_x100.h              |   98 ++
 include/uapi/linux/Kbuild                     |    2 +
 include/uapi/linux/mic_common.h               |  238 ++++
 include/uapi/linux/mic_ioctl.h                |   74 ++
 41 files changed, 9401 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-class-mic.txt
 create mode 100644 Documentation/mic/mic_overview.txt
 create mode 100644 Documentation/mic/mpssd/.gitignore
 create mode 100644 Documentation/mic/mpssd/Makefile
 create mode 100755 Documentation/mic/mpssd/micctrl
 create mode 100755 Documentation/mic/mpssd/mpss
 create mode 100644 Documentation/mic/mpssd/mpssd.c
 create mode 100644 Documentation/mic/mpssd/mpssd.h
 create mode 100644 Documentation/mic/mpssd/sysfs.c
 create mode 100644 drivers/misc/mic/Kconfig
 create mode 100644 drivers/misc/mic/Makefile
 create mode 100644 drivers/misc/mic/card/Makefile
 create mode 100644 drivers/misc/mic/card/mic_debugfs.c
 create mode 100644 drivers/misc/mic/card/mic_device.c
 create mode 100644 drivers/misc/mic/card/mic_device.h
 create mode 100644 drivers/misc/mic/card/mic_virtio.c
 create mode 100644 drivers/misc/mic/card/mic_virtio.h
 create mode 100644 drivers/misc/mic/card/mic_x100.c
 create mode 100644 drivers/misc/mic/card/mic_x100.h
 create mode 100644 drivers/misc/mic/common/mic_device.h
 create mode 100644 drivers/misc/mic/host/Makefile
 create mode 100644 drivers/misc/mic/host/mic_boot.c
 create mode 100644 drivers/misc/mic/host/mic_debugfs.c
 create mode 100644 drivers/misc/mic/host/mic_device.h
 create mode 100644 drivers/misc/mic/host/mic_fops.c
 create mode 100644 drivers/misc/mic/host/mic_fops.h
 create mode 100644 drivers/misc/mic/host/mic_intr.c
 create mode 100644 drivers/misc/mic/host/mic_intr.h
 create mode 100644 drivers/misc/mic/host/mic_main.c
 create mode 100644 drivers/misc/mic/host/mic_smpt.c
 create mode 100644 drivers/misc/mic/host/mic_smpt.h
 create mode 100644 drivers/misc/mic/host/mic_sysfs.c
 create mode 100644 drivers/misc/mic/host/mic_virtio.c
 create mode 100644 drivers/misc/mic/host/mic_virtio.h
 create mode 100644 drivers/misc/mic/host/mic_x100.c
 create mode 100644 drivers/misc/mic/host/mic_x100.h
 create mode 100644 include/uapi/linux/mic_common.h
 create mode 100644 include/uapi/linux/mic_ioctl.h
Signature · 1 line
1.8.2.1
#1 lore raw

PATCH RESEND v3 1/7 Intel MIC Host Driver for X100 family.

This patch enables the following:
a) Initializes the Intel MIC X100 PCIe devices.
b) Provides sysfs entries for family and stepping information.

Co-author: Dasaratharaman Chandramouli <[email protected]>

---
 Documentation/ABI/testing/sysfs-class-mic.txt |  34 +++
 drivers/misc/Kconfig                          |   1 +
 drivers/misc/Makefile                         |   1 +
 drivers/misc/mic/Kconfig                      |  19 ++
 drivers/misc/mic/Makefile                     |   5 +
 drivers/misc/mic/common/mic_device.h          |  37 +++
 drivers/misc/mic/host/Makefile                |   8 +
 drivers/misc/mic/host/mic_device.h            | 109 +++++++++
 drivers/misc/mic/host/mic_main.c              | 309 ++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_sysfs.c             |  97 ++++++++
 drivers/misc/mic/host/mic_x100.c              |  75 +++++++
 drivers/misc/mic/host/mic_x100.h              |  47 ++++
 12 files changed, 742 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-class-mic.txt
 create mode 100644 drivers/misc/mic/Kconfig
 create mode 100644 drivers/misc/mic/Makefile
 create mode 100644 drivers/misc/mic/common/mic_device.h
 create mode 100644 drivers/misc/mic/host/Makefile
 create mode 100644 drivers/misc/mic/host/mic_device.h
 create mode 100644 drivers/misc/mic/host/mic_main.c
 create mode 100644 drivers/misc/mic/host/mic_sysfs.c
 create mode 100644 drivers/misc/mic/host/mic_x100.c
 create mode 100644 drivers/misc/mic/host/mic_x100.h
Documentation/ABI/testing/sysfs-class-mic.txt new file +34 -0
new file mode 100644
index 0000000..09eb3c6
@@ -0,0 +1,34 @@
+What:		/sys/class/mic/
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		The mic class directory belongs to Intel MIC devices and
+		provides information per MIC device. An Intel MIC device is a
+		PCIe form factor add-in Coprocessor card based on the Intel Many
+		Integrated Core (MIC) architecture that runs a Linux OS.
+
+What:		/sys/class/mic/mic(x)
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		The directories /sys/class/mic/mic0, /sys/class/mic/mic1 etc.,
+		represent MIC devices (0,1,..etc). Each directory has
+		information specific to that MIC device.
+
+What:		/sys/class/mic/mic(x)/family
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		Provides information about the Coprocessor family for an Intel
+		MIC device. For example - "x100"
+
+What:		/sys/class/mic/mic(x)/stepping
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		Provides information about the silicon stepping for an Intel
+		MIC device. For example - "A0" or "B0"
drivers/misc/Kconfig +1 -0
index 8dacd4c..e760715 100644
@@ -537,4 +537,5 @@ source "drivers/misc/carma/Kconfig"
 source "drivers/misc/altera-stapl/Kconfig"
 source "drivers/misc/mei/Kconfig"
 source "drivers/misc/vmw_vmci/Kconfig"
+source "drivers/misc/mic/Kconfig"
 endmenu
drivers/misc/Makefile +1 -0
index c235d5b..0b7ea3e 100644
@@ -53,3 +53,4 @@ obj-$(CONFIG_INTEL_MEI)		+= mei/
 obj-$(CONFIG_VMWARE_VMCI)	+= vmw_vmci/
 obj-$(CONFIG_LATTICE_ECP3_CONFIG)	+= lattice-ecp3-config.o
 obj-$(CONFIG_SRAM)		+= sram.o
+obj-y				+= mic/
drivers/misc/mic/Kconfig new file +19 -0
new file mode 100644
index 0000000..aaefd0c
@@ -0,0 +1,19 @@
+comment "Intel MIC Host Driver"
+
+config INTEL_MIC_HOST
+	tristate "Intel MIC Host Driver"
+	depends on 64BIT && PCI
+	default N
+	help
+	  This enables Host Driver support for the Intel Many Integrated
+	  Core (MIC) family of PCIe form factor coprocessor devices that
+	  run a 64 bit Linux OS. The driver manages card OS state and
+	  enables communication between host and card. Intel MIC X100
+	  devices are currently supported.
+
+	  If you are building a host kernel with an Intel MIC device then
+	  say M (recommended) or Y, else say N. If unsure say N.
+
+	  More information about the Intel MIC family as well as the Linux
+	  OS and tools for MIC to use with this driver are available from
+	  <http://software.intel.com/en-us/mic-developer>.
drivers/misc/mic/Makefile new file +5 -0
new file mode 100644
index 0000000..8e72421
@@ -0,0 +1,5 @@
+#
+# Makefile - Intel MIC Linux driver.
+# Copyright(c) 2013, Intel Corporation.
+#
+obj-$(CONFIG_INTEL_MIC_HOST) += host/
drivers/misc/mic/common/mic_device.h new file +37 -0
new file mode 100644
index 0000000..f02262e
@@ -0,0 +1,37 @@
+/*
+ * Intel MIC Platform Software Stack (MPSS)
+ *
+ * Copyright(c) 2013 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Intel MIC driver.
+ *
+ */
+#ifndef __MIC_COMMON_DEVICE_H_
+#define __MIC_COMMON_DEVICE_H_
+
+/**
+ * struct mic_mw - MIC memory window
+ *
+ * @pa: Base physical address.
+ * @va: Base ioremap'd virtual address.
+ * @len: Size of the memory window.
+ */
+struct mic_mw {
+	phys_addr_t pa;
+	void __iomem *va;
+	resource_size_t len;
+};
+
+#endif
drivers/misc/mic/host/Makefile new file +8 -0
new file mode 100644
index 0000000..93b9d25
@@ -0,0 +1,8 @@
+#
+# Makefile - Intel MIC Linux driver.
+# Copyright(c) 2013, Intel Corporation.
+#
+obj-$(CONFIG_INTEL_MIC_HOST) += mic_host.o
+mic_host-objs := mic_main.o
+mic_host-objs += mic_x100.o
+mic_host-objs += mic_sysfs.o
drivers/misc/mic/host/mic_device.h new file +109 -0

109 lines not on this page · raw message

drivers/misc/mic/host/mic_main.c new file +309 -0

309 lines not on this page · raw message

drivers/misc/mic/host/mic_sysfs.c new file +97 -0

97 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.c new file +75 -0

75 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.h new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#5 lore raw

PATCH RESEND v3 2/7 Intel MIC Host Driver Interrupt/SMPT support.

In reply to Sudeep Dutt

From: Dasaratharaman Chandramouli <[email protected]>

This patch enables the following features:
a) MSIx, MSI and legacy interrupt support.
b) System Memory Page Table(SMPT) support. SMPT enables system memory access from the card. On X100 devices the host can program 32 SMPT registers each capable of accessing 16GB of system memory address space from X100 devices. The registers can thereby be used to access a cumulative 512GB of system memory address space from X100 devices at any point in time.

Co-author: Sudeep Dutt <[email protected]>

---
 drivers/misc/mic/host/Makefile     |   2 +
 drivers/misc/mic/host/mic_device.h |  21 ++
 drivers/misc/mic/host/mic_intr.c   | 628 +++++++++++++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_intr.h   | 137 ++++++++
 drivers/misc/mic/host/mic_main.c   |  26 +-
 drivers/misc/mic/host/mic_smpt.c   | 442 ++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_smpt.h   |  98 ++++++
 drivers/misc/mic/host/mic_x100.c   | 247 +++++++++++++++
 drivers/misc/mic/host/mic_x100.h   |  39 +++
 9 files changed, 1638 insertions(+), 2 deletions(-)
 create mode 100644 drivers/misc/mic/host/mic_intr.c
 create mode 100644 drivers/misc/mic/host/mic_intr.h
 create mode 100644 drivers/misc/mic/host/mic_smpt.c
 create mode 100644 drivers/misc/mic/host/mic_smpt.h
drivers/misc/mic/host/Makefile +2 -0
index 93b9d25..6ff5550 100644
@@ -6,3 +6,5 @@ obj-$(CONFIG_INTEL_MIC_HOST) += mic_host.o
 mic_host-objs := mic_main.o
 mic_host-objs += mic_x100.o
 mic_host-objs += mic_sysfs.o
+mic_host-objs += mic_smpt.o
+mic_host-objs += mic_intr.o
drivers/misc/mic/host/mic_device.h +21 -0
index 6cd904c..dd34b65 100644
@@ -21,6 +21,10 @@
 #ifndef _MIC_DEVICE_H_
 #define _MIC_DEVICE_H_
 
+#include <linux/idr.h>
+
+#include "mic_intr.h"
+
 /* The maximum number of MIC devices supported in a single host system. */
 #define MIC_MAX_NUM_DEVS 256
 
@@ -53,6 +57,12 @@ enum mic_stepping {
  * @stepping: Stepping ID.
  * @attr_group: Pointer to list of sysfs attribute groups.
  * @sdev: Device for sysfs entries.
+ * @mic_mutex: Mutex for synchronizing access to mic_device.
+ * @intr_ops: HW specific interrupt operations.
+ * @smpt_ops: Hardware specific SMPT operations.
+ * @smpt: MIC SMPT information.
+ * @intr_info: H/W specific interrupt information.
+ * @irq_info: The OS specific irq information
  */
 struct mic_device {
 	struct mic_mw mmio;
@@ -63,6 +73,12 @@ struct mic_device {
 	enum mic_stepping stepping;
 	const struct attribute_group **attr_group;
 	struct device *sdev;
+	struct mutex mic_mutex;
+	struct mic_hw_intr_ops *intr_ops;
+	struct mic_smpt_ops *smpt_ops;
+	struct mic_smpt_info *smpt;
+	struct mic_intr_info *intr_info;
+	struct mic_irq_info irq_info;
 };
 
 /**
@@ -71,12 +87,17 @@ struct mic_device {
  * @mmio_bar: MMIO bar resource number.
  * @read_spad: Read from scratch pad register.
  * @write_spad: Write to scratch pad register.
+ * @send_intr: Send an interrupt for a particular doorbell on the card.
+ * @ack_interrupt: Hardware specific operations to ack the h/w on
+ * receipt of an interrupt.
  */
 struct mic_hw_ops {
 	u8 aper_bar;
 	u8 mmio_bar;
 	u32 (*read_spad)(struct mic_device *mdev, unsigned int idx);
 	void (*write_spad)(struct mic_device *mdev, unsigned int idx, u32 val);
+	void (*send_intr)(struct mic_device *mdev, int doorbell);
+	u32 (*ack_interrupt)(struct mic_device *mdev);
 };
 
 /**
drivers/misc/mic/host/mic_intr.c new file +628 -0

628 lines not on this page · raw message

drivers/misc/mic/host/mic_intr.h new file +137 -0

137 lines not on this page · raw message

drivers/misc/mic/host/mic_main.c +24 -2
index 228c96c..332a15e 100644
@@ -23,13 +23,13 @@
  * 2) Enable per vring interrupt support.
  */
 #include <linux/fs.h>
-#include <linux/idr.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_x100.h"
+#include "mic_smpt.h"
 
 static const char mic_driver_name[] = "mic";
 
@@ -75,6 +75,8 @@ static void mic_ops_init(struct mic_device *mdev)
 	switch (mdev->family) {
 	case MIC_FAMILY_X100:
 		mdev->ops = &mic_x100_ops;
+		mdev->intr_ops = &mic_x100_intr_ops;
+		mdev->smpt_ops = &mic_x100_smpt_ops;
 		break;
 	default:
 		break;
@@ -132,6 +134,8 @@ mic_device_init(struct mic_device *mdev, struct pci_dev *pdev)
 	mdev->stepping = pdev->revision;
 	mic_ops_init(mdev);
 	mic_sysfs_init(mdev);
+	mutex_init(&mdev->mic_mutex);
+	mdev->irq_info.next_avail_src = 0;
 }
 
 /**
@@ -201,6 +205,18 @@ static int mic_probe(struct pci_dev *pdev,
 		goto unmap_mmio;
 	}
 
+	mdev->intr_ops->intr_init(mdev);
+	rc = mic_setup_interrupts(mdev, pdev);
+	if (rc) {
+		dev_err(&pdev->dev, "mic_setup_interrupts failed %d\n", rc);
+		goto unmap_aper;
+	}
+	rc = mic_smpt_init(mdev);
+	if (rc) {
+		dev_err(&pdev->dev, "smpt_init failed %d\n", rc);
+		goto free_interrupts;
+	}
+
 	pci_set_drvdata(pdev, mdev);
 
 	mdev->sdev = device_create_with_groups(g_mic_class, &pdev->dev,
@@ -210,9 +226,13 @@ static int mic_probe(struct pci_dev *pdev,
 		rc = PTR_ERR(mdev->sdev);
 		dev_err(&pdev->dev,
 			"device_create_with_groups failed rc %d\n", rc);
-		goto unmap_aper;
+		goto smpt_uninit;
 	}
 	return 0;
+smpt_uninit:
+	mic_smpt_uninit(mdev);
+free_interrupts:
+	mic_free_interrupts(mdev, pdev);
 unmap_aper:
 	iounmap(mdev->aper.va);
 unmap_mmio:
@@ -246,6 +266,8 @@ static void mic_remove(struct pci_dev *pdev)
 		return;
 
 	device_destroy(g_mic_class, MKDEV(MAJOR(g_mic_devno), mdev->id));
+	mic_smpt_uninit(mdev);
+	mic_free_interrupts(mdev, pdev);
 	iounmap(mdev->mmio.va);
 	iounmap(mdev->aper.va);
 	pci_release_regions(pdev);
drivers/misc/mic/host/mic_smpt.c new file +442 -0

442 lines not on this page · raw message

drivers/misc/mic/host/mic_smpt.h new file +98 -0

98 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.c +247 -0

262 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.h +39 -0
index 1f4e630..642cae9 100644
@@ -42,6 +42,45 @@
 
 #define MIC_X100_SBOX_BASE_ADDRESS 0x00010000
 #define MIC_X100_SBOX_SPAD0 0x0000AB20
+#define MIC_X100_SBOX_SICR0_DBR(x) ((x) & 0xf)
+#define MIC_X100_SBOX_SICR0_DMA(x) (((x) >> 8) & 0xff)
+#define MIC_X100_SBOX_SICE0_DBR(x) ((x) & 0xf)
+#define MIC_X100_SBOX_DBR_BITS(x) ((x) & 0xf)
+#define MIC_X100_SBOX_SICE0_DMA(x) (((x) >> 8) & 0xff)
+#define MIC_X100_SBOX_DMA_BITS(x) (((x) & 0xff) << 8)
+
+#define MIC_X100_SBOX_APICICR0 0x0000A9D0
+#define MIC_X100_SBOX_SICR0 0x00009004
+#define MIC_X100_SBOX_SICE0 0x0000900C
+#define MIC_X100_SBOX_SICC0 0x00009010
+#define MIC_X100_SBOX_SIAC0 0x00009014
+#define MIC_X100_SBOX_MSIXPBACR 0x00009084
+#define MIC_X100_SBOX_MXAR0 0x00009044
+#define MIC_X100_SBOX_SMPT00 0x00003100
+#define MIC_X100_SBOX_RDMASR0 0x0000B180
+
+#define MIC_X100_DOORBELL_IDX_START 0
+#define MIC_X100_NUM_DOORBELL 4
+#define MIC_X100_DMA_IDX_START 8
+#define MIC_X100_NUM_DMA 8
+#define MIC_X100_ERR_IDX_START 30
+#define MIC_X100_NUM_ERR 1
+
+#define MIC_X100_NUM_SBOX_IRQ 8
+#define MIC_X100_NUM_RDMASR_IRQ 8
+#define MIC_X100_RDMASR_IRQ_BASE 17
+
+static const u16 mic_x100_intr_init[] = {
+		MIC_X100_DOORBELL_IDX_START,
+		MIC_X100_DMA_IDX_START,
+		MIC_X100_ERR_IDX_START,
+		MIC_X100_NUM_DOORBELL,
+		MIC_X100_NUM_DMA,
+		MIC_X100_NUM_ERR,
+};
+
 extern struct mic_hw_ops mic_x100_ops;
+extern struct mic_smpt_ops mic_x100_smpt_ops;
+extern struct mic_hw_intr_ops mic_x100_intr_ops;
 
 #endif
Signature · 1 line
1.8.2.1
#7 lore raw

PATCH RESEND v3 3/7 Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt

This patch enables the following features:
a) Boots and shuts down the card via sysfs entries.
b) Allocates and maps a device page for communication with the card driver and updates the device page address via scratchpad registers.
c) Provides sysfs entries for shutdown status, kernel command line, ramdisk and log buffer information.

Co-author: Dasaratharaman Chandramouli <[email protected]>

---
 Documentation/ABI/testing/sysfs-class-mic.txt | 113 ++++++++
 drivers/misc/mic/common/mic_device.h          |   7 +
 drivers/misc/mic/host/Makefile                |   2 +
 drivers/misc/mic/host/mic_boot.c              | 184 +++++++++++++
 drivers/misc/mic/host/mic_debugfs.c           | 355 +++++++++++++++++++++++++
 drivers/misc/mic/host/mic_device.h            |  60 +++++
 drivers/misc/mic/host/mic_main.c              | 129 ++++++++-
 drivers/misc/mic/host/mic_sysfs.c             | 369 ++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_x100.c              | 251 ++++++++++++++++++
 drivers/misc/mic/host/mic_x100.h              |  12 +
 include/uapi/linux/Kbuild                     |   1 +
 include/uapi/linux/mic_common.h               |  74 ++++++
 12 files changed, 1553 insertions(+), 4 deletions(-)
 create mode 100644 drivers/misc/mic/host/mic_boot.c
 create mode 100644 drivers/misc/mic/host/mic_debugfs.c
 create mode 100644 include/uapi/linux/mic_common.h
Documentation/ABI/testing/sysfs-class-mic.txt +113 -0
index 09eb3c6..82cdad3 100644
@@ -32,3 +32,116 @@ Contact:	Sudeep Dutt <[email protected]>
 Description:
 		Provides information about the silicon stepping for an Intel
 		MIC device. For example - "A0" or "B0"
+
+What:		/sys/class/mic/mic(x)/state
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this entry provides the current state of an Intel
+		MIC device in the context of the card OS. Possible values that
+		will be read are:
+		"offline" - The MIC device is ready to boot the card OS.
+		"online" - The MIC device has initiated booting a card OS.
+		"shutting_down" - The card OS is shutting down.
+		"reset_failed" - The MIC device has failed to reset.
+
+		When written, this sysfs entry triggers different state change
+		operations depending upon the current state of the card OS.
+		Acceptable values are:
+		"boot" - Boot the card OS image specified by the combination
+			 of firmware, ramdisk, cmdline and bootmode
+			sysfs entries.
+		"reset" - Initiates device reset.
+		"shutdown" - Initiates card OS shutdown.
+
+What:		/sys/class/mic/mic(x)/shutdown_status
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. This
+		OS can shutdown because of various reasons. When read, this
+		entry provides the status on why the card OS was shutdown.
+		Possible values are:
+		"nop" -  shutdown status is not applicable, when the card OS is
+			"online"
+		"crashed" - Shutdown because of a HW or SW crash.
+		"halted" - Shutdown because of a halt command.
+		"poweroff" - Shutdown because of a poweroff command.
+		"restart" - Shutdown because of a restart command.
+
+What:		/sys/class/mic/mic(x)/cmdline
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. Before
+		booting this card OS, it is possible to pass kernel command line
+		options to configure various features in it, similar to
+		self-bootable machines. When read, this entry provides
+		information about the current kernel command line options set to
+		boot the card OS. This entry can be written to change the
+		existing kernel command line options. Typically, the user would
+		want to read the current command line options, append new ones
+		or modify existing ones and then write the whole kernel command
+		line back to this entry.
+
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.
+
+What:		/sys/class/mic/mic(x)/ramdisk
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the ramdisk image to be used during card
+		OS boot can be found. The entry can be written to change
+		the ramdisk image location under /lib/firmware/.
+
+What:		/sys/class/mic/mic(x)/bootmode
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the current bootmode for
+		the card. This sysfs entry can be written with the following
+		valid strings:
+		a) linux - Boot a Linux image.
+		b) elf - Boot an elf image for flash updates.
+
+What:		/sys/class/mic/mic(x)/log_buf_addr
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. For
+		debugging purpose and early kernel boot messages, the user can
+		access the card OS log buffer via debugfs. When read, this entry
+		provides the kernel virtual address of the buffer where the card
+		OS log buffer can be read. This entry is written by the host
+		configuration daemon to set the log buffer address. The correct
+		log buffer address to be written can be found in the System.map
+		file of the card OS.
+
+What:		/sys/class/mic/mic(x)/log_buf_len
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. For
+		debugging purpose and early kernel boot messages, the user can
+		access the card OS log buffer via debugfs. When read, this entry
+		provides the kernel virtual address where the card OS log buffer
+		length can be read. This entry is written by host configuration
+		daemon to set the log buffer length address. The correct log
+		buffer length address to be written can be found in the
+		System.map file of the card OS.
drivers/misc/mic/common/mic_device.h +7 -0
index f02262e..6440e9d 100644
@@ -34,4 +34,11 @@ struct mic_mw {
 	resource_size_t len;
 };
 
+/*
+ * Scratch pad register offsets used by the host to communicate
+ * device page DMA address to the card.
+ */
+#define MIC_DPLO_SPAD 14
+#define MIC_DPHI_SPAD 15
+
 #endif
drivers/misc/mic/host/Makefile +2 -0
index 6ff5550..a375dd3 100644
@@ -8,3 +8,5 @@ mic_host-objs += mic_x100.o
 mic_host-objs += mic_sysfs.o
 mic_host-objs += mic_smpt.o
 mic_host-objs += mic_intr.o
+mic_host-objs += mic_boot.o
+mic_host-objs += mic_debugfs.o
drivers/misc/mic/host/mic_boot.c new file +184 -0

184 lines not on this page · raw message

drivers/misc/mic/host/mic_debugfs.c new file +355 -0

355 lines not on this page · raw message

drivers/misc/mic/host/mic_device.h +60 -0

88 lines not on this page · raw message

drivers/misc/mic/host/mic_main.c +125 -4

195 lines not on this page · raw message

drivers/misc/mic/host/mic_sysfs.c +369 -0

387 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.c +251 -0

269 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.h +12 -0
index 642cae9..8b7daa1 100644
@@ -69,6 +69,15 @@
 #define MIC_X100_NUM_SBOX_IRQ 8
 #define MIC_X100_NUM_RDMASR_IRQ 8
 #define MIC_X100_RDMASR_IRQ_BASE 17
+#define MIC_X100_SPAD2_DOWNLOAD_STATUS(x) ((x) & 0x1)
+#define MIC_X100_SPAD2_APIC_ID(x)	(((x) >> 1) & 0x1ff)
+#define MIC_X100_SPAD2_DOWNLOAD_ADDR(x) ((x) & 0xfffff000)
+#define MIC_X100_SBOX_APICICR7 0x0000AA08
+#define MIC_X100_SBOX_RGCR 0x00004010
+#define MIC_X100_SBOX_SDBIC0 0x0000CC90
+#define MIC_X100_DOWNLOAD_INFO 2
+#define MIC_X100_FW_SIZE 5
+#define MIC_X100_POSTCODE 0x242c
 
 static const u16 mic_x100_intr_init[] = {
 		MIC_X100_DOORBELL_IDX_START,
@@ -79,6 +88,9 @@ static const u16 mic_x100_intr_init[] = {
 		MIC_X100_NUM_ERR,
 };
 
+/* Host->Card(bootstrap) Interrupt Vector */
+#define MIC_X100_BSP_INTERRUPT_VECTOR 229
+
 extern struct mic_hw_ops mic_x100_ops;
 extern struct mic_smpt_ops mic_x100_smpt_ops;
 extern struct mic_hw_intr_ops mic_x100_intr_ops;
include/uapi/linux/Kbuild +1 -0
index 997f9f2..e3f7b37 100644
@@ -240,6 +240,7 @@ header-y += media.h
 header-y += mei.h
 header-y += mempolicy.h
 header-y += meye.h
+header-y += mic_common.h
 header-y += mii.h
 header-y += minix_fs.h
 header-y += mman.h
include/uapi/linux/mic_common.h new file +74 -0

74 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#19 lore raw

PATCH RESEND v3 4/7 Intel MIC Card Driver for X100 family.

In reply to Sudeep Dutt

This patch does the following:
a) Initializes the Intel MIC X100 platform device and driver.
b) Sets up support to handle shutdown requests from the host.
c) Maps the device page after obtaining the device page address from the scratchpad registers updated by the host.
d) Informs the host upon a card crash by registering a panic notifier.
e) Informs the host upon a poweroff/halt event.

Co-author: Dasaratharaman Chandramouli <[email protected]>

---
 drivers/misc/mic/Kconfig            |  18 +++
 drivers/misc/mic/Makefile           |   1 +
 drivers/misc/mic/card/Makefile      |  10 ++
 drivers/misc/mic/card/mic_debugfs.c | 130 ++++++++++++++++
 drivers/misc/mic/card/mic_device.c  | 299 ++++++++++++++++++++++++++++++++++++
 drivers/misc/mic/card/mic_device.h  | 133 ++++++++++++++++
 drivers/misc/mic/card/mic_x100.c    | 256 ++++++++++++++++++++++++++++++
 drivers/misc/mic/card/mic_x100.h    |  48 ++++++
 8 files changed, 895 insertions(+)
 create mode 100644 drivers/misc/mic/card/Makefile
 create mode 100644 drivers/misc/mic/card/mic_debugfs.c
 create mode 100644 drivers/misc/mic/card/mic_device.c
 create mode 100644 drivers/misc/mic/card/mic_device.h
 create mode 100644 drivers/misc/mic/card/mic_x100.c
 create mode 100644 drivers/misc/mic/card/mic_x100.h
drivers/misc/mic/Kconfig +18 -0
index aaefd0c..279a2e6 100644
@@ -17,3 +17,21 @@ config INTEL_MIC_HOST
 	  More information about the Intel MIC family as well as the Linux
 	  OS and tools for MIC to use with this driver are available from
 	  <http://software.intel.com/en-us/mic-developer>.
+
+comment "Intel MIC Card Driver"
+
+config INTEL_MIC_CARD
+	tristate "Intel MIC Card Driver"
+	depends on 64BIT
+	default N
+	help
+	  This enables card driver support for the Intel Many Integrated
+	  Core (MIC) device family. The card driver communicates shutdown/
+	  crash events to the host and allows registration/configuration of
+	  virtio devices. Intel MIC X100 devices are currently supported.
+
+	  If you are building a card kernel for an Intel MIC device then
+	  say M (recommended) or Y, else say N. If unsure say N.
+
+	  For more information see
+	  <http://software.intel.com/en-us/mic-developer>.
drivers/misc/mic/Makefile +1 -0
index 8e72421..05b34d6 100644
@@ -3,3 +3,4 @@
 # Copyright(c) 2013, Intel Corporation.
 #
 obj-$(CONFIG_INTEL_MIC_HOST) += host/
+obj-$(CONFIG_INTEL_MIC_CARD) += card/
drivers/misc/mic/card/Makefile new file +10 -0
new file mode 100644
index 0000000..6e9675e
@@ -0,0 +1,10 @@
+#
+# Makefile - Intel MIC Linux driver.
+# Copyright(c) 2013, Intel Corporation.
+#
+ccflags-y += -DINTEL_MIC_CARD
+
+obj-$(CONFIG_INTEL_MIC_CARD) += mic_card.o
+mic_card-y += mic_x100.o
+mic_card-y += mic_device.o
+mic_card-y += mic_debugfs.o
drivers/misc/mic/card/mic_debugfs.c new file +130 -0

130 lines not on this page · raw message

drivers/misc/mic/card/mic_device.c new file +299 -0

299 lines not on this page · raw message

drivers/misc/mic/card/mic_device.h new file +133 -0

133 lines not on this page · raw message

drivers/misc/mic/card/mic_x100.c new file +256 -0

256 lines not on this page · raw message

drivers/misc/mic/card/mic_x100.h new file +48 -0

48 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#20 lore raw

PATCH RESEND v3 5/7 Intel MIC Host Driver Changes for Virtio Devices.

In reply to Sudeep Dutt

From: Ashutosh Dixit <[email protected]>

This patch introduces the host "Virtio over PCIe" interface for Intel MIC. It allows creating user space backends on the host and instantiating virtio devices for them on the Intel MIC card. It uses the existing VRINGH infrastructure in the kernel to access virtio rings from the host. A character device per MIC is exposed with IOCTL, mmap and poll callbacks. This allows the user space backend to:
(a) add/remove a virtio device via a device page.
(b) map (R/O) virtio rings and device page to user space.
(c) poll for availability of data.
(d) copy a descriptor or entire descriptor chain to/from the card.
(e) modify virtio configuration.
(f) handle virtio device reset.
The buffers are copied over using CPU copies for this initial patch and host initiated MIC DMA support is planned for future patches. The avail and desc virtio rings are in host memory and the used ring is in card memory to maximize writes across PCIe for performance.

Co-author: Sudeep Dutt <[email protected]>

---
 drivers/misc/mic/Kconfig             |   1 +
 drivers/misc/mic/common/mic_device.h |   7 +
 drivers/misc/mic/host/Makefile       |   2 +
 drivers/misc/mic/host/mic_boot.c     |   3 +-
 drivers/misc/mic/host/mic_debugfs.c  | 140 +++++++
 drivers/misc/mic/host/mic_device.h   |   5 +
 drivers/misc/mic/host/mic_fops.c     | 221 +++++++++++
 drivers/misc/mic/host/mic_fops.h     |  32 ++
 drivers/misc/mic/host/mic_main.c     |  26 ++
 drivers/misc/mic/host/mic_virtio.c   | 703 +++++++++++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_virtio.h   | 138 +++++++
 include/uapi/linux/Kbuild            |   1 +
 include/uapi/linux/mic_common.h      | 166 ++++++++-
 include/uapi/linux/mic_ioctl.h       |  74 ++++
 14 files changed, 1517 insertions(+), 2 deletions(-)
 create mode 100644 drivers/misc/mic/host/mic_fops.c
 create mode 100644 drivers/misc/mic/host/mic_fops.h
 create mode 100644 drivers/misc/mic/host/mic_virtio.c
 create mode 100644 drivers/misc/mic/host/mic_virtio.h
 create mode 100644 include/uapi/linux/mic_ioctl.h
drivers/misc/mic/Kconfig +1 -0
index 279a2e6..01f1a4a 100644
@@ -3,6 +3,7 @@ comment "Intel MIC Host Driver"
 config INTEL_MIC_HOST
 	tristate "Intel MIC Host Driver"
 	depends on 64BIT && PCI
+	select VHOST_RING
 	default N
 	help
 	  This enables Host Driver support for the Intel Many Integrated
drivers/misc/mic/common/mic_device.h +7 -0
index 6440e9d..01eb74f 100644
@@ -41,4 +41,11 @@ struct mic_mw {
 #define MIC_DPLO_SPAD 14
 #define MIC_DPHI_SPAD 15
 
+/*
+ * These values are supposed to be in the config_change field of the
+ * device page when the host sends a config change interrupt to the card.
+ */
+#define MIC_VIRTIO_PARAM_DEV_REMOVE 0x1
+#define MIC_VIRTIO_PARAM_CONFIG_CHANGED 0x2
+
 #endif
drivers/misc/mic/host/Makefile +2 -0
index a375dd3..c2197f9 100644
@@ -10,3 +10,5 @@ mic_host-objs += mic_smpt.o
 mic_host-objs += mic_intr.o
 mic_host-objs += mic_boot.o
 mic_host-objs += mic_debugfs.o
+mic_host-objs += mic_fops.o
+mic_host-objs += mic_virtio.o
drivers/misc/mic/host/mic_boot.c +2 -1
index 936fc58..fd9ff6d 100644
@@ -20,12 +20,12 @@
  */
 #include <linux/delay.h>
 #include <linux/firmware.h>
-#include <linux/interrupt.h>
 
 #include <linux/mic_common.h>
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_smpt.h"
+#include "mic_virtio.h"
 
 /**
  * mic_reset - Reset the MIC device.
@@ -117,6 +117,7 @@ void mic_stop(struct mic_device *mdev, bool force)
 {
 	mutex_lock(&mdev->mic_mutex);
 	if (MIC_OFFLINE != mdev->state || force) {
+		mic_virtio_reset_devices(mdev);
 		mic_bootparam_init(mdev);
 		mic_reset(mdev);
 		if (MIC_RESET_FAILED == mdev->state)
drivers/misc/mic/host/mic_debugfs.c +140 -0

171 lines not on this page · raw message

drivers/misc/mic/host/mic_device.h +5 -0
index 50b8b88..dcba2a5 100644
@@ -21,6 +21,7 @@
 #ifndef _MIC_DEVICE_H_
 #define _MIC_DEVICE_H_
 
+#include <linux/cdev.h>
 #include <linux/idr.h>
 
 #include "mic_intr.h"
@@ -80,6 +81,8 @@ enum mic_stepping {
  * @dp_dma_addr: virtio device page DMA address.
  * @shutdown_db: shutdown doorbell.
  * @shutdown_cookie: shutdown cookie.
+ * @cdev: Character device for MIC.
+ * @vdev_list: list of virtio devices.
  */
 struct mic_device {
 	struct mic_mw mmio;
@@ -113,6 +116,8 @@ struct mic_device {
 	dma_addr_t dp_dma_addr;
 	int shutdown_db;
 	struct mic_irq *shutdown_cookie;
+	struct cdev cdev;
+	struct list_head vdev_list;
 };
 
 /**
drivers/misc/mic/host/mic_fops.c new file +221 -0

221 lines not on this page · raw message

drivers/misc/mic/host/mic_fops.h new file +32 -0
new file mode 100644
index 0000000..dc3893d
@@ -0,0 +1,32 @@
+/*
+ * Intel MIC Platform Software Stack (MPSS)
+ *
+ * Copyright(c) 2013 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Intel MIC Host driver.
+ *
+ */
+#ifndef _MIC_FOPS_H_
+#define _MIC_FOPS_H_
+
+int mic_open(struct inode *inode, struct file *filp);
+int mic_release(struct inode *inode, struct file *filp);
+ssize_t mic_read(struct file *filp, char __user *buf,
+			size_t count, loff_t *pos);
+long mic_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
+int mic_mmap(struct file *f, struct vm_area_struct *vma);
+unsigned int mic_poll(struct file *f, poll_table *wait);
+
+#endif
drivers/misc/mic/host/mic_main.c +26 -0
index 998a20a..a8965d4 100644
@@ -25,12 +25,15 @@
 #include <linux/fs.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/poll.h>
 
 #include <linux/mic_common.h>
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_x100.h"
 #include "mic_smpt.h"
+#include "mic_fops.h"
+#include "mic_virtio.h"
 
 static const char mic_driver_name[] = "mic";
 
@@ -64,6 +67,15 @@ static struct class *g_mic_class;
 /* Base device node number for MIC devices */
 static dev_t g_mic_devno;
 
+static const struct file_operations mic_fops = {
+	.open = mic_open,
+	.release = mic_release,
+	.unlocked_ioctl = mic_ioctl,
+	.poll = mic_poll,
+	.mmap = mic_mmap,
+	.owner = THIS_MODULE,
+};
+
 /* Initialize the device page */
 static int mic_dp_init(struct mic_device *mdev)
 {
@@ -193,6 +205,7 @@ mic_device_init(struct mic_device *mdev, struct pci_dev *pdev)
 	mdev->irq_info.next_avail_src = 0;
 	INIT_WORK(&mdev->reset_trigger_work, mic_reset_trigger_work);
 	INIT_WORK(&mdev->shutdown_work, mic_shutdown_work);
+	INIT_LIST_HEAD(&mdev->vdev_list);
 }
 
 /**
@@ -330,7 +343,19 @@ static int mic_probe(struct pci_dev *pdev,
 	mic_bootparam_init(mdev);
 
 	mic_create_debug_dir(mdev);
+	cdev_init(&mdev->cdev, &mic_fops);
+	mdev->cdev.owner = THIS_MODULE;
+	rc = cdev_add(&mdev->cdev, MKDEV(MAJOR(g_mic_devno), mdev->id), 1);
+	if (rc) {
+		dev_err(&pdev->dev, "cdev_add err id %d rc %d\n", mdev->id, rc);
+		goto cleanup_debug_dir;
+	}
 	return 0;
+cleanup_debug_dir:
+	mic_delete_debug_dir(mdev);
+	mutex_lock(&mdev->mic_mutex);
+	mic_free_irq(mdev, mdev->shutdown_cookie, mdev);
+	mutex_unlock(&mdev->mic_mutex);
 dp_uninit:
 	mic_dp_uninit(mdev);
 sysfs_put:
@@ -375,6 +400,7 @@ static void mic_remove(struct pci_dev *pdev)
 		return;
 
 	mic_stop(mdev, false);
+	cdev_del(&mdev->cdev);
 	mic_delete_debug_dir(mdev);
 	mutex_lock(&mdev->mic_mutex);
 	mic_free_irq(mdev, mdev->shutdown_cookie, mdev);
drivers/misc/mic/host/mic_virtio.c new file +703 -0

703 lines not on this page · raw message

drivers/misc/mic/host/mic_virtio.h new file +138 -0

138 lines not on this page · raw message

include/uapi/linux/Kbuild +1 -0
index e3f7b37..c73dbd0 100644
@@ -241,6 +241,7 @@ header-y += mei.h
 header-y += mempolicy.h
 header-y += meye.h
 header-y += mic_common.h
+header-y += mic_ioctl.h
 header-y += mii.h
 header-y += minix_fs.h
 header-y += mman.h
include/uapi/linux/mic_common.h +165 -1

178 lines not on this page · raw message

include/uapi/linux/mic_ioctl.h new file +74 -0

74 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#22 lore raw

PATCH RESEND v3 6/7 Intel MIC Card Driver Changes for Virtio Devices.

In reply to Sudeep Dutt

From: Ashutosh Dixit <[email protected]>

This patch introduces the card "Virtio over PCIe" interface for Intel MIC. It allows virtio drivers on the card to communicate with their user space backends on the host via a device page. Ring 3 apps on the host can add, remove and configure virtio devices. A thin MIC specific virtio_config_ops is implemented which is borrowed heavily from previous similar implementations in lguest and s390 @ drivers/lguest/lguest_device.c
drivers/s390/kvm/kvm_virtio.c

Co-author: Sudeep Dutt <[email protected]>

---
 drivers/misc/mic/Kconfig           |   1 +
 drivers/misc/mic/card/Makefile     |   1 +
 drivers/misc/mic/card/mic_device.c |   7 +
 drivers/misc/mic/card/mic_virtio.c | 631 +++++++++++++++++++++++++++++++++++++
 drivers/misc/mic/card/mic_virtio.h |  77 +++++
 5 files changed, 717 insertions(+)
 create mode 100644 drivers/misc/mic/card/mic_virtio.c
 create mode 100644 drivers/misc/mic/card/mic_virtio.h
drivers/misc/mic/Kconfig +1 -0
index 01f1a4a..d453768 100644
@@ -24,6 +24,7 @@ comment "Intel MIC Card Driver"
 config INTEL_MIC_CARD
 	tristate "Intel MIC Card Driver"
 	depends on 64BIT
+	select VIRTIO
 	default N
 	help
 	  This enables card driver support for the Intel Many Integrated
drivers/misc/mic/card/Makefile +1 -0
index 6e9675e..69d58be 100644
@@ -8,3 +8,4 @@ obj-$(CONFIG_INTEL_MIC_CARD) += mic_card.o
 mic_card-y += mic_x100.o
 mic_card-y += mic_device.o
 mic_card-y += mic_debugfs.o
+mic_card-y += mic_virtio.o
drivers/misc/mic/card/mic_device.c +7 -0
index 3c5c302..4125217 100644
@@ -32,6 +32,7 @@
 #include <linux/mic_common.h>
 #include "../common/mic_device.h"
 #include "mic_device.h"
+#include "mic_virtio.h"
 
 static struct mic_driver *g_drv;
 static struct mic_irq *shutdown_cookie;
@@ -265,10 +266,15 @@ int __init mic_driver_init(struct mic_driver *mdrv)
 	rc = mic_shutdown_init();
 	if (rc)
 		goto irq_uninit;
+	rc = mic_devices_init(mdrv);
+	if (rc)
+		goto shutdown_uninit;
 	mic_create_card_debug_dir(mdrv);
 	atomic_notifier_chain_register(&panic_notifier_list, &mic_panic);
 done:
 	return rc;
+shutdown_uninit:
+	mic_shutdown_uninit();
 irq_uninit:
 	mic_uninit_irq();
 dp_uninit:
@@ -286,6 +292,7 @@ put:
 void mic_driver_uninit(struct mic_driver *mdrv)
 {
 	mic_delete_card_debug_dir(mdrv);
+	mic_devices_uninit(mdrv);
 	/*
 	 * Inform the host about the shutdown status i.e. poweroff/restart etc.
 	 * The module cannot be unloaded so the only code path to call
drivers/misc/mic/card/mic_virtio.c new file +631 -0

631 lines not on this page · raw message

drivers/misc/mic/card/mic_virtio.h new file +77 -0

77 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#6 lore raw

PATCH RESEND v3 2/7 Intel MIC Host Driver Interrupt/SMPT support.

In reply to Sudeep Dutt

From: Dasaratharaman Chandramouli <[email protected]>

This patch enables the following features:
a) MSIx, MSI and legacy interrupt support.
b) System Memory Page Table(SMPT) support. SMPT enables system memory access from the card. On X100 devices the host can program 32 SMPT registers each capable of accessing 16GB of system memory address space from X100 devices. The registers can thereby be used to access a cumulative 512GB of system memory address space from X100 devices at any point in time.

Co-author: Sudeep Dutt <[email protected]>

---
 drivers/misc/mic/host/Makefile     |   2 +
 drivers/misc/mic/host/mic_device.h |  21 ++
 drivers/misc/mic/host/mic_intr.c   | 628 +++++++++++++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_intr.h   | 137 ++++++++
 drivers/misc/mic/host/mic_main.c   |  26 +-
 drivers/misc/mic/host/mic_smpt.c   | 442 ++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_smpt.h   |  98 ++++++
 drivers/misc/mic/host/mic_x100.c   | 247 +++++++++++++++
 drivers/misc/mic/host/mic_x100.h   |  39 +++
 9 files changed, 1638 insertions(+), 2 deletions(-)
 create mode 100644 drivers/misc/mic/host/mic_intr.c
 create mode 100644 drivers/misc/mic/host/mic_intr.h
 create mode 100644 drivers/misc/mic/host/mic_smpt.c
 create mode 100644 drivers/misc/mic/host/mic_smpt.h
drivers/misc/mic/host/Makefile +2 -0
index 93b9d25..6ff5550 100644
@@ -6,3 +6,5 @@ obj-$(CONFIG_INTEL_MIC_HOST) += mic_host.o
 mic_host-objs := mic_main.o
 mic_host-objs += mic_x100.o
 mic_host-objs += mic_sysfs.o
+mic_host-objs += mic_smpt.o
+mic_host-objs += mic_intr.o
drivers/misc/mic/host/mic_device.h +21 -0
index 6cd904c..dd34b65 100644
@@ -21,6 +21,10 @@
 #ifndef _MIC_DEVICE_H_
 #define _MIC_DEVICE_H_
 
+#include <linux/idr.h>
+
+#include "mic_intr.h"
+
 /* The maximum number of MIC devices supported in a single host system. */
 #define MIC_MAX_NUM_DEVS 256
 
@@ -53,6 +57,12 @@ enum mic_stepping {
  * @stepping: Stepping ID.
  * @attr_group: Pointer to list of sysfs attribute groups.
  * @sdev: Device for sysfs entries.
+ * @mic_mutex: Mutex for synchronizing access to mic_device.
+ * @intr_ops: HW specific interrupt operations.
+ * @smpt_ops: Hardware specific SMPT operations.
+ * @smpt: MIC SMPT information.
+ * @intr_info: H/W specific interrupt information.
+ * @irq_info: The OS specific irq information
  */
 struct mic_device {
 	struct mic_mw mmio;
@@ -63,6 +73,12 @@ struct mic_device {
 	enum mic_stepping stepping;
 	const struct attribute_group **attr_group;
 	struct device *sdev;
+	struct mutex mic_mutex;
+	struct mic_hw_intr_ops *intr_ops;
+	struct mic_smpt_ops *smpt_ops;
+	struct mic_smpt_info *smpt;
+	struct mic_intr_info *intr_info;
+	struct mic_irq_info irq_info;
 };
 
 /**
@@ -71,12 +87,17 @@ struct mic_device {
  * @mmio_bar: MMIO bar resource number.
  * @read_spad: Read from scratch pad register.
  * @write_spad: Write to scratch pad register.
+ * @send_intr: Send an interrupt for a particular doorbell on the card.
+ * @ack_interrupt: Hardware specific operations to ack the h/w on
+ * receipt of an interrupt.
  */
 struct mic_hw_ops {
 	u8 aper_bar;
 	u8 mmio_bar;
 	u32 (*read_spad)(struct mic_device *mdev, unsigned int idx);
 	void (*write_spad)(struct mic_device *mdev, unsigned int idx, u32 val);
+	void (*send_intr)(struct mic_device *mdev, int doorbell);
+	u32 (*ack_interrupt)(struct mic_device *mdev);
 };
 
 /**
drivers/misc/mic/host/mic_intr.c new file +628 -0

628 lines not on this page · raw message

drivers/misc/mic/host/mic_intr.h new file +137 -0

137 lines not on this page · raw message

drivers/misc/mic/host/mic_main.c +24 -2
index 228c96c..332a15e 100644
@@ -23,13 +23,13 @@
  * 2) Enable per vring interrupt support.
  */
 #include <linux/fs.h>
-#include <linux/idr.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_x100.h"
+#include "mic_smpt.h"
 
 static const char mic_driver_name[] = "mic";
 
@@ -75,6 +75,8 @@ static void mic_ops_init(struct mic_device *mdev)
 	switch (mdev->family) {
 	case MIC_FAMILY_X100:
 		mdev->ops = &mic_x100_ops;
+		mdev->intr_ops = &mic_x100_intr_ops;
+		mdev->smpt_ops = &mic_x100_smpt_ops;
 		break;
 	default:
 		break;
@@ -132,6 +134,8 @@ mic_device_init(struct mic_device *mdev, struct pci_dev *pdev)
 	mdev->stepping = pdev->revision;
 	mic_ops_init(mdev);
 	mic_sysfs_init(mdev);
+	mutex_init(&mdev->mic_mutex);
+	mdev->irq_info.next_avail_src = 0;
 }
 
 /**
@@ -201,6 +205,18 @@ static int mic_probe(struct pci_dev *pdev,
 		goto unmap_mmio;
 	}
 
+	mdev->intr_ops->intr_init(mdev);
+	rc = mic_setup_interrupts(mdev, pdev);
+	if (rc) {
+		dev_err(&pdev->dev, "mic_setup_interrupts failed %d\n", rc);
+		goto unmap_aper;
+	}
+	rc = mic_smpt_init(mdev);
+	if (rc) {
+		dev_err(&pdev->dev, "smpt_init failed %d\n", rc);
+		goto free_interrupts;
+	}
+
 	pci_set_drvdata(pdev, mdev);
 
 	mdev->sdev = device_create_with_groups(g_mic_class, &pdev->dev,
@@ -210,9 +226,13 @@ static int mic_probe(struct pci_dev *pdev,
 		rc = PTR_ERR(mdev->sdev);
 		dev_err(&pdev->dev,
 			"device_create_with_groups failed rc %d\n", rc);
-		goto unmap_aper;
+		goto smpt_uninit;
 	}
 	return 0;
+smpt_uninit:
+	mic_smpt_uninit(mdev);
+free_interrupts:
+	mic_free_interrupts(mdev, pdev);
 unmap_aper:
 	iounmap(mdev->aper.va);
 unmap_mmio:
@@ -246,6 +266,8 @@ static void mic_remove(struct pci_dev *pdev)
 		return;
 
 	device_destroy(g_mic_class, MKDEV(MAJOR(g_mic_devno), mdev->id));
+	mic_smpt_uninit(mdev);
+	mic_free_interrupts(mdev, pdev);
 	iounmap(mdev->mmio.va);
 	iounmap(mdev->aper.va);
 	pci_release_regions(pdev);
drivers/misc/mic/host/mic_smpt.c new file +442 -0

442 lines not on this page · raw message

drivers/misc/mic/host/mic_smpt.h new file +98 -0

98 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.c +247 -0

262 lines not on this page · raw message

drivers/misc/mic/host/mic_x100.h +39 -0
index 1f4e630..642cae9 100644
@@ -42,6 +42,45 @@
 
 #define MIC_X100_SBOX_BASE_ADDRESS 0x00010000
 #define MIC_X100_SBOX_SPAD0 0x0000AB20
+#define MIC_X100_SBOX_SICR0_DBR(x) ((x) & 0xf)
+#define MIC_X100_SBOX_SICR0_DMA(x) (((x) >> 8) & 0xff)
+#define MIC_X100_SBOX_SICE0_DBR(x) ((x) & 0xf)
+#define MIC_X100_SBOX_DBR_BITS(x) ((x) & 0xf)
+#define MIC_X100_SBOX_SICE0_DMA(x) (((x) >> 8) & 0xff)
+#define MIC_X100_SBOX_DMA_BITS(x) (((x) & 0xff) << 8)
+
+#define MIC_X100_SBOX_APICICR0 0x0000A9D0
+#define MIC_X100_SBOX_SICR0 0x00009004
+#define MIC_X100_SBOX_SICE0 0x0000900C
+#define MIC_X100_SBOX_SICC0 0x00009010
+#define MIC_X100_SBOX_SIAC0 0x00009014
+#define MIC_X100_SBOX_MSIXPBACR 0x00009084
+#define MIC_X100_SBOX_MXAR0 0x00009044
+#define MIC_X100_SBOX_SMPT00 0x00003100
+#define MIC_X100_SBOX_RDMASR0 0x0000B180
+
+#define MIC_X100_DOORBELL_IDX_START 0
+#define MIC_X100_NUM_DOORBELL 4
+#define MIC_X100_DMA_IDX_START 8
+#define MIC_X100_NUM_DMA 8
+#define MIC_X100_ERR_IDX_START 30
+#define MIC_X100_NUM_ERR 1
+
+#define MIC_X100_NUM_SBOX_IRQ 8
+#define MIC_X100_NUM_RDMASR_IRQ 8
+#define MIC_X100_RDMASR_IRQ_BASE 17
+
+static const u16 mic_x100_intr_init[] = {
+		MIC_X100_DOORBELL_IDX_START,
+		MIC_X100_DMA_IDX_START,
+		MIC_X100_ERR_IDX_START,
+		MIC_X100_NUM_DOORBELL,
+		MIC_X100_NUM_DMA,
+		MIC_X100_NUM_ERR,
+};
+
 extern struct mic_hw_ops mic_x100_ops;
+extern struct mic_smpt_ops mic_x100_smpt_ops;
+extern struct mic_hw_intr_ops mic_x100_intr_ops;
 
 #endif
Signature · 1 line
1.8.2.1
#23 lore raw

PATCH RESEND v3 7/7 Sample Implementation of Intel MIC User Space Daemon.

In reply to Sudeep Dutt

From: Caz Yokoyama <[email protected]>

This patch introduces a sample user space daemon which implements the virtio device backends on the host. The daemon creates/removes/configures virtio device backends by communicating with the Intel MIC Host Driver. The virtio devices currently supported are virtio net, virtio console and virtio block. Virtio net supports TSO/GSO. The daemon also monitors card shutdown status and takes appropriate actions like killing the virtio backends and resetting the card upon card shutdown and crashes.

Co-author: Ashutosh Dixit <[email protected]>
Co-author: Sudeep Dutt <[email protected]>

---
 Documentation/mic/mic_overview.txt |   49 ++
 Documentation/mic/mpssd/.gitignore |    1 +
 Documentation/mic/mpssd/Makefile   |   19 +
 Documentation/mic/mpssd/micctrl    |  173 ++++
 Documentation/mic/mpssd/mpss       |  202 +++++
 Documentation/mic/mpssd/mpssd.c    | 1701 ++++++++++++++++++++++++++++++++++++
 Documentation/mic/mpssd/mpssd.h    |  100 +++
 Documentation/mic/mpssd/sysfs.c    |  102 +++
 8 files changed, 2347 insertions(+)
 create mode 100644 Documentation/mic/mic_overview.txt
 create mode 100644 Documentation/mic/mpssd/.gitignore
 create mode 100644 Documentation/mic/mpssd/Makefile
 create mode 100755 Documentation/mic/mpssd/micctrl
 create mode 100755 Documentation/mic/mpssd/mpss
 create mode 100644 Documentation/mic/mpssd/mpssd.c
 create mode 100644 Documentation/mic/mpssd/mpssd.h
 create mode 100644 Documentation/mic/mpssd/sysfs.c
Documentation/mic/mic_overview.txt new file +49 -0
new file mode 100644
index 0000000..c4424ed
@@ -0,0 +1,49 @@
+An Intel MIC X100 device is a PCIe form factor add-in coprocessor
+card based on the Intel Many Integrated Core (MIC) architecture
+that runs a Linux OS. It is a PCIe endpoint in a platform and therefore
+implements the three required standard address spaces i.e. configuration,
+memory and I/O. The host OS loads a device driver as is typical for
+PCIe devices. The card itself runs a bootstrap after reset that
+transfers control to the card OS downloaded from the host driver.
+The card OS as shipped by Intel is a Linux kernel with modifications
+for the X100 devices.
+
+Since it is a PCIe card, it does not have the ability to host hardware
+devices for networking, storage and console. We provide these devices
+on X100 coprocessors thus enabling a self-bootable equivalent environment
+for applications. A key benefit of our solution is that it leverages
+the standard virtio framework for network, disk and console devices,
+though in our case the virtio framework is used across a PCIe bus.
+
+Here is a block diagram of the various components described above. The
+virtio backends are situated on the host rather than the card given better
+single threaded performance for the host compared to MIC, the ability of
+the host to initiate DMA's to/from the card using the MIC DMA engine and
+the fact that the virtio block storage backend can only be on the host.
+
+                              |
+       +----------+           |             +----------+
+       | Card OS  |           |             | Host OS  |
+       +----------+           |             +----------+
+                              |
++-------+ +--------+ +------+ | +---------+  +--------+ +--------+
+| Virtio| |Virtio  | |Virtio| | |Virtio   |  |Virtio  | |Virtio  |
+| Net   | |Console | |Block | | |Net      |  |Console | |Block   |
+| Driver| |Driver  | |Driver| | |backend  |  |backend | |backend |
++-------+ +--------+ +------+ | +---------+  +--------+ +--------+
+    |         |         |     |      |            |         |
+    |         |         |     |User  |            |         |
+    |         |         |     |------|------------|---------|-------
+    +-------------------+     |Kernel +--------------------------+
+              |               |       | Virtio over PCIe IOCTLs  |
+              |               |       +--------------------------+
+      +--------------+        |                   |
+      |Intel MIC     |        |            +---------------+
+      |Card Driver   |        |            |Intel MIC      |
+      +--------------+        |            |Host Driver    |
+              |               |            +---------------+
+              |               |                   |
+     +-------------------------------------------------------------+
+     |                                                             |
+     |                    PCIe Bus                                 |
+     +-------------------------------------------------------------+
Documentation/mic/mpssd/.gitignore new file +1 -0
new file mode 100644
index 0000000..8b7c72f
@@ -0,0 +1 @@
+mpssd
Documentation/mic/mpssd/Makefile new file +19 -0
new file mode 100644
index 0000000..eb860a7
@@ -0,0 +1,19 @@
+#
+# Makefile - Intel MIC User Space Tools.
+# Copyright(c) 2013, Intel Corporation.
+#
+ifdef DEBUG
+CFLAGS += $(USERWARNFLAGS) -I. -g -Wall -DDEBUG=$(DEBUG)
+else
+CFLAGS += $(USERWARNFLAGS) -I. -g -Wall
+endif
+
+mpssd: mpssd.o sysfs.o
+	$(CC) $(CFLAGS) -o $@ $^ -lpthread
+
+install:
+	install mpssd /usr/sbin/mpssd
+	install micctrl /usr/sbin/micctrl
+
+clean:
+	rm -f mpssd *.o
Documentation/mic/mpssd/micctrl new file +173 -0

173 lines not on this page · raw message

Documentation/mic/mpssd/mpss new file +202 -0

202 lines not on this page · raw message

Documentation/mic/mpssd/mpssd.c new file +1701 -0

1,701 lines not on this page · raw message

Documentation/mic/mpssd/mpssd.h new file +100 -0

100 lines not on this page · raw message

Documentation/mic/mpssd/sysfs.c new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#21 lore raw

PATCH RESEND v3 5/7 Intel MIC Host Driver Changes for Virtio Devices.

In reply to Sudeep Dutt

From: Ashutosh Dixit <[email protected]>

This patch introduces the host "Virtio over PCIe" interface for Intel MIC. It allows creating user space backends on the host and instantiating virtio devices for them on the Intel MIC card. It uses the existing VRINGH infrastructure in the kernel to access virtio rings from the host. A character device per MIC is exposed with IOCTL, mmap and poll callbacks. This allows the user space backend to:
(a) add/remove a virtio device via a device page.
(b) map (R/O) virtio rings and device page to user space.
(c) poll for availability of data.
(d) copy a descriptor or entire descriptor chain to/from the card.
(e) modify virtio configuration.
(f) handle virtio device reset.
The buffers are copied over using CPU copies for this initial patch and host initiated MIC DMA support is planned for future patches. The avail and desc virtio rings are in host memory and the used ring is in card memory to maximize writes across PCIe for performance.

Co-author: Sudeep Dutt <[email protected]>

---
 drivers/misc/mic/Kconfig             |   1 +
 drivers/misc/mic/common/mic_device.h |   7 +
 drivers/misc/mic/host/Makefile       |   2 +
 drivers/misc/mic/host/mic_boot.c     |   3 +-
 drivers/misc/mic/host/mic_debugfs.c  | 140 +++++++
 drivers/misc/mic/host/mic_device.h   |   5 +
 drivers/misc/mic/host/mic_fops.c     | 221 +++++++++++
 drivers/misc/mic/host/mic_fops.h     |  32 ++
 drivers/misc/mic/host/mic_main.c     |  26 ++
 drivers/misc/mic/host/mic_virtio.c   | 703 +++++++++++++++++++++++++++++++++++
 drivers/misc/mic/host/mic_virtio.h   | 138 +++++++
 include/uapi/linux/Kbuild            |   1 +
 include/uapi/linux/mic_common.h      | 166 ++++++++-
 include/uapi/linux/mic_ioctl.h       |  74 ++++
 14 files changed, 1517 insertions(+), 2 deletions(-)
 create mode 100644 drivers/misc/mic/host/mic_fops.c
 create mode 100644 drivers/misc/mic/host/mic_fops.h
 create mode 100644 drivers/misc/mic/host/mic_virtio.c
 create mode 100644 drivers/misc/mic/host/mic_virtio.h
 create mode 100644 include/uapi/linux/mic_ioctl.h
drivers/misc/mic/Kconfig +1 -0
index 279a2e6..01f1a4a 100644
@@ -3,6 +3,7 @@ comment "Intel MIC Host Driver"
 config INTEL_MIC_HOST
 	tristate "Intel MIC Host Driver"
 	depends on 64BIT && PCI
+	select VHOST_RING
 	default N
 	help
 	  This enables Host Driver support for the Intel Many Integrated
drivers/misc/mic/common/mic_device.h +7 -0
index 6440e9d..01eb74f 100644
@@ -41,4 +41,11 @@ struct mic_mw {
 #define MIC_DPLO_SPAD 14
 #define MIC_DPHI_SPAD 15
 
+/*
+ * These values are supposed to be in the config_change field of the
+ * device page when the host sends a config change interrupt to the card.
+ */
+#define MIC_VIRTIO_PARAM_DEV_REMOVE 0x1
+#define MIC_VIRTIO_PARAM_CONFIG_CHANGED 0x2
+
 #endif
drivers/misc/mic/host/Makefile +2 -0
index a375dd3..c2197f9 100644
@@ -10,3 +10,5 @@ mic_host-objs += mic_smpt.o
 mic_host-objs += mic_intr.o
 mic_host-objs += mic_boot.o
 mic_host-objs += mic_debugfs.o
+mic_host-objs += mic_fops.o
+mic_host-objs += mic_virtio.o
drivers/misc/mic/host/mic_boot.c +2 -1
index 936fc58..fd9ff6d 100644
@@ -20,12 +20,12 @@
  */
 #include <linux/delay.h>
 #include <linux/firmware.h>
-#include <linux/interrupt.h>
 
 #include <linux/mic_common.h>
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_smpt.h"
+#include "mic_virtio.h"
 
 /**
  * mic_reset - Reset the MIC device.
@@ -117,6 +117,7 @@ void mic_stop(struct mic_device *mdev, bool force)
 {
 	mutex_lock(&mdev->mic_mutex);
 	if (MIC_OFFLINE != mdev->state || force) {
+		mic_virtio_reset_devices(mdev);
 		mic_bootparam_init(mdev);
 		mic_reset(mdev);
 		if (MIC_RESET_FAILED == mdev->state)
drivers/misc/mic/host/mic_debugfs.c +140 -0

171 lines not on this page · raw message

drivers/misc/mic/host/mic_device.h +5 -0
index 50b8b88..dcba2a5 100644
@@ -21,6 +21,7 @@
 #ifndef _MIC_DEVICE_H_
 #define _MIC_DEVICE_H_
 
+#include <linux/cdev.h>
 #include <linux/idr.h>
 
 #include "mic_intr.h"
@@ -80,6 +81,8 @@ enum mic_stepping {
  * @dp_dma_addr: virtio device page DMA address.
  * @shutdown_db: shutdown doorbell.
  * @shutdown_cookie: shutdown cookie.
+ * @cdev: Character device for MIC.
+ * @vdev_list: list of virtio devices.
  */
 struct mic_device {
 	struct mic_mw mmio;
@@ -113,6 +116,8 @@ struct mic_device {
 	dma_addr_t dp_dma_addr;
 	int shutdown_db;
 	struct mic_irq *shutdown_cookie;
+	struct cdev cdev;
+	struct list_head vdev_list;
 };
 
 /**
drivers/misc/mic/host/mic_fops.c new file +221 -0

221 lines not on this page · raw message

drivers/misc/mic/host/mic_fops.h new file +32 -0
new file mode 100644
index 0000000..dc3893d
@@ -0,0 +1,32 @@
+/*
+ * Intel MIC Platform Software Stack (MPSS)
+ *
+ * Copyright(c) 2013 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * Intel MIC Host driver.
+ *
+ */
+#ifndef _MIC_FOPS_H_
+#define _MIC_FOPS_H_
+
+int mic_open(struct inode *inode, struct file *filp);
+int mic_release(struct inode *inode, struct file *filp);
+ssize_t mic_read(struct file *filp, char __user *buf,
+			size_t count, loff_t *pos);
+long mic_ioctl(struct file *filp, unsigned int cmd, unsigned long arg);
+int mic_mmap(struct file *f, struct vm_area_struct *vma);
+unsigned int mic_poll(struct file *f, poll_table *wait);
+
+#endif
drivers/misc/mic/host/mic_main.c +26 -0
index 998a20a..a8965d4 100644
@@ -25,12 +25,15 @@
 #include <linux/fs.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/poll.h>
 
 #include <linux/mic_common.h>
 #include "../common/mic_device.h"
 #include "mic_device.h"
 #include "mic_x100.h"
 #include "mic_smpt.h"
+#include "mic_fops.h"
+#include "mic_virtio.h"
 
 static const char mic_driver_name[] = "mic";
 
@@ -64,6 +67,15 @@ static struct class *g_mic_class;
 /* Base device node number for MIC devices */
 static dev_t g_mic_devno;
 
+static const struct file_operations mic_fops = {
+	.open = mic_open,
+	.release = mic_release,
+	.unlocked_ioctl = mic_ioctl,
+	.poll = mic_poll,
+	.mmap = mic_mmap,
+	.owner = THIS_MODULE,
+};
+
 /* Initialize the device page */
 static int mic_dp_init(struct mic_device *mdev)
 {
@@ -193,6 +205,7 @@ mic_device_init(struct mic_device *mdev, struct pci_dev *pdev)
 	mdev->irq_info.next_avail_src = 0;
 	INIT_WORK(&mdev->reset_trigger_work, mic_reset_trigger_work);
 	INIT_WORK(&mdev->shutdown_work, mic_shutdown_work);
+	INIT_LIST_HEAD(&mdev->vdev_list);
 }
 
 /**
@@ -330,7 +343,19 @@ static int mic_probe(struct pci_dev *pdev,
 	mic_bootparam_init(mdev);
 
 	mic_create_debug_dir(mdev);
+	cdev_init(&mdev->cdev, &mic_fops);
+	mdev->cdev.owner = THIS_MODULE;
+	rc = cdev_add(&mdev->cdev, MKDEV(MAJOR(g_mic_devno), mdev->id), 1);
+	if (rc) {
+		dev_err(&pdev->dev, "cdev_add err id %d rc %d\n", mdev->id, rc);
+		goto cleanup_debug_dir;
+	}
 	return 0;
+cleanup_debug_dir:
+	mic_delete_debug_dir(mdev);
+	mutex_lock(&mdev->mic_mutex);
+	mic_free_irq(mdev, mdev->shutdown_cookie, mdev);
+	mutex_unlock(&mdev->mic_mutex);
 dp_uninit:
 	mic_dp_uninit(mdev);
 sysfs_put:
@@ -375,6 +400,7 @@ static void mic_remove(struct pci_dev *pdev)
 		return;
 
 	mic_stop(mdev, false);
+	cdev_del(&mdev->cdev);
 	mic_delete_debug_dir(mdev);
 	mutex_lock(&mdev->mic_mutex);
 	mic_free_irq(mdev, mdev->shutdown_cookie, mdev);
drivers/misc/mic/host/mic_virtio.c new file +703 -0

703 lines not on this page · raw message

drivers/misc/mic/host/mic_virtio.h new file +138 -0

138 lines not on this page · raw message

include/uapi/linux/Kbuild +1 -0
index e3f7b37..c73dbd0 100644
@@ -241,6 +241,7 @@ header-y += mei.h
 header-y += mempolicy.h
 header-y += meye.h
 header-y += mic_common.h
+header-y += mic_ioctl.h
 header-y += mii.h
 header-y += minix_fs.h
 header-y += mman.h
include/uapi/linux/mic_common.h +165 -1

178 lines not on this page · raw message

include/uapi/linux/mic_ioctl.h new file +74 -0

74 lines not on this page · raw message

Signature · 1 line
1.8.2.1
#24 lore raw
In reply to Sudeep Dutt

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

 drivers/misc/mic/card/mic_virtio.c  | 17 ++++---
 drivers/misc/mic/card/mic_x100.c    |  4 +-
 drivers/misc/mic/host/mic_debugfs.c | 91 ++++++++++++++++++-------------------
 drivers/misc/mic/host/mic_fops.c    |  6 +--
 drivers/misc/mic/host/mic_intr.c    | 37 ++++++++-------
 drivers/misc/mic/host/mic_smpt.c    | 17 +++----
 drivers/misc/mic/host/mic_sysfs.c   | 18 ++++----
 drivers/misc/mic/host/mic_virtio.c  | 34 ++++++--------
 drivers/misc/mic/host/mic_x100.c    | 29 ++++++------
 9 files changed, 122 insertions(+), 131 deletions(-)
drivers/misc/mic/card/mic_virtio.c +8 -9
index 38275c1..6071aec 100644
@@ -103,7 +103,7 @@ static void mic_finalize_features(struct virtio_device *vdev)
 	for (i = 0; i < bits; i++) {
 		if (test_bit(i, vdev->features))
 			iowrite8(ioread8(&out_features[i / 8]) | (1 << (i % 8)),
-				&out_features[i / 8]);
+				 &out_features[i / 8]);
 	}
 }
 
@@ -197,10 +197,9 @@ static void mic_notify(struct virtqueue *vq)
 static void mic_del_vq(struct virtqueue *vq, int n)
 {
 	struct mic_vdev *mvdev = to_micvdev(vq->vdev);
-	struct vring *vr = (struct vring *) (vq + 1);
+	struct vring *vr = (struct vring *)(vq + 1);
 
-	free_pages((unsigned long) vr->used,
-		get_order(mvdev->used_size[n]));
+	free_pages((unsigned long) vr->used, get_order(mvdev->used_size[n]));
 	vring_del_virtqueue(vq);
 	mic_card_unmap(mvdev->mdev, mvdev->vr[n]);
 	mvdev->vr[n] = NULL;
@@ -274,8 +273,8 @@ static struct virtqueue *mic_find_vq(struct virtio_device *vdev,
 	/* Allocate and reassign used ring now */
 	mvdev->used_size[index] = PAGE_ALIGN(sizeof(__u16) * 3 +
 			sizeof(struct vring_used_elem) * config.num);
-	used = (void *) __get_free_pages(GFP_KERNEL | __GFP_ZERO,
-				get_order(mvdev->used_size[index]));
+	used = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
+					get_order(mvdev->used_size[index]));
 	if (!used) {
 		err = -ENOMEM;
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
@@ -291,7 +290,7 @@ static struct virtqueue *mic_find_vq(struct virtio_device *vdev,
 	 * vring_new_virtqueue() would ensure that
 	 *  (&vq->vring == (struct vring *) (&vq->vq + 1));
 	 */
-	vr = (struct vring *) (vq + 1);
+	vr = (struct vring *)(vq + 1);
 	vr->used = used;
 
 	vq->priv = mvdev;
@@ -544,7 +543,7 @@ static void mic_scan_devices(struct mic_driver *mdrv, bool remove)
 		if (dev) {
 			if (remove)
 				iowrite8(MIC_VIRTIO_PARAM_DEV_REMOVE,
-					&dc->config_change);
+					 &dc->config_change);
 			put_device(dev);
 			mic_handle_config_change(d, i, mdrv);
 			ret = mic_remove_device(d, i, mdrv);
@@ -559,7 +558,7 @@ static void mic_scan_devices(struct mic_driver *mdrv, bool remove)
 
 		/* new device */
 		dev_dbg(mdrv->dev, "%s %d Adding new virtio device %p\n",
-				__func__, __LINE__, d);
+			__func__, __LINE__, d);
 		if (!remove)
 			mic_add_device(d, i, mdrv);
 	}
drivers/misc/mic/card/mic_x100.c +2 -2
index 7cb3469..e54dfcb 100644
@@ -66,8 +66,8 @@ void mic_send_intr(struct mic_device *mdev, int doorbell)
 	/* Ensure that the interrupt is ordered w.r.t previous stores. */
 	wmb();
 	mic_mmio_write(mw, MIC_X100_SBOX_SDBIC0_DBREQ_BIT,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		(MIC_X100_SBOX_SDBIC0 + (4 * doorbell)));
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       (MIC_X100_SBOX_SDBIC0 + (4 * doorbell)));
 }
 
 /**
drivers/misc/mic/host/mic_debugfs.c +43 -48

189 lines not on this page · raw message

drivers/misc/mic/host/mic_fops.c +3 -3
index 661469a..297d40e 100644
@@ -140,9 +140,9 @@ unsigned int mic_poll(struct file *f, poll_table *wait)
 
 	poll_wait(f, &mvdev->waitq, wait);
 
-	if (mic_vdev_inited(mvdev))
+	if (mic_vdev_inited(mvdev)) {
 		mask = POLLERR;
-	else if (mvdev->poll_wake) {
+	} else if (mvdev->poll_wake) {
 		mvdev->poll_wake = 0;
 		mask = POLLIN | POLLOUT;
 	}
@@ -152,7 +152,7 @@ unsigned int mic_poll(struct file *f, poll_table *wait)
 
 static inline int
 mic_query_offset(struct mic_vdev *mvdev, unsigned long offset,
-	unsigned long *size, unsigned long *pa)
+		 unsigned long *size, unsigned long *pa)
 {
 	struct mic_device *mdev = mvdev->mdev;
 	unsigned long start = MIC_DP_SIZE;
drivers/misc/mic/host/mic_intr.c +20 -17
index cdae314..cb13fe1 100644
@@ -71,8 +71,8 @@ static irqreturn_t mic_interrupt(int irq, void *dev)
 
 /* Return the interrupt offset from the index. Index is 0 based. */
 static u16 mic_map_src_to_offset(struct mic_device *mdev,
-		int intr_src, enum mic_intr_type type) {
-
+		int intr_src, enum mic_intr_type type)
+{
 	if (type >= MIC_NUM_INTR_TYPES)
 		return MIC_NUM_OFFSETS;
 	if (intr_src >= mdev->intr_info->intr_len[type])
@@ -112,7 +112,8 @@ static struct mic_intr_cb *mic_register_intr_callback(struct mic_device *mdev,
 	struct mic_intr_cb *intr_cb;
 	unsigned long flags;
 	int rc;
-	intr_cb = kmalloc(sizeof(struct mic_intr_cb), GFP_KERNEL);
+
+	intr_cb = kmalloc(sizeof(*intr_cb), GFP_KERNEL);
 
 	if (!intr_cb)
 		return ERR_PTR(-ENOMEM);
@@ -159,7 +160,7 @@ static u8 mic_unregister_intr_callback(struct mic_device *mdev, u32 idx)
 			if (intr_cb->cb_id == idx) {
 				list_del(pos);
 				ida_simple_remove(&mdev->irq_info.cb_ida,
-					intr_cb->cb_id);
+						  intr_cb->cb_id);
 				kfree(intr_cb);
 				spin_unlock_irqrestore(
 					&mdev->irq_info.mic_intr_lock, flags);
@@ -183,8 +184,9 @@ static int mic_setup_msix(struct mic_device *mdev, struct pci_dev *pdev)
 {
 	int rc, i;
 
-	mdev->irq_info.msix_entries = kmalloc(sizeof(struct msix_entry) *
-			MIC_MIN_MSIX, GFP_KERNEL);
+	mdev->irq_info.msix_entries = kmalloc_array(MIC_MIN_MSIX,
+						    sizeof(struct msix_entry),
+						    GFP_KERNEL);
 	if (!mdev->irq_info.msix_entries) {
 		rc = -ENOMEM;
 		goto err_nomem1;
@@ -231,8 +233,9 @@ static int mic_setup_callbacks(struct mic_device *mdev)
 {
 	int i;
 
-	mdev->irq_info.cb_list = kmalloc(sizeof(struct list_head) *
-		MIC_NUM_OFFSETS, GFP_KERNEL);
+	mdev->irq_info.cb_list = kmalloc_array(MIC_NUM_OFFSETS,
+					       sizeof(struct list_head),
+					       GFP_KERNEL);
 	if (!mdev->irq_info.cb_list)
 		return -ENOMEM;
 
@@ -261,7 +264,7 @@ static void mic_release_callbacks(struct mic_device *mdev)
 
 		if (list_empty(&mdev->irq_info.cb_list[i])) {
 			spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock,
-				flags);
+					       flags);
 			break;
 		}
 
@@ -269,7 +272,7 @@ static void mic_release_callbacks(struct mic_device *mdev)
 			intr_cb = list_entry(pos, struct mic_intr_cb, list);
 			list_del(pos);
 			ida_simple_remove(&mdev->irq_info.cb_ida,
-				intr_cb->cb_id);
+					  intr_cb->cb_id);
 			kfree(intr_cb);
 		}
 		spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
@@ -427,8 +430,8 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 	offset = mic_map_src_to_offset(mdev, intr_src, type);
 	if (offset >= MIC_NUM_OFFSETS) {
 		dev_err(mdev->sdev->parent,
-				"Error mapping index %d to a valid source id.\n",
-				intr_src);
+			"Error mapping index %d to a valid source id\n",
+			intr_src);
 		rc = -EINVAL;
 		goto err;
 	}
@@ -437,7 +440,7 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 		msix = mic_get_available_vector(mdev);
 		if (!msix) {
 			dev_err(mdev->sdev->parent,
-			"No MSIx vectors available for use.\n");
+				"No MSIx vectors available for use\n");
 			rc = -ENOSPC;
 			goto err;
 		}
@@ -460,7 +463,7 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 				offset, func, data);
 		if (IS_ERR(intr_cb)) {
 			dev_err(mdev->sdev->parent,
-			"No available callback entries for use\n");
+				"No available callback entries for use\n");
 			rc = PTR_ERR(intr_cb);
 			goto err;
 		}
@@ -506,8 +509,8 @@ void mic_free_irq(struct mic_device *mdev,
 	if (mdev->irq_info.num_vectors > 1) {
 		if (entry >= mdev->irq_info.num_vectors) {
 			dev_warn(mdev->sdev->parent,
-				"entry %d should be < num_irq %d\n",
-				entry, mdev->irq_info.num_vectors);
+				 "entry %d should be < num_irq %d\n",
+				 entry, mdev->irq_info.num_vectors);
 			return;
 		}
 		irq = mdev->irq_info.msix_entries[entry].vector;
@@ -581,7 +584,7 @@ void mic_free_interrupts(struct mic_device *mdev, struct pci_dev *pdev)
 		for (i = 0; i < mdev->irq_info.num_vectors; i++) {
 			if (mdev->irq_info.mic_msi_map[i])
 				dev_warn(&pdev->dev, "irq %d may still be in use.\n",
-					mdev->irq_info.msix_entries[i].vector);
+					 mdev->irq_info.msix_entries[i].vector);
 		}
 		kfree(mdev->irq_info.mic_msi_map);
 		kfree(mdev->irq_info.msix_entries);
drivers/misc/mic/host/mic_smpt.c +9 -8
index a346207..6d5eba6 100644
@@ -84,7 +84,7 @@ static void mic_add_smpt_entry(int spt, s64 *ref, u64 addr,
 	for (i = spt; i < spt + entries; i++,
 		addr += smpt_info->info.page_size) {
 		if (!smpt_info->entry[i].ref_count &&
-			(smpt_info->entry[i].dma_addr != addr)) {
+		    (smpt_info->entry[i].dma_addr != addr)) {
 			mdev->smpt_ops->set(mdev, addr, i);
 			smpt_info->entry[i].dma_addr = addr;
 		}
@@ -183,7 +183,7 @@ mic_to_dma_addr(struct mic_device *mdev, dma_addr_t mic_addr)
 
 	if (!mic_is_system_addr(mdev, mic_addr)) {
 		dev_err(mdev->sdev->parent,
-		"mic_addr is invalid. mic_addr = 0x%llx\n", mic_addr);
+			"mic_addr is invalid. mic_addr = 0x%llx\n", mic_addr);
 		return -EINVAL;
 	}
 	spt = mic_sys_addr_to_smpt(mdev, mic_addr);
@@ -286,7 +286,7 @@ void mic_unmap(struct mic_device *mdev, dma_addr_t mic_addr, size_t size)
 		smpt_info->entry[i].ref_count -= ref[i - spt];
 		if (smpt_info->entry[i].ref_count < 0)
 			dev_warn(mdev->sdev->parent,
-				"ref count for entry %d is negative\n", i);
+				 "ref count for entry %d is negative\n", i);
 	}
 	spin_unlock_irqrestore(&smpt_info->smpt_lock, flags);
 	kfree(ref);
@@ -319,8 +319,8 @@ dma_addr_t mic_map_single(struct mic_device *mdev, void *va, size_t size)
 			dev_err(mdev->sdev->parent,
 				"mic_map failed dma_addr 0x%llx size 0x%lx\n",
 				dma_addr, size);
-			pci_unmap_single(pdev, dma_addr,
-				size, PCI_DMA_BIDIRECTIONAL);
+			pci_unmap_single(pdev, dma_addr, size,
+					 PCI_DMA_BIDIRECTIONAL);
 		}
 	}
 	return mic_addr;
@@ -366,8 +366,9 @@ int mic_smpt_init(struct mic_device *mdev)
 
 	smpt_info = mdev->smpt;
 	mdev->smpt_ops->init(mdev);
-	smpt_info->entry = kmalloc(sizeof(struct mic_smpt)
-			* smpt_info->info.num_reg, GFP_KERNEL);
+	smpt_info->entry = kmalloc_array(smpt_info->info.num_reg,
+					 sizeof(struct mic_smpt),
+					 GFP_KERNEL);
 	if (!smpt_info->entry) {
 		err = -ENOMEM;
 		goto free_smpt;
@@ -412,7 +413,7 @@ void mic_smpt_uninit(struct mic_device *mdev)
 			smpt_info->entry[i].ref_count);
 		if (smpt_info->entry[i].ref_count)
 			dev_warn(mdev->sdev->parent,
-			"ref count for entry %d is not zero\n", i);
+				 "ref count for entry %d is not zero\n", i);
 	}
 	kfree(smpt_info->entry);
 	kfree(smpt_info);
drivers/misc/mic/host/mic_sysfs.c +9 -9
index aaf8499..4d8bb09 100644
@@ -130,7 +130,7 @@ mic_show_state(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_state(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		const char *buf, size_t count)
 {
 	int rc = 0;
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
@@ -194,7 +194,7 @@ mic_show_cmdline(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_cmdline(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		  const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -241,7 +241,7 @@ mic_show_firmware(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_firmware(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		   const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -287,7 +287,7 @@ mic_show_ramdisk(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_ramdisk(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		  const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -334,7 +334,7 @@ mic_show_bootmode(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_bootmode(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		   const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -368,7 +368,7 @@ static DEVICE_ATTR(bootmode, S_IRUGO | S_IWUSR,
 
 static ssize_t
 mic_show_log_buf_addr(struct device *dev, struct device_attribute *attr,
-	char *buf)
+		      char *buf)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -380,7 +380,7 @@ mic_show_log_buf_addr(struct device *dev, struct device_attribute *attr,
 
 static ssize_t
 mic_store_log_buf_addr(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		       const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 	int ret;
@@ -403,7 +403,7 @@ static DEVICE_ATTR(log_buf_addr, S_IRUGO | S_IWUSR,
 
 static ssize_t
 mic_show_log_buf_len(struct device *dev, struct device_attribute *attr,
-	char *buf)
+		     char *buf)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -415,7 +415,7 @@ mic_show_log_buf_len(struct device *dev, struct device_attribute *attr,
 
 static ssize_t
 mic_store_log_buf_len(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		      const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 	int ret;
drivers/misc/mic/host/mic_virtio.c +15 -19
index be2a1f0..4ac3096 100644
@@ -195,7 +195,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 			MIC_VRINGH_READ, &out_len);
 		if (ret) {
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
-					__func__, __LINE__, ret);
+				__func__, __LINE__, ret);
 			break;
 		}
 		len -= out_len;
@@ -206,7 +206,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 			!MIC_VRINGH_READ, &out_len);
 		if (ret) {
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
-					__func__, __LINE__, ret);
+				__func__, __LINE__, ret);
 			break;
 		}
 		len -= out_len;
@@ -225,8 +225,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 	 * Update the used ring if a descriptor was available and some data was
 	 * copied in/out and the user asked for a used ring update.
 	 */
-	if (*head != USHRT_MAX && copy->out_len &&
-		copy->update_used) {
+	if (*head != USHRT_MAX && copy->out_len && copy->update_used) {
 		u32 total = 0;
 
 		/* Determine the total data consumed */
@@ -367,7 +366,6 @@ void mic_bh_handler(struct work_struct *work)
 
 static irqreturn_t mic_virtio_intr_handler(int irq, void *data)
 {
-
 	struct mic_vdev *mvdev = data;
 	struct mic_device *mdev = mvdev->mdev;
 
@@ -394,7 +392,7 @@ int mic_virtio_config_change(struct mic_vdev *mvdev,
 	}
 
 	if (copy_from_user(mic_vq_configspace(mvdev->dd),
-				argp, mvdev->dd->config_len)) {
+			   argp, mvdev->dd->config_len)) {
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
 			__func__, __LINE__, -EFAULT);
 		ret = -EFAULT;
@@ -438,8 +436,8 @@ static int mic_copy_dp_entry(struct mic_vdev *mvdev,
 		return -EFAULT;
 	}
 
-	if (mic_aligned_desc_size(&dd) > MIC_MAX_DESC_BLK_SIZE
-		|| dd.num_vq > MIC_MAX_VRINGS) {
+	if (mic_aligned_desc_size(&dd) > MIC_MAX_DESC_BLK_SIZE ||
+	    dd.num_vq > MIC_MAX_VRINGS) {
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
 			__func__, __LINE__, -EINVAL);
 		return -EINVAL;
@@ -551,7 +549,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 			sizeof(struct _mic_vring_info));
 		vr->va = (void *)
 			__get_free_pages(GFP_KERNEL | __GFP_ZERO,
-			get_order(vr_size));
+					 get_order(vr_size));
 		if (!vr->va) {
 			ret = -ENOMEM;
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
@@ -564,8 +562,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 		vqconfig[i].address = mic_map_single(mdev,
 			vr->va, vr_size);
 		if (mic_map_error(vqconfig[i].address)) {
-			free_pages((unsigned long)vr->va,
-				get_order(vr_size));
+			free_pages((unsigned long)vr->va, get_order(vr_size));
 			ret = -ENOMEM;
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
 				__func__, __LINE__, ret);
@@ -573,8 +570,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 		}
 		vqconfig[i].address = cpu_to_le64(vqconfig[i].address);
 
-		vring_init(&vr->vr, num,
-			vr->va, MIC_VIRTIO_RING_ALIGN);
+		vring_init(&vr->vr, num, vr->va, MIC_VIRTIO_RING_ALIGN);
 		ret = vringh_init_kern(&mvr->vrh,
 			*(u32 *)mic_vq_features(mvdev->dd), num, false,
 			vr->vr.desc, vr->vr.avail, vr->vr.used);
@@ -593,8 +589,8 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 			__func__, __LINE__, i, vr->va, vr->info, vr_size);
 	}
 
-	snprintf(irqname, sizeof(irqname),
-		"mic%dvirtio%d", mdev->id, mvdev->virtio_id);
+	snprintf(irqname, sizeof(irqname), "mic%dvirtio%d",
+		 mdev->id, mvdev->virtio_id);
 	mvdev->virtio_db = mic_next_db(mdev);
 	mvdev->virtio_cookie = mic_request_irq(mdev, mic_virtio_intr_handler,
 			irqname, mvdev, mvdev->virtio_db, MIC_INTR_DB);
@@ -628,9 +624,9 @@ err:
 	for (j = 0; j < i; j++) {
 		struct mic_vringh *mvr = &mvdev->mvr[j];
 		mic_unmap_single(mdev, le64_to_cpu(vqconfig[j].address),
-				mvr->vring.len);
+				 mvr->vring.len);
 		free_pages((unsigned long)mvr->vring.va,
-			get_order(mvr->vring.len));
+			   get_order(mvr->vring.len));
 	}
 	mutex_unlock(&mdev->mic_mutex);
 	return ret;
@@ -676,9 +672,9 @@ skip_hot_remove:
 		vringh_kiov_cleanup(&mvr->riov);
 		vringh_kiov_cleanup(&mvr->wiov);
 		mic_unmap_single(mdev, le64_to_cpu(vqconfig[i].address),
-				mvr->vring.len);
+				 mvr->vring.len);
 		free_pages((unsigned long)mvr->vring.va,
-			get_order(mvr->vring.len));
+			   get_order(mvr->vring.len));
 	}
 
 	list_for_each_safe(pos, tmp, &mdev->vdev_list) {
drivers/misc/mic/host/mic_x100.c +13 -16
index a12ae5c..a611c28 100644
@@ -46,8 +46,8 @@ mic_x100_write_spad(struct mic_device *mdev, unsigned int idx, u32 val)
 	dev_dbg(mdev->sdev->parent, "Writing 0x%x to scratch pad index %d\n",
 		val, idx);
 	mic_mmio_write(&mdev->mmio, val,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		MIC_X100_SBOX_SPAD0 + idx * 4);
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       MIC_X100_SBOX_SPAD0 + idx * 4);
 }
 
 /**
@@ -139,7 +139,7 @@ static void mic_x100_send_sbox_intr(struct mic_device *mdev,
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(mw, apicicr_low,
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
 }
 
 /**
@@ -153,7 +153,7 @@ static void mic_x100_send_rdmasr_intr(struct mic_device *mdev,
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(&mdev->mmio, 0,
-		MIC_X100_SBOX_BASE_ADDRESS + rdmasr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + rdmasr_offset);
 }
 
 /**
@@ -212,7 +212,7 @@ done:
  */
 static void mic_x100_hw_intr_init(struct mic_device *mdev)
 {
-	mdev->intr_info = (struct mic_intr_info *) mic_x100_intr_init;
+	mdev->intr_info = (struct mic_intr_info *)mic_x100_intr_init;
 }
 
 /**
@@ -244,7 +244,7 @@ mic_x100_read_msi_to_src_map(struct mic_device *mdev, int idx)
  */
 static void
 mic_x100_program_msi_to_src_map(struct mic_device *mdev,
-			int idx, int offset, bool set)
+				int idx, int offset, bool set)
 {
 	unsigned long reg;
 	struct mic_mw *mw = &mdev->mmio;
@@ -308,12 +308,12 @@ static void mic_x100_send_firmware_intr(struct mic_device *mdev)
 	apicicr_low = (vector | (1 << 13));
 
 	mic_mmio_write(mw, mic_x100_get_apic_id(mdev),
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset + 4);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset + 4);
 
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(mw, apicicr_low,
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
 }
 
 /**
@@ -365,8 +365,7 @@ mic_x100_load_command_line(struct mic_device *mdev, const struct firmware *fw)
 	len += snprintf(buf, CMDLINE_SIZE - len,
 		" mem=%dM", boot_mem);
 	if (mdev->cmdline)
-		snprintf(buf + len, CMDLINE_SIZE - len,
-				" %s", mdev->cmdline);
+		snprintf(buf + len, CMDLINE_SIZE - len, " %s", mdev->cmdline);
 	memcpy_toio(cmd_line_va, buf, strlen(buf) + 1);
 	kfree(buf);
 	return 0;
@@ -397,8 +396,7 @@ mic_x100_load_ramdisk(struct mic_device *mdev)
 	 * Typically the bootaddr for card OS is 64M
 	 * so copy over the ramdisk @ 128M.
 	 */
-	memcpy_toio(mdev->aper.va + (mdev->bootaddr << 1),
-		fw->data, fw->size);
+	memcpy_toio(mdev->aper.va + (mdev->bootaddr << 1), fw->data, fw->size);
 	iowrite32(cpu_to_le32(mdev->bootaddr << 1), &bp->hdr.ramdisk_image);
 	iowrite32(cpu_to_le32(fw->size), &bp->hdr.ramdisk_size);
 	release_firmware(fw);
@@ -484,8 +482,7 @@ mic_x100_load_firmware(struct mic_device *mdev, const char *buf)
 	if (mdev->ramdisk)
 		rc = mic_x100_load_ramdisk(mdev);
 error:
-	dev_dbg(mdev->sdev->parent, "%s %d rc %d\n",
-			__func__, __LINE__, rc);
+	dev_dbg(mdev->sdev->parent, "%s %d rc %d\n", __func__, __LINE__, rc);
 done:
 	return rc;
 }
@@ -524,8 +521,8 @@ mic_x100_smpt_set(struct mic_device *mdev, dma_addr_t dma_addr, u8 index)
 	uint32_t smpt_reg_val = BUILD_SMPT(SNOOP_ON,
 			dma_addr >> mdev->smpt->info.page_shift);
 	mic_mmio_write(&mdev->mmio, smpt_reg_val,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		MIC_X100_SBOX_SMPT00 + (4 * index));
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       MIC_X100_SBOX_SMPT00 + (4 * index));
 }
 
 /**
#25 lore raw
In reply to Sudeep Dutt

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

 drivers/misc/mic/card/mic_virtio.c  | 17 ++++---
 drivers/misc/mic/card/mic_x100.c    |  4 +-
 drivers/misc/mic/host/mic_debugfs.c | 91 ++++++++++++++++++-------------------
 drivers/misc/mic/host/mic_fops.c    |  6 +--
 drivers/misc/mic/host/mic_intr.c    | 37 ++++++++-------
 drivers/misc/mic/host/mic_smpt.c    | 17 +++----
 drivers/misc/mic/host/mic_sysfs.c   | 18 ++++----
 drivers/misc/mic/host/mic_virtio.c  | 34 ++++++--------
 drivers/misc/mic/host/mic_x100.c    | 29 ++++++------
 9 files changed, 122 insertions(+), 131 deletions(-)
drivers/misc/mic/card/mic_virtio.c +8 -9
index 38275c1..6071aec 100644
@@ -103,7 +103,7 @@ static void mic_finalize_features(struct virtio_device *vdev)
 	for (i = 0; i < bits; i++) {
 		if (test_bit(i, vdev->features))
 			iowrite8(ioread8(&out_features[i / 8]) | (1 << (i % 8)),
-				&out_features[i / 8]);
+				 &out_features[i / 8]);
 	}
 }
 
@@ -197,10 +197,9 @@ static void mic_notify(struct virtqueue *vq)
 static void mic_del_vq(struct virtqueue *vq, int n)
 {
 	struct mic_vdev *mvdev = to_micvdev(vq->vdev);
-	struct vring *vr = (struct vring *) (vq + 1);
+	struct vring *vr = (struct vring *)(vq + 1);
 
-	free_pages((unsigned long) vr->used,
-		get_order(mvdev->used_size[n]));
+	free_pages((unsigned long) vr->used, get_order(mvdev->used_size[n]));
 	vring_del_virtqueue(vq);
 	mic_card_unmap(mvdev->mdev, mvdev->vr[n]);
 	mvdev->vr[n] = NULL;
@@ -274,8 +273,8 @@ static struct virtqueue *mic_find_vq(struct virtio_device *vdev,
 	/* Allocate and reassign used ring now */
 	mvdev->used_size[index] = PAGE_ALIGN(sizeof(__u16) * 3 +
 			sizeof(struct vring_used_elem) * config.num);
-	used = (void *) __get_free_pages(GFP_KERNEL | __GFP_ZERO,
-				get_order(mvdev->used_size[index]));
+	used = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
+					get_order(mvdev->used_size[index]));
 	if (!used) {
 		err = -ENOMEM;
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
@@ -291,7 +290,7 @@ static struct virtqueue *mic_find_vq(struct virtio_device *vdev,
 	 * vring_new_virtqueue() would ensure that
 	 *  (&vq->vring == (struct vring *) (&vq->vq + 1));
 	 */
-	vr = (struct vring *) (vq + 1);
+	vr = (struct vring *)(vq + 1);
 	vr->used = used;
 
 	vq->priv = mvdev;
@@ -544,7 +543,7 @@ static void mic_scan_devices(struct mic_driver *mdrv, bool remove)
 		if (dev) {
 			if (remove)
 				iowrite8(MIC_VIRTIO_PARAM_DEV_REMOVE,
-					&dc->config_change);
+					 &dc->config_change);
 			put_device(dev);
 			mic_handle_config_change(d, i, mdrv);
 			ret = mic_remove_device(d, i, mdrv);
@@ -559,7 +558,7 @@ static void mic_scan_devices(struct mic_driver *mdrv, bool remove)
 
 		/* new device */
 		dev_dbg(mdrv->dev, "%s %d Adding new virtio device %p\n",
-				__func__, __LINE__, d);
+			__func__, __LINE__, d);
 		if (!remove)
 			mic_add_device(d, i, mdrv);
 	}
drivers/misc/mic/card/mic_x100.c +2 -2
index 7cb3469..e54dfcb 100644
@@ -66,8 +66,8 @@ void mic_send_intr(struct mic_device *mdev, int doorbell)
 	/* Ensure that the interrupt is ordered w.r.t previous stores. */
 	wmb();
 	mic_mmio_write(mw, MIC_X100_SBOX_SDBIC0_DBREQ_BIT,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		(MIC_X100_SBOX_SDBIC0 + (4 * doorbell)));
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       (MIC_X100_SBOX_SDBIC0 + (4 * doorbell)));
 }
 
 /**
drivers/misc/mic/host/mic_debugfs.c +43 -48

189 lines not on this page · raw message

drivers/misc/mic/host/mic_fops.c +3 -3
index 661469a..297d40e 100644
@@ -140,9 +140,9 @@ unsigned int mic_poll(struct file *f, poll_table *wait)
 
 	poll_wait(f, &mvdev->waitq, wait);
 
-	if (mic_vdev_inited(mvdev))
+	if (mic_vdev_inited(mvdev)) {
 		mask = POLLERR;
-	else if (mvdev->poll_wake) {
+	} else if (mvdev->poll_wake) {
 		mvdev->poll_wake = 0;
 		mask = POLLIN | POLLOUT;
 	}
@@ -152,7 +152,7 @@ unsigned int mic_poll(struct file *f, poll_table *wait)
 
 static inline int
 mic_query_offset(struct mic_vdev *mvdev, unsigned long offset,
-	unsigned long *size, unsigned long *pa)
+		 unsigned long *size, unsigned long *pa)
 {
 	struct mic_device *mdev = mvdev->mdev;
 	unsigned long start = MIC_DP_SIZE;
drivers/misc/mic/host/mic_intr.c +20 -17
index cdae314..cb13fe1 100644
@@ -71,8 +71,8 @@ static irqreturn_t mic_interrupt(int irq, void *dev)
 
 /* Return the interrupt offset from the index. Index is 0 based. */
 static u16 mic_map_src_to_offset(struct mic_device *mdev,
-		int intr_src, enum mic_intr_type type) {
-
+		int intr_src, enum mic_intr_type type)
+{
 	if (type >= MIC_NUM_INTR_TYPES)
 		return MIC_NUM_OFFSETS;
 	if (intr_src >= mdev->intr_info->intr_len[type])
@@ -112,7 +112,8 @@ static struct mic_intr_cb *mic_register_intr_callback(struct mic_device *mdev,
 	struct mic_intr_cb *intr_cb;
 	unsigned long flags;
 	int rc;
-	intr_cb = kmalloc(sizeof(struct mic_intr_cb), GFP_KERNEL);
+
+	intr_cb = kmalloc(sizeof(*intr_cb), GFP_KERNEL);
 
 	if (!intr_cb)
 		return ERR_PTR(-ENOMEM);
@@ -159,7 +160,7 @@ static u8 mic_unregister_intr_callback(struct mic_device *mdev, u32 idx)
 			if (intr_cb->cb_id == idx) {
 				list_del(pos);
 				ida_simple_remove(&mdev->irq_info.cb_ida,
-					intr_cb->cb_id);
+						  intr_cb->cb_id);
 				kfree(intr_cb);
 				spin_unlock_irqrestore(
 					&mdev->irq_info.mic_intr_lock, flags);
@@ -183,8 +184,9 @@ static int mic_setup_msix(struct mic_device *mdev, struct pci_dev *pdev)
 {
 	int rc, i;
 
-	mdev->irq_info.msix_entries = kmalloc(sizeof(struct msix_entry) *
-			MIC_MIN_MSIX, GFP_KERNEL);
+	mdev->irq_info.msix_entries = kmalloc_array(MIC_MIN_MSIX,
+						    sizeof(struct msix_entry),
+						    GFP_KERNEL);
 	if (!mdev->irq_info.msix_entries) {
 		rc = -ENOMEM;
 		goto err_nomem1;
@@ -231,8 +233,9 @@ static int mic_setup_callbacks(struct mic_device *mdev)
 {
 	int i;
 
-	mdev->irq_info.cb_list = kmalloc(sizeof(struct list_head) *
-		MIC_NUM_OFFSETS, GFP_KERNEL);
+	mdev->irq_info.cb_list = kmalloc_array(MIC_NUM_OFFSETS,
+					       sizeof(struct list_head),
+					       GFP_KERNEL);
 	if (!mdev->irq_info.cb_list)
 		return -ENOMEM;
 
@@ -261,7 +264,7 @@ static void mic_release_callbacks(struct mic_device *mdev)
 
 		if (list_empty(&mdev->irq_info.cb_list[i])) {
 			spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock,
-				flags);
+					       flags);
 			break;
 		}
 
@@ -269,7 +272,7 @@ static void mic_release_callbacks(struct mic_device *mdev)
 			intr_cb = list_entry(pos, struct mic_intr_cb, list);
 			list_del(pos);
 			ida_simple_remove(&mdev->irq_info.cb_ida,
-				intr_cb->cb_id);
+					  intr_cb->cb_id);
 			kfree(intr_cb);
 		}
 		spin_unlock_irqrestore(&mdev->irq_info.mic_intr_lock, flags);
@@ -427,8 +430,8 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 	offset = mic_map_src_to_offset(mdev, intr_src, type);
 	if (offset >= MIC_NUM_OFFSETS) {
 		dev_err(mdev->sdev->parent,
-				"Error mapping index %d to a valid source id.\n",
-				intr_src);
+			"Error mapping index %d to a valid source id\n",
+			intr_src);
 		rc = -EINVAL;
 		goto err;
 	}
@@ -437,7 +440,7 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 		msix = mic_get_available_vector(mdev);
 		if (!msix) {
 			dev_err(mdev->sdev->parent,
-			"No MSIx vectors available for use.\n");
+				"No MSIx vectors available for use\n");
 			rc = -ENOSPC;
 			goto err;
 		}
@@ -460,7 +463,7 @@ struct mic_irq *mic_request_irq(struct mic_device *mdev,
 				offset, func, data);
 		if (IS_ERR(intr_cb)) {
 			dev_err(mdev->sdev->parent,
-			"No available callback entries for use\n");
+				"No available callback entries for use\n");
 			rc = PTR_ERR(intr_cb);
 			goto err;
 		}
@@ -506,8 +509,8 @@ void mic_free_irq(struct mic_device *mdev,
 	if (mdev->irq_info.num_vectors > 1) {
 		if (entry >= mdev->irq_info.num_vectors) {
 			dev_warn(mdev->sdev->parent,
-				"entry %d should be < num_irq %d\n",
-				entry, mdev->irq_info.num_vectors);
+				 "entry %d should be < num_irq %d\n",
+				 entry, mdev->irq_info.num_vectors);
 			return;
 		}
 		irq = mdev->irq_info.msix_entries[entry].vector;
@@ -581,7 +584,7 @@ void mic_free_interrupts(struct mic_device *mdev, struct pci_dev *pdev)
 		for (i = 0; i < mdev->irq_info.num_vectors; i++) {
 			if (mdev->irq_info.mic_msi_map[i])
 				dev_warn(&pdev->dev, "irq %d may still be in use.\n",
-					mdev->irq_info.msix_entries[i].vector);
+					 mdev->irq_info.msix_entries[i].vector);
 		}
 		kfree(mdev->irq_info.mic_msi_map);
 		kfree(mdev->irq_info.msix_entries);
drivers/misc/mic/host/mic_smpt.c +9 -8
index a346207..6d5eba6 100644
@@ -84,7 +84,7 @@ static void mic_add_smpt_entry(int spt, s64 *ref, u64 addr,
 	for (i = spt; i < spt + entries; i++,
 		addr += smpt_info->info.page_size) {
 		if (!smpt_info->entry[i].ref_count &&
-			(smpt_info->entry[i].dma_addr != addr)) {
+		    (smpt_info->entry[i].dma_addr != addr)) {
 			mdev->smpt_ops->set(mdev, addr, i);
 			smpt_info->entry[i].dma_addr = addr;
 		}
@@ -183,7 +183,7 @@ mic_to_dma_addr(struct mic_device *mdev, dma_addr_t mic_addr)
 
 	if (!mic_is_system_addr(mdev, mic_addr)) {
 		dev_err(mdev->sdev->parent,
-		"mic_addr is invalid. mic_addr = 0x%llx\n", mic_addr);
+			"mic_addr is invalid. mic_addr = 0x%llx\n", mic_addr);
 		return -EINVAL;
 	}
 	spt = mic_sys_addr_to_smpt(mdev, mic_addr);
@@ -286,7 +286,7 @@ void mic_unmap(struct mic_device *mdev, dma_addr_t mic_addr, size_t size)
 		smpt_info->entry[i].ref_count -= ref[i - spt];
 		if (smpt_info->entry[i].ref_count < 0)
 			dev_warn(mdev->sdev->parent,
-				"ref count for entry %d is negative\n", i);
+				 "ref count for entry %d is negative\n", i);
 	}
 	spin_unlock_irqrestore(&smpt_info->smpt_lock, flags);
 	kfree(ref);
@@ -319,8 +319,8 @@ dma_addr_t mic_map_single(struct mic_device *mdev, void *va, size_t size)
 			dev_err(mdev->sdev->parent,
 				"mic_map failed dma_addr 0x%llx size 0x%lx\n",
 				dma_addr, size);
-			pci_unmap_single(pdev, dma_addr,
-				size, PCI_DMA_BIDIRECTIONAL);
+			pci_unmap_single(pdev, dma_addr, size,
+					 PCI_DMA_BIDIRECTIONAL);
 		}
 	}
 	return mic_addr;
@@ -366,8 +366,9 @@ int mic_smpt_init(struct mic_device *mdev)
 
 	smpt_info = mdev->smpt;
 	mdev->smpt_ops->init(mdev);
-	smpt_info->entry = kmalloc(sizeof(struct mic_smpt)
-			* smpt_info->info.num_reg, GFP_KERNEL);
+	smpt_info->entry = kmalloc_array(smpt_info->info.num_reg,
+					 sizeof(struct mic_smpt),
+					 GFP_KERNEL);
 	if (!smpt_info->entry) {
 		err = -ENOMEM;
 		goto free_smpt;
@@ -412,7 +413,7 @@ void mic_smpt_uninit(struct mic_device *mdev)
 			smpt_info->entry[i].ref_count);
 		if (smpt_info->entry[i].ref_count)
 			dev_warn(mdev->sdev->parent,
-			"ref count for entry %d is not zero\n", i);
+				 "ref count for entry %d is not zero\n", i);
 	}
 	kfree(smpt_info->entry);
 	kfree(smpt_info);
drivers/misc/mic/host/mic_sysfs.c +9 -9
index aaf8499..4d8bb09 100644
@@ -130,7 +130,7 @@ mic_show_state(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_state(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		const char *buf, size_t count)
 {
 	int rc = 0;
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
@@ -194,7 +194,7 @@ mic_show_cmdline(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_cmdline(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		  const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -241,7 +241,7 @@ mic_show_firmware(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_firmware(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		   const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -287,7 +287,7 @@ mic_show_ramdisk(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_ramdisk(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		  const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -334,7 +334,7 @@ mic_show_bootmode(struct device *dev, struct device_attribute *attr, char *buf)
 
 static ssize_t
 mic_store_bootmode(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		   const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -368,7 +368,7 @@ static DEVICE_ATTR(bootmode, S_IRUGO | S_IWUSR,
 
 static ssize_t
 mic_show_log_buf_addr(struct device *dev, struct device_attribute *attr,
-	char *buf)
+		      char *buf)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -380,7 +380,7 @@ mic_show_log_buf_addr(struct device *dev, struct device_attribute *attr,
 
 static ssize_t
 mic_store_log_buf_addr(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		       const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 	int ret;
@@ -403,7 +403,7 @@ static DEVICE_ATTR(log_buf_addr, S_IRUGO | S_IWUSR,
 
 static ssize_t
 mic_show_log_buf_len(struct device *dev, struct device_attribute *attr,
-	char *buf)
+		     char *buf)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 
@@ -415,7 +415,7 @@ mic_show_log_buf_len(struct device *dev, struct device_attribute *attr,
 
 static ssize_t
 mic_store_log_buf_len(struct device *dev, struct device_attribute *attr,
-	const char *buf, size_t count)
+		      const char *buf, size_t count)
 {
 	struct mic_device *mdev = dev_get_drvdata(dev->parent);
 	int ret;
drivers/misc/mic/host/mic_virtio.c +15 -19
index be2a1f0..4ac3096 100644
@@ -195,7 +195,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 			MIC_VRINGH_READ, &out_len);
 		if (ret) {
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
-					__func__, __LINE__, ret);
+				__func__, __LINE__, ret);
 			break;
 		}
 		len -= out_len;
@@ -206,7 +206,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 			!MIC_VRINGH_READ, &out_len);
 		if (ret) {
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
-					__func__, __LINE__, ret);
+				__func__, __LINE__, ret);
 			break;
 		}
 		len -= out_len;
@@ -225,8 +225,7 @@ static int _mic_virtio_copy(struct mic_vdev *mvdev,
 	 * Update the used ring if a descriptor was available and some data was
 	 * copied in/out and the user asked for a used ring update.
 	 */
-	if (*head != USHRT_MAX && copy->out_len &&
-		copy->update_used) {
+	if (*head != USHRT_MAX && copy->out_len && copy->update_used) {
 		u32 total = 0;
 
 		/* Determine the total data consumed */
@@ -367,7 +366,6 @@ void mic_bh_handler(struct work_struct *work)
 
 static irqreturn_t mic_virtio_intr_handler(int irq, void *data)
 {
-
 	struct mic_vdev *mvdev = data;
 	struct mic_device *mdev = mvdev->mdev;
 
@@ -394,7 +392,7 @@ int mic_virtio_config_change(struct mic_vdev *mvdev,
 	}
 
 	if (copy_from_user(mic_vq_configspace(mvdev->dd),
-				argp, mvdev->dd->config_len)) {
+			   argp, mvdev->dd->config_len)) {
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
 			__func__, __LINE__, -EFAULT);
 		ret = -EFAULT;
@@ -438,8 +436,8 @@ static int mic_copy_dp_entry(struct mic_vdev *mvdev,
 		return -EFAULT;
 	}
 
-	if (mic_aligned_desc_size(&dd) > MIC_MAX_DESC_BLK_SIZE
-		|| dd.num_vq > MIC_MAX_VRINGS) {
+	if (mic_aligned_desc_size(&dd) > MIC_MAX_DESC_BLK_SIZE ||
+	    dd.num_vq > MIC_MAX_VRINGS) {
 		dev_err(mic_dev(mvdev), "%s %d err %d\n",
 			__func__, __LINE__, -EINVAL);
 		return -EINVAL;
@@ -551,7 +549,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 			sizeof(struct _mic_vring_info));
 		vr->va = (void *)
 			__get_free_pages(GFP_KERNEL | __GFP_ZERO,
-			get_order(vr_size));
+					 get_order(vr_size));
 		if (!vr->va) {
 			ret = -ENOMEM;
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
@@ -564,8 +562,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 		vqconfig[i].address = mic_map_single(mdev,
 			vr->va, vr_size);
 		if (mic_map_error(vqconfig[i].address)) {
-			free_pages((unsigned long)vr->va,
-				get_order(vr_size));
+			free_pages((unsigned long)vr->va, get_order(vr_size));
 			ret = -ENOMEM;
 			dev_err(mic_dev(mvdev), "%s %d err %d\n",
 				__func__, __LINE__, ret);
@@ -573,8 +570,7 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 		}
 		vqconfig[i].address = cpu_to_le64(vqconfig[i].address);
 
-		vring_init(&vr->vr, num,
-			vr->va, MIC_VIRTIO_RING_ALIGN);
+		vring_init(&vr->vr, num, vr->va, MIC_VIRTIO_RING_ALIGN);
 		ret = vringh_init_kern(&mvr->vrh,
 			*(u32 *)mic_vq_features(mvdev->dd), num, false,
 			vr->vr.desc, vr->vr.avail, vr->vr.used);
@@ -593,8 +589,8 @@ int mic_virtio_add_device(struct mic_vdev *mvdev,
 			__func__, __LINE__, i, vr->va, vr->info, vr_size);
 	}
 
-	snprintf(irqname, sizeof(irqname),
-		"mic%dvirtio%d", mdev->id, mvdev->virtio_id);
+	snprintf(irqname, sizeof(irqname), "mic%dvirtio%d",
+		 mdev->id, mvdev->virtio_id);
 	mvdev->virtio_db = mic_next_db(mdev);
 	mvdev->virtio_cookie = mic_request_irq(mdev, mic_virtio_intr_handler,
 			irqname, mvdev, mvdev->virtio_db, MIC_INTR_DB);
@@ -628,9 +624,9 @@ err:
 	for (j = 0; j < i; j++) {
 		struct mic_vringh *mvr = &mvdev->mvr[j];
 		mic_unmap_single(mdev, le64_to_cpu(vqconfig[j].address),
-				mvr->vring.len);
+				 mvr->vring.len);
 		free_pages((unsigned long)mvr->vring.va,
-			get_order(mvr->vring.len));
+			   get_order(mvr->vring.len));
 	}
 	mutex_unlock(&mdev->mic_mutex);
 	return ret;
@@ -676,9 +672,9 @@ skip_hot_remove:
 		vringh_kiov_cleanup(&mvr->riov);
 		vringh_kiov_cleanup(&mvr->wiov);
 		mic_unmap_single(mdev, le64_to_cpu(vqconfig[i].address),
-				mvr->vring.len);
+				 mvr->vring.len);
 		free_pages((unsigned long)mvr->vring.va,
-			get_order(mvr->vring.len));
+			   get_order(mvr->vring.len));
 	}
 
 	list_for_each_safe(pos, tmp, &mdev->vdev_list) {
drivers/misc/mic/host/mic_x100.c +13 -16
index a12ae5c..a611c28 100644
@@ -46,8 +46,8 @@ mic_x100_write_spad(struct mic_device *mdev, unsigned int idx, u32 val)
 	dev_dbg(mdev->sdev->parent, "Writing 0x%x to scratch pad index %d\n",
 		val, idx);
 	mic_mmio_write(&mdev->mmio, val,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		MIC_X100_SBOX_SPAD0 + idx * 4);
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       MIC_X100_SBOX_SPAD0 + idx * 4);
 }
 
 /**
@@ -139,7 +139,7 @@ static void mic_x100_send_sbox_intr(struct mic_device *mdev,
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(mw, apicicr_low,
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
 }
 
 /**
@@ -153,7 +153,7 @@ static void mic_x100_send_rdmasr_intr(struct mic_device *mdev,
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(&mdev->mmio, 0,
-		MIC_X100_SBOX_BASE_ADDRESS + rdmasr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + rdmasr_offset);
 }
 
 /**
@@ -212,7 +212,7 @@ done:
  */
 static void mic_x100_hw_intr_init(struct mic_device *mdev)
 {
-	mdev->intr_info = (struct mic_intr_info *) mic_x100_intr_init;
+	mdev->intr_info = (struct mic_intr_info *)mic_x100_intr_init;
 }
 
 /**
@@ -244,7 +244,7 @@ mic_x100_read_msi_to_src_map(struct mic_device *mdev, int idx)
  */
 static void
 mic_x100_program_msi_to_src_map(struct mic_device *mdev,
-			int idx, int offset, bool set)
+				int idx, int offset, bool set)
 {
 	unsigned long reg;
 	struct mic_mw *mw = &mdev->mmio;
@@ -308,12 +308,12 @@ static void mic_x100_send_firmware_intr(struct mic_device *mdev)
 	apicicr_low = (vector | (1 << 13));
 
 	mic_mmio_write(mw, mic_x100_get_apic_id(mdev),
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset + 4);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset + 4);
 
 	/* Ensure that the interrupt is ordered w.r.t. previous stores. */
 	wmb();
 	mic_mmio_write(mw, apicicr_low,
-		MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
+		       MIC_X100_SBOX_BASE_ADDRESS + apic_icr_offset);
 }
 
 /**
@@ -365,8 +365,7 @@ mic_x100_load_command_line(struct mic_device *mdev, const struct firmware *fw)
 	len += snprintf(buf, CMDLINE_SIZE - len,
 		" mem=%dM", boot_mem);
 	if (mdev->cmdline)
-		snprintf(buf + len, CMDLINE_SIZE - len,
-				" %s", mdev->cmdline);
+		snprintf(buf + len, CMDLINE_SIZE - len, " %s", mdev->cmdline);
 	memcpy_toio(cmd_line_va, buf, strlen(buf) + 1);
 	kfree(buf);
 	return 0;
@@ -397,8 +396,7 @@ mic_x100_load_ramdisk(struct mic_device *mdev)
 	 * Typically the bootaddr for card OS is 64M
 	 * so copy over the ramdisk @ 128M.
 	 */
-	memcpy_toio(mdev->aper.va + (mdev->bootaddr << 1),
-		fw->data, fw->size);
+	memcpy_toio(mdev->aper.va + (mdev->bootaddr << 1), fw->data, fw->size);
 	iowrite32(cpu_to_le32(mdev->bootaddr << 1), &bp->hdr.ramdisk_image);
 	iowrite32(cpu_to_le32(fw->size), &bp->hdr.ramdisk_size);
 	release_firmware(fw);
@@ -484,8 +482,7 @@ mic_x100_load_firmware(struct mic_device *mdev, const char *buf)
 	if (mdev->ramdisk)
 		rc = mic_x100_load_ramdisk(mdev);
 error:
-	dev_dbg(mdev->sdev->parent, "%s %d rc %d\n",
-			__func__, __LINE__, rc);
+	dev_dbg(mdev->sdev->parent, "%s %d rc %d\n", __func__, __LINE__, rc);
 done:
 	return rc;
 }
@@ -524,8 +521,8 @@ mic_x100_smpt_set(struct mic_device *mdev, dma_addr_t dma_addr, u8 index)
 	uint32_t smpt_reg_val = BUILD_SMPT(SNOOP_ON,
 			dma_addr >> mdev->smpt->info.page_shift);
 	mic_mmio_write(&mdev->mmio, smpt_reg_val,
-		MIC_X100_SBOX_BASE_ADDRESS +
-		MIC_X100_SBOX_SMPT00 + (4 * index));
+		       MIC_X100_SBOX_BASE_ADDRESS +
+		       MIC_X100_SBOX_SMPT00 + (4 * index));
 }
 
 /**
#2 lore raw

Re: [PATCH RESEND v3 1/7] Intel MIC Host Driver for X100 family.

In reply to Sudeep Dutt

Very minor nits, you can change this in a future add-on patch:

Quoted 1 line

+static DEVICE_ATTR(family, S_IRUGO, mic_show_family, NULL);

This should use DEVICE_ATTR_RO(), so that we don't have to audit the permissions of your DEVICE_ATTR() files.

Quoted 1 line

+static DEVICE_ATTR(stepping, S_IRUGO, mic_show_stepping, NULL);

Same here.

Quoted 5 lines
drivers/misc/mic/host/mic_sysfs.c new file +5 -0
+static struct attribute *mic_default_attrs[] = {
+	&dev_attr_family.attr,
+	&dev_attr_stepping.attr,
+
+	NULL

drivers/misc/mic/host/mic_sysfs.c line 92

Quoted 10 lines
drivers/misc/mic/host/mic_sysfs.c new file +10 -0
+};
+
+static struct attribute_group mic_attr_group = {
+	.attrs = mic_default_attrs,
+};
+
+static const struct attribute_group *__mic_attr_group[] = {
+	&mic_attr_group,
+	NULL
+};

These last two structures can be replaced with:
ATTRIBUTE_GROUPS(mic_default);

drivers/misc/mic/host/mic_sysfs.c line 97

Quoted 4 lines
drivers/misc/mic/host/mic_sysfs.c new file +4 -0
+void mic_sysfs_init(struct mic_device *mdev)
+{
+	mdev->attr_group = __mic_attr_group;
+}

This is "odd", why not just export the data structure and reference it in the other code? The pci core does this, and so do other busses.

Anyway, it's not a big deal, just a bit strange to me.

thanks,

greg k-h

#3 lore raw

Re: [PATCH RESEND v3 1/7] Intel MIC Host Driver for X100 family.

In reply to Sudeep Dutt
Quoted 2 lines · Sudeep Dutt
 drivers/misc/mic/common/mic_device.h          |  37 +++
 drivers/misc/mic/host/mic_device.h            | 109 +++++++++

Two different files, with the same name? You are asking for trouble in the future, getting them confused :)

Please try to pick a unique name, especially when you later do things like:

Review

Quoted 2 lines
(quoted diff) +2 -0
+#include "../common/mic_device.h"
+#include "mic_device.h"

Which just looks odd.

Again, not a big deal, follow-on patch can fix this.

thanks,

greg k-h

#8 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt

Again, very minor fixups for later (I can even do them...)

Quoted 1 line

+static DEVICE_ATTR(state, S_IRUGO|S_IWUSR, mic_show_state, mic_store_state);

DEVICE_ATTR_RW() please.

Same for the other attributes you create in this patch.

thanks,

greg k-h

#4 lore raw

Re: [PATCH RESEND v3 1/7] Intel MIC Host Driver for X100 family.

In reply to Sudeep Dutt
Quoted 2 lines · Sudeep Dutt
 drivers/misc/mic/common/mic_device.h          |  37 +++
 drivers/misc/mic/host/mic_device.h            | 109 +++++++++

Two different files, with the same name? You are asking for trouble in the future, getting them confused :)

Please try to pick a unique name, especially when you later do things like:

Review

Quoted 2 lines
(quoted diff) +2 -0
+#include "../common/mic_device.h"
+#include "mic_device.h"

Which just looks odd.

Again, not a big deal, follow-on patch can fix this.

thanks,

greg k-h

#11 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt
Quoted 5 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +5 -0
+What:		/sys/class/mic/mic(x)/cmdline
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 88

Quoted 10 lines
Documentation/ABI/testing/sysfs-class-mic.txt +10 -0
+		An Intel MIC device runs a Linux OS during its operation. Before
+		booting this card OS, it is possible to pass kernel command line
+		options to configure various features in it, similar to
+		self-bootable machines. When read, this entry provides
+		information about the current kernel command line options set to
+		boot the card OS. This entry can be written to change the
+		existing kernel command line options. Typically, the user would
+		want to read the current command line options, append new ones
+		or modify existing ones and then write the whole kernel command
+		line back to this entry.

Is a PAGE_SIZE value going to be big enough for your command line? I know some embedded systems have horribly long command lines, hopefully this will be enough for you.

thanks,

greg k-h

#14 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 98

Quoted 9 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +9 -0
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.

I don't understand, is the path under the HOST device, or the Client device's disk? Why do you need to change the path on the HOST? What's wrong with the existing firmware path selection we have in the kernel?

thanks,

greg k-h

#26 lore raw
In reply to Joe Perches
Quoted 7 lines · Joe Perches

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

Hi Joe,

We will incorporate your changes in the next version of this patch series and run all future patches via the "--strict" checkpatch option.

Thanks a lot for the patch!
Sudeep Dutt

#9 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Greg Kroah-Hartman
Quoted 8 lines · Greg Kroah-Hartman

Again, very minor fixups for later (I can even do them...)

Quoted 1 line

+static DEVICE_ATTR(state, S_IRUGO|S_IWUSR, mic_show_state, mic_store_state);

DEVICE_ATTR_RW() please.

Same for the other attributes you create in this patch.

Sure, we will incorporate these changes along with your other feedback in patch 1 and post the next revision of this patch series.

Thanks for the review!
Sudeep Dutt

#12 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Greg Kroah-Hartman
Quoted 12 lines · Greg Kroah-Hartman
Quoted 11 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +11 -0
+What:		/sys/class/mic/mic(x)/cmdline
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. Before
+		booting this card OS, it is possible to pass kernel command line
+		options to configure various features in it, similar to
+		self-bootable machines. When read, this entry provides
+		information about the current kernel command line options set to
+		boot the card OS. This entry can be written to change the
Quoted 9 lines

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 88

Quoted 4 lines
Documentation/ABI/testing/sysfs-class-mic.txt +4 -0
+		existing kernel command line options. Typically, the user would
+		want to read the current command line options, append new ones
+		or modify existing ones and then write the whole kernel command
+		line back to this entry.

Is a PAGE_SIZE value going to be big enough for your command line? I know some embedded systems have horribly long command lines, hopefully this will be enough for you.

Yes, PAGE_SIZE is more than sufficient for our command line. Thanks,
Sudeep Dutt

#13 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Greg Kroah-Hartman
Quoted 12 lines · Greg Kroah-Hartman
Quoted 11 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +11 -0
+What:		/sys/class/mic/mic(x)/cmdline
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		An Intel MIC device runs a Linux OS during its operation. Before
+		booting this card OS, it is possible to pass kernel command line
+		options to configure various features in it, similar to
+		self-bootable machines. When read, this entry provides
+		information about the current kernel command line options set to
+		boot the card OS. This entry can be written to change the
Quoted 9 lines

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 88

Quoted 4 lines
Documentation/ABI/testing/sysfs-class-mic.txt +4 -0
+		existing kernel command line options. Typically, the user would
+		want to read the current command line options, append new ones
+		or modify existing ones and then write the whole kernel command
+		line back to this entry.

Is a PAGE_SIZE value going to be big enough for your command line? I know some embedded systems have horribly long command lines, hopefully this will be enough for you.

Yes, PAGE_SIZE is more than sufficient for our command line. Thanks,
Sudeep Dutt

#15 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Greg Kroah-Hartman
Quoted 15 lines · Greg Kroah-Hartman

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 98

Quoted 9 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +9 -0
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.

I don't understand, is the path under the HOST device, or the Client device's disk? Why do you need to change the path on the HOST? What's wrong with the existing firmware path selection we have in the kernel?

The path is on the host. The card does not have a physical persistent disk device. Our customers like the flexibility of changing the card firmware/ramdisk contents and file names for individual MIC cards. This flexibility is not possible with a static set of firmware file names in the kernel for all cards.

Once the firmware/ramdisk path under /lib/firmware/ is set up via sysfs, card boot is initiated via the "state" sysfs entry. The host driver then obtains the contents of the firmware and ramdisk via the standard request_firmware(..) interface, copies the contents to card memory and interrupts the card BIOS to initiate boot.

Thanks,
Sudeep Dutt

#10 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt
Quoted 5 lines · Sudeep Dutt
Quoted 4 lines · Greg Kroah-Hartman

Again, very minor fixups for later (I can even do them...)

Quoted 1 line

+static DEVICE_ATTR(state, S_IRUGO|S_IWUSR, mic_show_state, mic_store_state);

Quoted 7 lines
Quoted 4 lines

DEVICE_ATTR_RW() please.

Same for the other attributes you create in this patch.

Sure, we will incorporate these changes along with your other feedback in patch 1 and post the next revision of this patch series.

There's no need to repost just yet. I can take these as-is after the merge window closes, and you can send follow-on patches for these minor issues, keeping you from having to do a respin, and me trying to remember what parts I reviewed and didn't review.

So give me some time (i.e. wait for after 3.12-rc1 is out) and if I have further problems with the patches, I'll let you know, otherwise you'll get an automated email from my patch collection system saying the patches are applied.

thanks,

greg k-h

#16 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Sudeep Dutt
Quoted 23 lines · Sudeep Dutt
Quoted 15 lines · Greg Kroah-Hartman

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 98

Quoted 9 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +9 -0
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.

I don't understand, is the path under the HOST device, or the Client device's disk? Why do you need to change the path on the HOST? What's wrong with the existing firmware path selection we have in the kernel?

The path is on the host. The card does not have a physical persistent disk device. Our customers like the flexibility of changing the card firmware/ramdisk contents and file names for individual MIC cards. This flexibility is not possible with a static set of firmware file names in the kernel for all cards.

Quoted 5 lines

Once the firmware/ramdisk path under /lib/firmware/ is set up via sysfs, card boot is initiated via the "state" sysfs entry. The host driver then obtains the contents of the firmware and ramdisk via the standard request_firmware(..) interface, copies the contents to card memory and interrupts the card BIOS to initiate boot.

So this is really a "filename" that might contain some directories as well, right? The fact you used "path" confused me, as that doesn't usually imply a filename.

And is the "firmware" just the initramfs image for the kernel to boot?

thanks,

greg k-h

#17 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

Quoted 30 lines · Greg Kroah-Hartman
Quoted 23 lines · Sudeep Dutt
Quoted 15 lines · Greg Kroah-Hartman

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 98

Quoted 9 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +9 -0
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.

I don't understand, is the path under the HOST device, or the Client device's disk? Why do you need to change the path on the HOST? What's wrong with the existing firmware path selection we have in the kernel?

The path is on the host. The card does not have a physical persistent disk device. Our customers like the flexibility of changing the card firmware/ramdisk contents and file names for individual MIC cards. This flexibility is not possible with a static set of firmware file names in the kernel for all cards.

Quoted 5 lines

Once the firmware/ramdisk path under /lib/firmware/ is set up via sysfs, card boot is initiated via the "state" sysfs entry. The host driver then obtains the contents of the firmware and ramdisk via the standard request_firmware(..) interface, copies the contents to card memory and interrupts the card BIOS to initiate boot.

Quoted 4 lines

So this is really a "filename" that might contain some directories as well, right? The fact you used "path" confused me, as that doesn't usually imply a filename.

Yes, it is a filename that might contain some directories. We will fix up the documentation here to read filename in future patches.

Quoted 2 lines

And is the "firmware" just the initramfs image for the kernel to boot?

The firmware is usually a Linux kernel. The ramdisk is usually an initramfs image. We have separate sysfs entries for firmware and ramdisk filenames.

Thanks,
Sudeep Dutt

Quoted 3 lines

thanks,

greg k-h

#18 lore raw

Re: [PATCH RESEND v3 3/7] Intel MIC Host Driver, card OS state management.

In reply to Greg Kroah-Hartman
Quoted 30 lines · Greg Kroah-Hartman
Quoted 23 lines · Sudeep Dutt
Quoted 15 lines · Greg Kroah-Hartman

Documentation/ABI/testing/sysfs-class-mic.txt Contact: Sudeep Dutt <[email protected]> · line 98

Quoted 9 lines · Sudeep Dutt
Documentation/ABI/testing/sysfs-class-mic.txt +9 -0
+What:		/sys/class/mic/mic(x)/firmware
+Date:		August 2013
+KernelVersion:	3.11
+Contact:	Sudeep Dutt <[email protected]>
+Description:
+		When read, this sysfs entry provides the path name under
+		/lib/firmware/ where the firmware image to be booted on the
+		card can be found. The entry can be written to change the
+		firmware image location under /lib/firmware/.

I don't understand, is the path under the HOST device, or the Client device's disk? Why do you need to change the path on the HOST? What's wrong with the existing firmware path selection we have in the kernel?

The path is on the host. The card does not have a physical persistent disk device. Our customers like the flexibility of changing the card firmware/ramdisk contents and file names for individual MIC cards. This flexibility is not possible with a static set of firmware file names in the kernel for all cards.

Quoted 5 lines

Once the firmware/ramdisk path under /lib/firmware/ is set up via sysfs, card boot is initiated via the "state" sysfs entry. The host driver then obtains the contents of the firmware and ramdisk via the standard request_firmware(..) interface, copies the contents to card memory and interrupts the card BIOS to initiate boot.

Quoted 4 lines

So this is really a "filename" that might contain some directories as well, right? The fact you used "path" confused me, as that doesn't usually imply a filename.

Yes, it is a filename that might contain some directories. We will fix up the documentation here to read filename in future patches.

Quoted 2 lines

And is the "firmware" just the initramfs image for the kernel to boot?

The firmware is usually a Linux kernel. The ramdisk is usually an initramfs image. We have separate sysfs entries for firmware and ramdisk filenames.

Thanks,
Sudeep Dutt

Quoted 3 lines

thanks,

greg k-h

#31 lore raw
In reply to Sudeep Dutt
Quoted 9 lines · Sudeep Dutt

ChangeLog:
=========

v2 => v3:
a) Patch 1 data structure cleanups, header file include cleanups, IDA interface reuse and switching to device_create_with_groups(..) as per feedback from Greg Kroah-Hartman.
b) Patch 7 signal documentation, sleep workaround removal and sysfs access API cleanups as per feedback from Michael S. Tsirkin.

Now applied, thanks.

Note, I'm getting some odd warnings about the uapi .h files when doing a build, so a follow-on patch to fix those up would be appreciated, along with the other minor cleanups I discussed with you previously.

thanks,

greg k-h

#27 lore raw
In reply to Joe Perches
Quoted 7 lines · Joe Perches

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

-ENOSIGNEDOFFBY :(

#28 lore raw
Quoted 10 lines · Greg Kroah-Hartman
Quoted 7 lines · Joe Perches

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

-ENOSIGNEDOFFBY :(

These were just suggestions for Sudeep.

Sudeep said he'd incorporate and resend the series.

Sudeep? Were you ever going to do that?

#29 lore raw
In reply to Greg Kroah-Hartman
Quoted 10 lines · Greg Kroah-Hartman
Quoted 7 lines · Joe Perches

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

-ENOSIGNEDOFFBY :(

These were just suggestions for Sudeep.

Sudeep said he'd incorporate and resend the series.

Sudeep? Were you ever going to do that?

#30 lore raw
Quoted 10 lines · Joe Perches
Quoted 9 lines · Greg Kroah-Hartman
Quoted 7 lines · Joe Perches

Whitespace neatening...

Multiline statement argument alignment.
Argument wrapping.
Use kmalloc_array instead of kmalloc.

---

Quoted 8 lines
Quoted 1 line

-ENOSIGNEDOFFBY :(

These were just suggestions for Sudeep.

Sudeep said he'd incorporate and resend the series.

Sudeep? Were you ever going to do that?

We were waiting for Greg to apply the base patches to the char misc next tree which happened today. We will send out the cleanup patch series which addresses all feedback soon.

Thanks,
Sudeep Dutt