Skip to contentSkip to conversation

Qualcomm Cloud AI 100 driver

RFC PATCH New Jeffrey Hugo 2 participants 15 messages May 19, 2020 – May 20, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Qualcomm Cloud AI 100 driver

Introduction:
Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated SoC ASIC for the purpose of efficently running Deep Learning inference workloads in a data center environment.

The offical press release can be found at -
https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference

The offical product website is -
https://www.qualcomm.com/products/datacenter-artificial-intelligence

At the time of the offical press release, numerious technology news sites also covered the product. Doing a search of your favorite site is likely to find their coverage of it.

It is our goal to have the kernel driver for the product fully upstream. The purpose of this RFC is to start that process. We are still doing development (see below), and thus not quite looking to gain acceptance quite yet, but now that we have a working driver we beleive we are at the stage where meaningful conversation with the community can occur.

Design:

+--------------------------------+
|       AI application           |
|       (userspace)              |
+-------------+------------------+
              |
              | Misc dev interface
              |
              |
+-------------+------------------+
|       QAIC driver              |
|       (kernel space)           |
|                                |
+----+------------------+--------+
     |                  |
     |                  |
     |                  |
     |                  |
     |Control path      | Data path
     |(MHI bus)         |
     |                  |
     |                  |
     |                  |
     |                  |
+--------------------------------+
| +--------+      +------------+ |
| | MHI HW |      |DMA Bridge  | |
| +--------+      |(DMA engine)| |
|                 +------------+ |
|                                |
|                                |
|                                |
|  Qualcomm Cloud AI 100 device  |
|                                |
|                                |
+--------------------------------+

A Qualcomm Cloud AI 100 device (QAIC device from here on) is a PCIe hardware accelerator for AI inference workloads. Over the PCIe bus fabric, a QAIC device exposes two interfaces via PCI BARs - a MHI hardware region and a DMA engine hardware region.

Before workloads can be run, a QAIC device needs to be initialized. Similar to other Qualcomm products with incorperate MHI, device firmware needs to be loaded onto the device from the host. This occurs in two stages. First, a secondary bootloader (SBL) needs to be loaded onto the device. This occurs via the BHI protocol, and is handled by the MHI bus. Once the SBL loaded and running, it activates the Sahara protocol. The Sahara protocol is used with a userspace application to load and initialize the remaining firmware. The Sahara protocol and associated userspace application are outside the scope of this series as they have no direct interaction with the QAIC driver.

Once a QAIC device is fully initialized, workloads can be sent to the device and run. This involves a per-device instance misc dev that the QAIC driver exposes to userspace. Running a workload involves two phases - configuring the device, and interacting with the workload.

To configure the device, commands are sent via a MHI channel. This is referred to as the control path. A command is a single message. A message contains one or more transactions. Transactions are operations that the device is requested to perform. Most commands are opaque to the kernel driver, however some are not. For example, if the user application wishes to DMA data to the device, it requires the assistance of the kernel driver to translate the data addresses to an address space that the device can understand. In this instance the transaction for DMAing the data is visible to the kernel driver, and the driver will do the required transformation when encoding the message.

To interact with the workload, the workload is assigned a DMA Bridge Channel (dbc). This is dedicated hardware within the DMA engine. Interacting with the workload consists of sending it input data, and receiving output data. The user application requests appropiate buffers from the kernel driver, prepares the buffers, and directs the kernel driver to queue them to the hardware.

The kernel driver is required to support multiple QAIC devices, and also N users per device.

Status:
This series introduces the driver for QAIC devices, and builds up the minimum functionality for running workloads. Several features which have been omitted or are still planned are indicated in the future work section.

Before exiting the RFC phase, and attempting full acceptance, we wish to complete two features which are currently under development as we expect there to be userspace interface changes as a result.

The first feature is a variable length control message between the kernel driver and the device. This allows us to support the total number of DMA transactions we require for certain platforms, while minimizing memory usage. The interface impact of this would be to allow us to drop the size of the manage buffer between userspace and the kernel driver from the current 16k, much of which is wasted.

The second feature is an optimization and extension of the data path interface. We plan to move the bulk of the data in the qaic_execute structure to the qaic_mem_req structure, which optimized our critical path processing. We also plan to extend the qaic_execute structure to allow for a batch submit of multiple buffers as an optimization and convenience for userspace.

Future work:
For simplicity, we have omitted work related to the following features, and intend to submit in future series:

-debugfs
-trace points
-hwmon (device telemetry)

We are also investigating what it might mean to support dma_bufs. We expect that such support would come as an extension of the interface.

Changelog:

RFC v2:
-Change U64_MAX to PHYS_ADDR_MAX to prevent overflow warning -Fix typo in the module description
-Use a misc dev in place of char dev
-Use KBUILD_MODNAME as driver name
-Drop _irqsave in qaic_execute_ioctl()
-Remove verbose ioctl cmd checks
-Use __leX data types for data sent/received with device -Use __aligned() on packed structures
-Use preferred variable array syntax
-Switch to readl/writel_relaxed, and document
-Clarify ioctl struct padding, and remove some unnecessary padding -Fix misc sparse warnings

Jeffrey Hugo (8):
qaic: Add skeleton driver
qaic: Add and init a basic mhi controller
qaic: Create misc dev
qaic: Implement control path
qaic: Implement data path
qaic: Implement PCI link status error handlers
qaic: Implement MHI error status handler
MAINTAINERS: Add entry for QAIC driver

 MAINTAINERS                        |    7 +
 drivers/misc/Kconfig               |    1 +
 drivers/misc/Makefile              |    1 +
 drivers/misc/qaic/Kconfig          |   20 +
 drivers/misc/qaic/Makefile         |   12 +
 drivers/misc/qaic/mhi_controller.c |  539 +++++++++++++++++++
 drivers/misc/qaic/mhi_controller.h |   20 +
 drivers/misc/qaic/qaic.h           |  113 ++++
 drivers/misc/qaic/qaic_control.c   | 1012 ++++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_data.c      |  979 ++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_drv.c       |  602 +++++++++++++++++++++
 include/uapi/misc/qaic.h           |  245 +++++++++
 12 files changed, 3551 insertions(+)
 create mode 100644 drivers/misc/qaic/Kconfig
 create mode 100644 drivers/misc/qaic/Makefile
 create mode 100644 drivers/misc/qaic/mhi_controller.c
 create mode 100644 drivers/misc/qaic/mhi_controller.h
 create mode 100644 drivers/misc/qaic/qaic.h
 create mode 100644 drivers/misc/qaic/qaic_control.c
 create mode 100644 drivers/misc/qaic/qaic_data.c
 create mode 100644 drivers/misc/qaic/qaic_drv.c
 create mode 100644 include/uapi/misc/qaic.h
Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#1 lore raw

RFC PATCH v2 1/8 qaic: Add skeleton driver

Add the Qualcomm Cloud AI 100 skeleton driver that does nothing but registers a pci driver, probes on the proper device, and supports removal of the module.

---
 drivers/misc/Kconfig         |  1 +
 drivers/misc/Makefile        |  1 +
 drivers/misc/qaic/Kconfig    | 20 ++++++++++++++
 drivers/misc/qaic/Makefile   |  9 +++++++
 drivers/misc/qaic/qaic_drv.c | 62 ++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 93 insertions(+)
 create mode 100644 drivers/misc/qaic/Kconfig
 create mode 100644 drivers/misc/qaic/Makefile
 create mode 100644 drivers/misc/qaic/qaic_drv.c
drivers/misc/Kconfig +1 -0
index 99e1514..ef91d062 100644
@@ -481,4 +481,5 @@ source "drivers/misc/ocxl/Kconfig"
 source "drivers/misc/cardreader/Kconfig"
 source "drivers/misc/habanalabs/Kconfig"
 source "drivers/misc/uacce/Kconfig"
+source "drivers/misc/qaic/Kconfig"
 endmenu
drivers/misc/Makefile +1 -0
index 9abf292..64f0770 100644
@@ -58,3 +58,4 @@ obj-$(CONFIG_PVPANIC)   	+= pvpanic.o
 obj-$(CONFIG_HABANA_AI)		+= habanalabs/
 obj-$(CONFIG_UACCE)		+= uacce/
 obj-$(CONFIG_XILINX_SDFEC)	+= xilinx_sdfec.o
+obj-$(CONFIG_QAIC)		+= qaic/
drivers/misc/qaic/Kconfig new file +20 -0
new file mode 100644
index 0000000..502d176
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Qualcomm Cloud AI 100 accelerators driver
+#
+
+config QAIC
+	tristate "Qualcomm Cloud AI 100 accelerators"
+	depends on PCI && HAS_IOMEM
+	depends on MHI_BUS
+	help
+	  Enables driver for Qualcomm's Cloud AI 100 accelerator PCIe cards
+	  that are designed to accelerate Deep Learning inference workloads.
+
+	  The driver manages the PCIe devices and provides an IOCTL interface
+	  for users to submit workloads to the devices.
+
+	  If unsure, say N.
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called qaic.
drivers/misc/qaic/Makefile new file +9 -0
new file mode 100644
index 0000000..9db6417
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for Qualcomm Cloud AI 100 accelerators driver
+#
+
+obj-$(CONFIG_QAIC)	:= qaic.o
+
+qaic-y := \
+	qaic_drv.o
drivers/misc/qaic/qaic_drv.c new file +62 -0

62 lines not on this page · raw message

Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#2 lore raw

RFC PATCH v2 2/8 qaic: Add and init a basic mhi controller

In reply to Jeffrey Hugo

An mhi controller is the "driver" for a specific mhi device with the mhi bus framework. Add a basic controller for the AIC100 device, and supporting code to init the controller. This will enable the PCIE device, init the mhi hardware, bring it to ready state, and use BHI to load the SBL image.

---
 drivers/misc/qaic/Makefile         |   3 +-
 drivers/misc/qaic/mhi_controller.c | 497 +++++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/mhi_controller.h |  14 ++
 drivers/misc/qaic/qaic.h           |  18 ++
 drivers/misc/qaic/qaic_drv.c       | 108 ++++++++
 5 files changed, 639 insertions(+), 1 deletion(-)
 create mode 100644 drivers/misc/qaic/mhi_controller.c
 create mode 100644 drivers/misc/qaic/mhi_controller.h
 create mode 100644 drivers/misc/qaic/qaic.h
drivers/misc/qaic/Makefile +2 -1
index 9db6417..b5fd819 100644
@@ -6,4 +6,5 @@
 obj-$(CONFIG_QAIC)	:= qaic.o
 
 qaic-y := \
-	qaic_drv.o
+	qaic_drv.o \
+	mhi_controller.o
drivers/misc/qaic/mhi_controller.c new file +497 -0

497 lines not on this page · raw message

drivers/misc/qaic/mhi_controller.h new file +14 -0
new file mode 100644
index 0000000..f5197f1
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (c) 2019-2020, The Linux Foundation. All rights reserved.
+ */
+
+#ifndef MHICONTROLLERQAIC_H_
+#define MHICONTROLLERQAIC_H_
+
+struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev,
+						    void __iomem *mhi_bar,
+						    int mhi_irq);
+
+void qaic_mhi_free_controller(struct mhi_controller *mhi_cntl, bool link_up);
+#endif /* MHICONTROLLERQAIC_H_ */
drivers/misc/qaic/qaic.h new file +18 -0
new file mode 100644
index 0000000..379aa82
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ *
+ * Copyright (c) 2019-2020, The Linux Foundation. All rights reserved.
+ */
+
+#ifndef QAICINTERNAL_H_
+#define QAICINTERNAL_H_
+
+#include <linux/mhi.h>
+#include <linux/pci.h>
+
+struct qaic_device {
+	struct pci_dev		*pdev;
+	int			bars;
+	void __iomem		*bar_0;
+	struct mhi_controller	*mhi_cntl;
+};
+#endif /* QAICINTERNAL_H_ */
drivers/misc/qaic/qaic_drv.c +108 -0

136 lines not on this page · raw message

Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#6 lore raw

RFC PATCH v2 4/8 qaic: Implement control path

In reply to Jeffrey Hugo

Once a qaic device is fully booted, the control path is available for use. The control path allows the host to use a MHI channel to send requests to the device. The canonical usecase for the control path is allowing userspace to configure the device for a workload. This is accomplished via the manage ioctl.

---
 drivers/misc/qaic/Makefile       |    3 +-
 drivers/misc/qaic/qaic.h         |   61 +++
 drivers/misc/qaic/qaic_control.c | 1012 ++++++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_drv.c     |  147 +++++-
 include/uapi/misc/qaic.h         |  117 +++++
 5 files changed, 1328 insertions(+), 12 deletions(-)
 create mode 100644 drivers/misc/qaic/qaic_control.c
 create mode 100644 include/uapi/misc/qaic.h
drivers/misc/qaic/Makefile +2 -1
index b5fd819..7a5513b 100644
@@ -7,4 +7,5 @@ obj-$(CONFIG_QAIC)	:= qaic.o
 
 qaic-y := \
 	qaic_drv.o \
-	mhi_controller.o
+	mhi_controller.o \
+	qaic_control.o
drivers/misc/qaic/qaic.h +61 -0

90 lines not on this page · raw message

drivers/misc/qaic/qaic_control.c new file +1012 -0

1,012 lines not on this page · raw message

drivers/misc/qaic/qaic_drv.c +136 -11

265 lines not on this page · raw message

include/uapi/misc/qaic.h new file +117 -0

117 lines not on this page · raw message

Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#12 lore raw

RFC PATCH v2 6/8 qaic: Implement PCI link status error handlers

In reply to Jeffrey Hugo

Whenever the link goes down, the device loses all state. Therefore, we need to be aware of when the link goes down and comes back up so that our state is kept in sync with the device. Implement the PCI error handlers to be notified of links tate changes.

---
 drivers/misc/qaic/mhi_controller.c | 16 ++++++++++++++++
 drivers/misc/qaic/mhi_controller.h |  4 ++++
 drivers/misc/qaic/qaic_drv.c       | 29 +++++++++++++++++++++++++++++
 3 files changed, 49 insertions(+)
drivers/misc/qaic/mhi_controller.c +16 -0
index a667a2f..9ba74fc 100644
@@ -495,3 +495,19 @@ void qaic_mhi_free_controller(struct mhi_controller *mhi_cntl, bool link_up)
 	kfree(mhi_cntl->irq);
 	kfree(mhi_cntl);
 }
+
+void qaic_mhi_link_down(struct mhi_controller *mhi_cntl)
+{
+	mhi_power_down(mhi_cntl, false);
+}
+
+void qaic_mhi_link_up(struct mhi_controller *mhi_cntl)
+{
+	struct pci_dev *pci_dev = container_of(mhi_cntl->cntrl_dev,
+					       struct pci_dev, dev);
+	int ret;
+
+	ret = mhi_async_power_up(mhi_cntl);
+	if (ret)
+		pci_err(pci_dev, "mhi_async_power_up failed when link came back %d\n", ret);
+}
drivers/misc/qaic/mhi_controller.h +4 -0
index f5197f1..6dca59d 100644
@@ -11,4 +11,8 @@ struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev,
 						    int mhi_irq);
 
 void qaic_mhi_free_controller(struct mhi_controller *mhi_cntl, bool link_up);
+
+void qaic_mhi_link_down(struct mhi_controller *mhi_cntl);
+void qaic_mhi_link_up(struct mhi_controller *mhi_cntl);
+
 #endif /* MHICONTROLLERQAIC_H_ */
drivers/misc/qaic/qaic_drv.c +29 -0
index e548881..ac4315a 100644
@@ -484,6 +484,28 @@ static void qaic_pci_remove(struct pci_dev *pdev)
 	kfree(qdev);
 }
 
+static pci_ers_result_t qaic_pci_error_detected(struct pci_dev *pdev,
+						enum pci_channel_state error)
+{
+	return PCI_ERS_RESULT_NEED_RESET;
+}
+
+static void qaic_pci_reset_prepare(struct pci_dev *pdev)
+{
+	struct qaic_device *qdev = pci_get_drvdata(pdev);
+
+	qaic_dev_reset_clean_local_state(qdev);
+	qaic_mhi_link_down(qdev->mhi_cntl);
+}
+
+static void qaic_pci_reset_done(struct pci_dev *pdev)
+{
+	struct qaic_device *qdev = pci_get_drvdata(pdev);
+
+	qdev->in_reset = false;
+	qaic_mhi_link_up(qdev->mhi_cntl);
+}
+
 static const struct mhi_device_id qaic_mhi_match_table[] = {
 	{ .chan = "QAIC_CONTROL", },
 	{},
@@ -507,11 +529,18 @@ static const struct pci_device_id ids[] = {
 };
 MODULE_DEVICE_TABLE(pci, ids);
 
+static const struct pci_error_handlers qaic_pci_err_handler = {
+	.error_detected = qaic_pci_error_detected,
+	.reset_prepare = qaic_pci_reset_prepare,
+	.reset_done = qaic_pci_reset_done,
+};
+
 static struct pci_driver qaic_pci_driver = {
 	.name = KBUILD_MODNAME,
 	.id_table = ids,
 	.probe = qaic_pci_probe,
 	.remove = qaic_pci_remove,
+	.err_handler = &qaic_pci_err_handler,
 };
 
 static int __init qaic_init(void)
Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#5 lore raw

RFC PATCH v2 3/8 qaic: Create misc dev

In reply to Jeffrey Hugo

Now that we can fully boot the device, we should start making it usable. The primary interface to the device will be via a misc dev. Add the necessary framework to detect when the device is fully booted, and create the misc dev at that point. The device is only usable when it is fully booted. The misc dev does nothing useful yet, but we can easily build on this to provide functionality.

---
 drivers/misc/qaic/qaic.h     |  21 ++++
 drivers/misc/qaic/qaic_drv.c | 233 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 254 insertions(+)
drivers/misc/qaic/qaic.h +21 -0
index 379aa82..7b430cf 100644
@@ -6,13 +6,34 @@
 #ifndef QAICINTERNAL_H_
 #define QAICINTERNAL_H_
 
+#include <linux/kref.h>
 #include <linux/mhi.h>
+#include <linux/miscdevice.h>
+#include <linux/mutex.h>
 #include <linux/pci.h>
+#include <linux/srcu.h>
+
+#define QAIC_MDEV_NAME_MAX	25
+
+struct qaic_user {
+	pid_t			handle;
+	struct qaic_device	*qdev;
+	struct list_head	node;
+	struct srcu_struct	qdev_lock;
+	struct kref		ref_count;
+};
 
 struct qaic_device {
 	struct pci_dev		*pdev;
 	int			bars;
 	void __iomem		*bar_0;
 	struct mhi_controller	*mhi_cntl;
+	struct mhi_device	*cntl_ch;
+	struct miscdevice	mdev;
+	char			mdev_name[QAIC_MDEV_NAME_MAX];
+	bool			in_reset;
+	struct srcu_struct	dev_lock;
+	struct list_head	users;
+	struct mutex		users_mutex;
 };
 #endif /* QAICINTERNAL_H_ */
drivers/misc/qaic/qaic_drv.c +233 -0

290 lines not on this page · raw message

Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#13 lore raw

RFC PATCH v2 7/8 qaic: Implement MHI error status handler

In reply to Jeffrey Hugo

If the MHI bus has detected some kind of error with the device, it will invoke the registered status_cb(). We are intrested in both the fatal error and sys error events.

For a fatal error, the device has essentially gone away, and needs to be reinited from scratch. We lose all state. The MHI bus expects the controller to fully handle this scenario

For a sys error, the device can recover by going through the MHI reset flow, but we lose all state. The MHI bus is going to handle the MHI reset flow, but we need to still clean up our local state.

---
 drivers/misc/qaic/mhi_controller.c | 26 ++++++++++++++++++++++++++
 drivers/misc/qaic/mhi_controller.h |  2 ++
 drivers/misc/qaic/qaic.h           |  2 ++
 drivers/misc/qaic/qaic_drv.c       | 16 +++++++++++++++-
 4 files changed, 45 insertions(+), 1 deletion(-)
drivers/misc/qaic/mhi_controller.c +26 -0
index 9ba74fc..50a078f 100644
@@ -8,6 +8,7 @@
 #include <linux/pci.h>
 
 #include "mhi_controller.h"
+#include "qaic.h"
 
 static unsigned int mhi_timeout = 20000; /* 20 sec default */
 module_param(mhi_timeout, uint, 0600);
@@ -426,6 +427,15 @@ static void mhi_runtime_put(struct mhi_controller *mhi_cntl)
 static void mhi_status_cb(struct mhi_controller *mhi_cntl,
 			  enum mhi_callback reason)
 {
+	struct qaic_device *qdev = (struct qaic_device *)pci_get_drvdata(
+					to_pci_dev(mhi_cntl->cntrl_dev));
+
+	/* this event occurs in atomic context */
+	if (reason == MHI_CB_FATAL_ERROR && !qdev->in_reset)
+		schedule_work(&qdev->reset_mhi_work);
+	/* this event occurs in non-atomic context */
+	if (reason == MHI_CB_SYS_ERROR && !qdev->in_reset)
+		qaic_dev_reset_clean_local_state(qdev);
 }
 
 struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev,
@@ -511,3 +521,19 @@ void qaic_mhi_link_up(struct mhi_controller *mhi_cntl)
 	if (ret)
 		pci_err(pci_dev, "mhi_async_power_up failed when link came back %d\n", ret);
 }
+
+void qaic_mhi_start_reset(struct mhi_controller *mhi_cntl)
+{
+	mhi_power_down(mhi_cntl, true);
+}
+
+void qaic_mhi_reset_done(struct mhi_controller *mhi_cntl)
+{
+	struct pci_dev *pci_dev = container_of(mhi_cntl->cntrl_dev,
+					       struct pci_dev, dev);
+	int ret;
+
+	ret = mhi_async_power_up(mhi_cntl);
+	if (ret)
+		pci_err(pci_dev, "mhi_async_power_up failed after reset %d\n", ret);
+}
drivers/misc/qaic/mhi_controller.h +2 -0
index 6dca59d..82b2b77 100644
@@ -14,5 +14,7 @@ void qaic_mhi_free_controller(struct mhi_controller *mhi_cntl, bool link_up);
 
 void qaic_mhi_link_down(struct mhi_controller *mhi_cntl);
 void qaic_mhi_link_up(struct mhi_controller *mhi_cntl);
+void qaic_mhi_start_reset(struct mhi_controller *mhi_cntl);
+void qaic_mhi_reset_done(struct mhi_controller *mhi_cntl);
 
 #endif /* MHICONTROLLERQAIC_H_ */
drivers/misc/qaic/qaic.h +2 -0
index 769f827..7c4eed6 100644
@@ -70,6 +70,7 @@ struct qaic_device {
 	struct miscdevice	mdev;
 	char			mdev_name[QAIC_MDEV_NAME_MAX];
 	struct dma_bridge_chan	dbc[QAIC_NUM_DBC];
+	struct work_struct	reset_mhi_work;
 	struct workqueue_struct	*cntl_wq;
 	bool			in_reset;
 	struct srcu_struct	dev_lock;
@@ -108,4 +109,5 @@ void wakeup_dbc(struct qaic_device *qdev, u32 dbc_id);
 void release_dbc(struct qaic_device *qdev, u32 dbc_id);
 
 void wake_all_cntl(struct qaic_device *qdev);
+void qaic_dev_reset_clean_local_state(struct qaic_device *qdev);
 #endif /* QAICINTERNAL_H_ */
drivers/misc/qaic/qaic_drv.c +15 -1
index ac4315a..3caafd8 100644
@@ -303,6 +303,18 @@ void qaic_dev_reset_clean_local_state(struct qaic_device *qdev)
 		release_dbc(qdev, i);
 }
 
+static void reset_mhi_work_func(struct work_struct *work)
+{
+	struct qaic_device *qdev;
+
+	qdev = container_of(work, struct qaic_device, reset_mhi_work);
+
+	qaic_dev_reset_clean_local_state(qdev);
+	qaic_mhi_start_reset(qdev->mhi_cntl);
+	qdev->in_reset = false;
+	qaic_mhi_reset_done(qdev->mhi_cntl);
+}
+
 static int qaic_pci_probe(struct pci_dev *pdev,
 			  const struct pci_device_id *id)
 {
@@ -328,6 +340,7 @@ static int qaic_pci_probe(struct pci_dev *pdev,
 	qdev->pdev = pdev;
 	mutex_init(&qdev->cntl_mutex);
 	INIT_LIST_HEAD(&qdev->cntl_xfer_list);
+	INIT_WORK(&qdev->reset_mhi_work, reset_mhi_work_func);
 	init_srcu_struct(&qdev->dev_lock);
 	INIT_LIST_HEAD(&qdev->users);
 	mutex_init(&qdev->users_mutex);
@@ -467,6 +480,7 @@ static void qaic_pci_remove(struct pci_dev *pdev)
 		return;
 
 	qaic_dev_reset_clean_local_state(qdev);
+	cancel_work_sync(&qdev->reset_mhi_work);
 	qaic_mhi_free_controller(qdev->mhi_cntl, link_up);
 	for (i = 0; i < QAIC_NUM_DBC; ++i) {
 		devm_free_irq(&pdev->dev, pci_irq_vector(pdev, i + 1),
@@ -585,4 +599,4 @@ module_exit(qaic_exit);
 MODULE_AUTHOR("Qualcomm Cloud AI 100 Accelerator Kernel Driver Team");
 MODULE_DESCRIPTION("Qualcomm Cloud AI 100 Accelerators Driver");
 MODULE_LICENSE("GPL v2");
-MODULE_VERSION("0.0.0"); /* MAJOR.MINOR.PATCH */
+MODULE_VERSION("3.0.1"); /* MAJOR.MINOR.PATCH */
Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#14 lore raw

RFC PATCH v2 8/8 MAINTAINERS: Add entry for QAIC driver

In reply to Jeffrey Hugo

Add MAINTAINERS entry for the Qualcomm Cloud AI 100 driver.

---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)
MAINTAINERS +7 -0
index 65c7cc9..e3d865b 100644
@@ -13963,6 +13963,13 @@ F:	Documentation/devicetree/bindings/media/qcom,camss.txt
 F:	Documentation/media/v4l-drivers/qcom_camss.rst
 F:	drivers/media/platform/qcom/camss/
 
+QUALCOMM CLOUD AI 100 (QAIC) DRIVER
+M:	Jeffrey Hugo <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	drivers/misc/qaic/
+F:	include/uapi/misc/qaic.h
+
 QUALCOMM CORE POWER REDUCTION (CPR) AVS DRIVER
 M:	Niklas Cassel <[email protected]>
 L:	[email protected]
Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#9 lore raw

RFC PATCH v2 5/8 qaic: Implement data path

In reply to Jeffrey Hugo

Once a user has configured the device via the control path for their workload, the user may use the data path to execute the workload. Using the data path involves several steps.

First, the user must use the memory ioctl to allocate one or more buffers for use by the workload. These buffers hold input data, and results. The memory ioctl can also be used to free buffers once no longer needed.

Next, the user must mmap() the buffers, to gain access to them.

To submit buffers to the device, the user uses the execute ioctl.

Finally, the user may use the wait for execute ioctl to determine when the device has completed its handling of submitted buffers (ie consumed input data, or produced output).

---
 drivers/misc/qaic/Makefile    |   3 +-
 drivers/misc/qaic/qaic.h      |  11 +
 drivers/misc/qaic/qaic_data.c | 979 ++++++++++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_drv.c  | 101 +++--
 include/uapi/misc/qaic.h      | 128 ++++++
 5 files changed, 1186 insertions(+), 36 deletions(-)
 create mode 100644 drivers/misc/qaic/qaic_data.c
drivers/misc/qaic/Makefile +2 -1
index 7a5513b..d17ee0a 100644
@@ -8,4 +8,5 @@ obj-$(CONFIG_QAIC)	:= qaic.o
 qaic-y := \
 	qaic_drv.o \
 	mhi_controller.o \
-	qaic_control.o
+	qaic_control.o \
+	qaic_data.o
drivers/misc/qaic/qaic.h +11 -0
index 641a39b..769f827 100644
@@ -7,6 +7,7 @@
 #define QAICINTERNAL_H_
 
 #include <linux/idr.h>
+#include <linux/interrupt.h>
 #include <linux/kref.h>
 #include <linux/mhi.h>
 #include <linux/miscdevice.h>
@@ -82,6 +83,14 @@ int get_cntl_version(struct qaic_device *qdev, struct qaic_user *usr,
 		     u16 *major, u16 *minor);
 int qaic_manage_ioctl(struct qaic_device *qdev, struct qaic_user *usr,
 		      unsigned long arg);
+int qaic_mem_ioctl(struct qaic_device *qdev, struct qaic_user *usr,
+		   unsigned long arg);
+int qaic_execute_ioctl(struct qaic_device *qdev, struct qaic_user *usr,
+		       unsigned long arg);
+int qaic_wait_exec_ioctl(struct qaic_device *qdev, struct qaic_user *usr,
+			 unsigned long arg);
+int qaic_data_mmap(struct qaic_device *qdev, struct qaic_user *usr,
+		   struct vm_area_struct *vma);
 
 void qaic_mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
 			 struct mhi_result *mhi_result);
@@ -93,7 +102,9 @@ int qaic_control_open(struct qaic_device *qdev);
 void qaic_control_close(struct qaic_device *qdev);
 void qaic_release_usr(struct qaic_device *qdev, struct qaic_user *usr);
 
+irqreturn_t dbc_irq_handler(int irq, void *data);
 int disable_dbc(struct qaic_device *qdev, u32 dbc_id, struct qaic_user *usr);
+void wakeup_dbc(struct qaic_device *qdev, u32 dbc_id);
 void release_dbc(struct qaic_device *qdev, u32 dbc_id);
 
 void wake_all_cntl(struct qaic_device *qdev);
drivers/misc/qaic/qaic_data.c new file +979 -0

979 lines not on this page · raw message

drivers/misc/qaic/qaic_drv.c +66 -35

173 lines not on this page · raw message

include/uapi/misc/qaic.h +128 -0

145 lines not on this page · raw message

Signature · 2 lines
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#7 lore raw

Re: [RFC v2 PATCH 4/8] qaic: Implement control path

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on char-misc/char-misc-testing]
[also build test ERROR on soc/for-next linus/master v5.7-rc6 next-20200518] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: powerpc-allyesconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>, old ones prefixed by <<):

drivers/misc/qaic/qaic_drv.c: In function 'qaic_ioctl':

>> drivers/misc/qaic/qaic_drv.c:150:2: error: case label does not reduce to an integer constant

150 | case QAIC_IOCTL_MANAGE:
| ^~~~
drivers/misc/qaic/qaic_drv.c: At top level:
drivers/misc/qaic/qaic_drv.c:220:6: warning: no previous prototype for 'qaic_dev_reset_clean_local_state' [-Wmissing-prototypes] 220 | void qaic_dev_reset_clean_local_state(struct qaic_device *qdev)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +150 drivers/misc/qaic/qaic_drv.c

127
128 static long qaic_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
129 {
130 struct qaic_user *usr = filp->private_data;
131 struct qaic_device *qdev = usr->qdev;
132 int qdev_rcu_id;
133 int usr_rcu_id;
134 int ret;
135
136 usr_rcu_id = srcu_read_lock(&usr->qdev_lock);
137 if (!qdev) {
138 srcu_read_unlock(&usr->qdev_lock, usr_rcu_id);
139 return -ENODEV;
140 }
141
142 qdev_rcu_id = srcu_read_lock(&qdev->dev_lock);
143 if (qdev->in_reset) {
144 srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id);
145 srcu_read_unlock(&usr->qdev_lock, usr_rcu_id);
146 return -ENODEV;
147 }
148
149 switch (cmd) {

 > 150		case QAIC_IOCTL_MANAGE:

151 ret = qaic_manage_ioctl(qdev, usr, arg);
152 break;
153 default:
154 ret = -ENOTTY;
155 }
156
157 srcu_read_unlock(&qdev->dev_lock, qdev_rcu_id);
158 srcu_read_unlock(&usr->qdev_lock, usr_rcu_id);
159 return ret;
160 }
161

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#10 lore raw

Re: [RFC v2 PATCH 5/8] qaic: Implement data path

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on char-misc/char-misc-testing]
[also build test WARNING on soc/for-next linus/master v5.7-rc6 next-20200518] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: xtensa-allyesconfig (attached as .config)
compiler: xtensa-linux-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=xtensa

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>, old ones prefixed by <<):

In file included from include/linux/kernel.h:11,
from include/linux/list.h:9,
from include/linux/swait.h:5,
from include/linux/completion.h:12,
from drivers/misc/qaic/qaic_data.c:5:
include/linux/scatterlist.h: In function 'sg_set_buf':
arch/xtensa/include/asm/page.h:193:9: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits] 193 | ((pfn) >= ARCH_PFN_OFFSET && ((pfn) - ARCH_PFN_OFFSET) < max_mapnr)
| ^~
include/linux/compiler.h:78:42: note: in definition of macro 'unlikely'
78 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
include/linux/scatterlist.h:143:2: note: in expansion of macro 'BUG_ON'
143 | BUG_ON(!virt_addr_valid(buf));
| ^~~~~~
arch/xtensa/include/asm/page.h:201:32: note: in expansion of macro 'pfn_valid'
201 | #define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT)
| ^~~~~~~~~
include/linux/scatterlist.h:143:10: note: in expansion of macro 'virt_addr_valid'
143 | BUG_ON(!virt_addr_valid(buf));
| ^~~~~~~~~~~~~~~
In file included from ./arch/xtensa/include/generated/asm/bug.h:1,
from include/linux/bug.h:5,
from include/linux/thread_info.h:12,
from include/asm-generic/preempt.h:5,
from ./arch/xtensa/include/generated/asm/preempt.h:1,
from include/linux/preempt.h:78,
from include/linux/spinlock.h:51,
from include/linux/swait.h:7,
from include/linux/completion.h:12,
from drivers/misc/qaic/qaic_data.c:5:
include/linux/dma-mapping.h: In function 'dma_map_resource':
arch/xtensa/include/asm/page.h:193:9: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits] 193 | ((pfn) >= ARCH_PFN_OFFSET && ((pfn) - ARCH_PFN_OFFSET) < max_mapnr)
| ^~
include/asm-generic/bug.h:139:27: note: in definition of macro 'WARN_ON_ONCE'
139 | int __ret_warn_once = !!(condition); | ^~~~~~~~~
include/linux/dma-mapping.h:352:19: note: in expansion of macro 'pfn_valid'
352 | if (WARN_ON_ONCE(pfn_valid(PHYS_PFN(phys_addr))))
| ^~~~~~~~~
In file included from include/linux/shm.h:6,
from include/linux/sched.h:16,
from include/linux/ratelimit.h:6,
from include/linux/dev_printk.h:16,
from include/linux/device.h:15,
from include/linux/dma-mapping.h:7,
from drivers/misc/qaic/qaic_data.c:6:
drivers/misc/qaic/qaic_data.c: In function 'reserve_pages':
arch/xtensa/include/asm/page.h:193:9: warning: comparison of unsigned expression >= 0 is always true [-Wtype-limits] 193 | ((pfn) >= ARCH_PFN_OFFSET && ((pfn) - ARCH_PFN_OFFSET) < max_mapnr)
| ^~

>> drivers/misc/qaic/qaic_data.c:103:8: note: in expansion of macro 'pfn_valid'

103 | if (!pfn_valid(pfn))
| ^~~~~~~~~

vim +/pfn_valid +103 drivers/misc/qaic/qaic_data.c

    94	
    95	static int reserve_pages(unsigned long start_pfn, unsigned long nr_pages,
    96				 bool reserve)
    97	{
    98		unsigned long pfn;
    99		unsigned long end_pfn = start_pfn + nr_pages;
   100		struct page *page;
   101	
   102		for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 > 103			if (!pfn_valid(pfn))

104 return -EINVAL;
105 page = pfn_to_page(pfn);
106 if (reserve)
107 SetPageReserved(page);
108 else
109 ClearPageReserved(page);
110 }
111 return 0;
112 }
113

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#3 lore raw

Re: [RFC v2 PATCH 2/8] qaic: Add and init a basic mhi controller

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on char-misc/char-misc-testing]
[also build test ERROR on soc/for-next linus/master v5.7-rc6 next-20200519] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: arm64-allmodconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>, old ones prefixed by <<):

drivers/misc/qaic/mhi_controller.c: In function 'qaic_mhi_register_controller':

>> drivers/misc/qaic/mhi_controller.c:457:10: error: 'struct mhi_controller' has no member named 'link_status'

457 | mhi_cntl->link_status = mhi_link_status;
| ^~

vim +457 drivers/misc/qaic/mhi_controller.c

430
431 struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev,
432 void __iomem *mhi_bar,
433 int mhi_irq)
434 {
435 struct mhi_controller *mhi_cntl;
436 int ret;
437
438 pci_dbg(pci_dev, "%s\n", __func__);
439
440 mhi_cntl = kzalloc(sizeof(*mhi_cntl), GFP_KERNEL);
441 if (!mhi_cntl)
442 return ERR_PTR(-ENOMEM);
443
444 mhi_cntl->cntrl_dev = &pci_dev->dev;
445
446 /*
447 * Covers the entire possible physical ram region. Remote side is
448 * going to calculate a size of this range, so subtract 1 to prevent
449 * rollover.
450 */
451 mhi_cntl->iova_start = 0;
452 mhi_cntl->iova_stop = PHYS_ADDR_MAX - 1;
453
454 mhi_cntl->status_cb = mhi_status_cb;
455 mhi_cntl->runtime_get = mhi_runtime_get;
456 mhi_cntl->runtime_put = mhi_runtime_put;

 > 457		mhi_cntl->link_status = mhi_link_status;

458 mhi_cntl->regs = mhi_bar;
459 mhi_cntl->nr_irqs = 1;
460 mhi_cntl->irq = kmalloc(sizeof(*mhi_cntl->irq), GFP_KERNEL);
461
462 if (!mhi_cntl->irq)
463 return ERR_PTR(-ENOMEM);
464
465 mhi_cntl->irq[0] = mhi_irq;
466
467 mhi_cntl->fw_image = "qcom/aic100/sbl.bin";
468
469 /* use latest configured timeout */
470 aic100_config.timeout_ms = mhi_timeout;
471 ret = mhi_register_controller(mhi_cntl, &aic100_config);
472 if (ret) {
473 pci_err(pci_dev, "register_mhi_controller failed %d\n", ret);
474 kfree(mhi_cntl->irq);
475 kfree(mhi_cntl);
476 return ERR_PTR(ret);
477 }
478
479 ret = mhi_async_power_up(mhi_cntl);
480 if (ret) {
481 pci_err(pci_dev, "mhi_async_power_up failed %d\n", ret);
482 mhi_unregister_controller(mhi_cntl);
483 kfree(mhi_cntl->irq);
484 kfree(mhi_cntl);
485 return ERR_PTR(ret);
486 }
487
488 return mhi_cntl;
489 }
490

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#8 lore raw

Re: [RFC v2 PATCH 4/8] qaic: Implement control path

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on char-misc/char-misc-testing]
[also build test WARNING on soc/for-next linus/master v5.7-rc6 next-20200519] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: x86_64-allmodconfig (attached as .config)
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-193-gb8fad4bc-dirty
        # save the attached .config to linux build tree
        make C=1 ARCH=x86_64 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag as appropriate

sparse warnings: (new ones prefixed by >>)

>> drivers/misc/qaic/qaic_control.c:207:70: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:209:21: sparse: sparse: restricted __le32 degrades to integer
>> drivers/misc/qaic/qaic_control.c:213:22: sparse: sparse: bad assignment (+=) to restricted __le32
>> drivers/misc/qaic/qaic_control.c:216:30: sparse: sparse: cast from restricted __le32
   drivers/misc/qaic/qaic_control.c:225:67: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:305:21: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:311:22: sparse: sparse: bad assignment (+=) to restricted __le32
   drivers/misc/qaic/qaic_control.c:369:74: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:375:21: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:405:22: sparse: sparse: bad assignment (+=) to restricted __le32
   drivers/misc/qaic/qaic_control.c:432:72: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:434:21: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:439:22: sparse: sparse: bad assignment (+=) to restricted __le32
>> drivers/misc/qaic/qaic_control.c:454:22: sparse: sparse: incorrect type in assignment (different base types) @@    expected restricted __le32 [usertype] len @@    got 2 [usertype] len @@
>> drivers/misc/qaic/qaic_control.c:454:22: sparse:    expected restricted __le32 [usertype] len
>> drivers/misc/qaic/qaic_control.c:454:22: sparse:    got unsigned long
>> drivers/misc/qaic/qaic_control.c:505:24: sparse: sparse: incorrect type in assignment (different base types) @@    expected restricted __le32 [usertype] count @@    got icted __le32 [usertype] count @@
>> drivers/misc/qaic/qaic_control.c:505:24: sparse:    expected restricted __le32 [usertype] count
>> drivers/misc/qaic/qaic_control.c:505:24: sparse:    got unsigned int [usertype] count
>> drivers/misc/qaic/qaic_control.c:525:31: sparse: sparse: cast to restricted __le32
   drivers/misc/qaic/qaic_control.c:609:31: sparse: sparse: cast to restricted __le32
   drivers/misc/qaic/qaic_control.c:611:28: sparse: sparse: cast to restricted __le32
>> drivers/misc/qaic/qaic_control.c:611:28: sparse: sparse: cast from restricted __le16
   drivers/misc/qaic/qaic_control.c:612:28: sparse: sparse: cast to restricted __le32
   drivers/misc/qaic/qaic_control.c:612:28: sparse: sparse: cast from restricted __le16
   drivers/misc/qaic/qaic_control.c:629:21: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:637:40: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:637:56: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:640:34: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:640:34: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:640:34: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:640:34: sparse: sparse: restricted __le32 degrades to integer
   drivers/misc/qaic/qaic_control.c:664:33: sparse: sparse: restricted __le32 degrades to integer
>> drivers/misc/qaic/qaic_control.c:786:31: sparse: sparse: incorrect type in assignment (different base types) @@    expected restricted __le32 [usertype] magic_number @@    got e] magic_number @@
>> drivers/misc/qaic/qaic_control.c:786:31: sparse:    expected restricted __le32 [usertype] magic_number
>> drivers/misc/qaic/qaic_control.c:786:31: sparse:    got int

drivers/misc/qaic/qaic_control.c:788:24: sparse: sparse: cast from restricted __le32 drivers/misc/qaic/qaic_control.c:789:26: sparse: sparse: cast from restricted __le32 drivers/misc/qaic/qaic_control.c:887:21: sparse: sparse: restricted __le32 degrades to integer
drivers/misc/qaic/qaic_control.c:981:31: sparse: sparse: incorrect type in assignment (different base types) @@ expected restricted __le32 [usertype] magic_number @@ got e] magic_number @@
drivers/misc/qaic/qaic_control.c:981:31: sparse: expected restricted __le32 [usertype] magic_number
drivers/misc/qaic/qaic_control.c:981:31: sparse: got int

vim +207 drivers/misc/qaic/qaic_control.c

202
203 static int encode_passthrough(struct qaic_device *qdev, void *trans,
204 struct _msg *msg, u32 *user_len)
205 {
206 struct qaic_manage_trans_passthrough *in_trans = trans;

 > 207		struct _trans_passthrough *out_trans = (void *)msg + msg->hdr.len;
   208	
   209		if (msg->hdr.len + in_trans->hdr.len > sizeof(*msg))
   210			return -ENOSPC;
   211	
   212		memcpy(out_trans, in_trans, in_trans->hdr.len);
 > 213		msg->hdr.len += in_trans->hdr.len;
   214		*user_len += in_trans->hdr.len;
   215		out_trans->hdr.type = cpu_to_le32(TRANS_PASSTHROUGH_TO_DEV);
 > 216		out_trans->hdr.len = cpu_to_le32(out_trans->hdr.len);

217
218 return 0;
219 }
220

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#11 lore raw

Re: [RFC v2 PATCH 5/8] qaic: Implement data path

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test WARNING on char-misc/char-misc-testing]
[also build test WARNING on soc/for-next linus/master v5.7-rc6 next-20200519] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: x86_64-allmodconfig (attached as .config)
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.1-193-gb8fad4bc-dirty
        # save the attached .config to linux build tree
        make C=1 ARCH=x86_64 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'

If you fix the issue, kindly add following tag as appropriate

sparse warnings: (new ones prefixed by >>)

>> drivers/misc/qaic/qaic_data.c:455:26: sparse: sparse: incorrect type in initializer (different base types) @@    expected restricted __le64 [usertype] db_data @@    got restricted __le64 [usertype] db_data @@
>> drivers/misc/qaic/qaic_data.c:455:26: sparse:    expected restricted __le64 [usertype] db_data
>> drivers/misc/qaic/qaic_data.c:455:26: sparse:    got restricted __le32 [usertype]
>> drivers/misc/qaic/qaic_data.c:465:16: sparse: sparse: incorrect type in assignment (different base types) @@    expected unsigned short [usertype] req_id @@    got resunsigned short [usertype] req_id @@
>> drivers/misc/qaic/qaic_data.c:465:16: sparse:    expected unsigned short [usertype] req_id
>> drivers/misc/qaic/qaic_data.c:465:16: sparse:    got restricted __le16 [usertype]
>> drivers/misc/qaic/qaic_data.c:506:37: sparse: sparse: incorrect type in assignment (different base types) @@    expected restricted __le16 [usertype] req_id @@    got tricted __le16 [usertype] req_id @@
>> drivers/misc/qaic/qaic_data.c:506:37: sparse:    expected restricted __le16 [usertype] req_id
>> drivers/misc/qaic/qaic_data.c:506:37: sparse:    got unsigned short [usertype] req_id
>> drivers/misc/qaic/qaic_data.c:556:30: sparse: sparse: incorrect type in assignment (different base types) @@    expected restricted __le32 [usertype] db_data @@    got restricted __le32 [usertype] db_data @@
>> drivers/misc/qaic/qaic_data.c:556:30: sparse:    expected restricted __le32 [usertype] db_data
>> drivers/misc/qaic/qaic_data.c:556:30: sparse:    got restricted __le64 [usertype] db_data
>> drivers/misc/qaic/qaic_data.c:595:20: sparse: sparse: cast to restricted __le32
   drivers/misc/qaic/qaic_data.c:596:20: sparse: sparse: cast to restricted __le32
>> drivers/misc/qaic/qaic_data.c:637:9: sparse: sparse: incorrect type in argument 1 (different base types) @@    expected unsigned int val @@    got restricted __le3unsigned int val @@
>> drivers/misc/qaic/qaic_data.c:637:9: sparse:    expected unsigned int val

drivers/misc/qaic/qaic_data.c:637:9: sparse: got restricted __le32 [usertype]
drivers/misc/qaic/qaic_data.c:789:16: sparse: sparse: cast to restricted __le32 drivers/misc/qaic/qaic_data.c:790:16: sparse: sparse: cast to restricted __le32
drivers/misc/qaic/qaic_data.c:833:17: sparse: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restricted __le3unsigned int val @@
drivers/misc/qaic/qaic_data.c:833:17: sparse: expected unsigned int val
drivers/misc/qaic/qaic_data.c:833:17: sparse: got restricted __le32 [usertype]

vim +455 drivers/misc/qaic/qaic_data.c

448
449 static int encode_execute(struct qaic_device *qdev, struct mem_handle *mem,
450 struct qaic_execute *exec, u16 req_id)
451 {
452 u8 cmd = BULK_XFER;
453 __le64 db_addr = cpu_to_le64(exec->db_addr);
454 u8 db_len;

 > 455		__le64 db_data = cpu_to_le32(exec->db_data);
   456		struct scatterlist *sg;
   457		u64 dev_addr;
   458		int presync_sem;
   459		int i;
   460	
   461		if (!mem->no_xfer)
   462			cmd |= (exec->dir == DMA_TO_DEVICE ? INBOUND_XFER :
   463									OUTBOUND_XFER);
   464	
 > 465		req_id = cpu_to_le16(req_id);
   466	
   467		if (exec->db_len && !IS_ALIGNED(exec->db_addr, exec->db_len / 8))
   468			return -EINVAL;
   469	
   470		presync_sem = exec->sem0.presync + exec->sem1.presync +
   471			      exec->sem2.presync + exec->sem3.presync;
   472		if (presync_sem > 1)
   473			return -EINVAL;
   474	
   475		presync_sem = exec->sem0.presync << 0 | exec->sem1.presync << 1 |
   476			      exec->sem2.presync << 2 | exec->sem3.presync << 3;
   477	
   478		switch (exec->db_len) {
   479		case 32:
   480			db_len = BIT(7);
   481			break;
   482		case 16:
   483			db_len = BIT(7) | 1;
   484			break;
   485		case 8:
   486			db_len = BIT(7) | 2;
   487			break;
   488		case 0:
   489			db_len = 0; /* doorbell is not active for this command */
   490			break;
   491		default:
   492			return -EINVAL; /* should never hit this */
   493		}
   494	
   495		/*
   496		 * When we end up splitting up a single request (ie a mem handle) into
   497		 * multiple DMA requests, we have to manage the sync data carefully.
   498		 * There can only be one presync sem.  That needs to be on every xfer
   499		 * so that the DMA engine doesn't transfer data before the receiver is
   500		 * ready.  We only do the doorbell and postsync sems after the xfer.
   501		 * To guarantee previous xfers for the request are complete, we use a
   502		 * fence.
   503		 */
   504		dev_addr = exec->dev_addr;
   505		for_each_sg(mem->sgt->sgl, sg, mem->nents, i) {
 > 506			mem->reqs[i].req_id = req_id;
   507			mem->reqs[i].cmd = cmd;
   508			mem->reqs[i].src_addr =
   509				cpu_to_le64(exec->dir == DMA_TO_DEVICE ?
   510						sg_dma_address(sg) : dev_addr);
   511			mem->reqs[i].dest_addr =
   512				cpu_to_le64(exec->dir == DMA_TO_DEVICE ?
   513						dev_addr : sg_dma_address(sg));
   514			mem->reqs[i].len = cpu_to_le32(sg_dma_len(sg));
   515			switch (presync_sem) {
   516			case BIT(0):
   517				mem->reqs[i].sem_cmd0 = cpu_to_le32(
   518							ENCODE_SEM(exec->sem0.val,
   519								exec->sem0.index,
   520								exec->sem0.presync,
   521								exec->sem0.cmd,
   522								exec->sem0.flags));
   523				break;
   524			case BIT(1):
   525				mem->reqs[i].sem_cmd1 = cpu_to_le32(
   526							ENCODE_SEM(exec->sem1.val,
   527								exec->sem1.index,
   528								exec->sem1.presync,
   529								exec->sem1.cmd,
   530								exec->sem1.flags));
   531				break;
   532			case BIT(2):
   533				mem->reqs[i].sem_cmd2 = cpu_to_le32(
   534							ENCODE_SEM(exec->sem2.val,
   535								exec->sem2.index,
   536								exec->sem2.presync,
   537								exec->sem2.cmd,
   538								exec->sem2.flags));
   539				break;
   540			case BIT(3):
   541				mem->reqs[i].sem_cmd3 = cpu_to_le32(
   542							ENCODE_SEM(exec->sem3.val,
   543								exec->sem3.index,
   544								exec->sem3.presync,
   545								exec->sem3.cmd,
   546								exec->sem3.flags));
   547				break;
   548			}
   549			dev_addr += sg_dma_len(sg);
   550		}
   551		/* add post transfer stuff to last segment */
   552		i--;
   553		mem->reqs[i].cmd |= GEN_COMPLETION;
   554		mem->reqs[i].db_addr = db_addr;
   555		mem->reqs[i].db_len = db_len;
 > 556		mem->reqs[i].db_data = db_data;
   557		exec->sem0.flags |= (exec->dir == DMA_TO_DEVICE ? SEM_INSYNCFENCE :
   558								SEM_OUTSYNCFENCE);
   559		mem->reqs[i].sem_cmd0 = cpu_to_le32(ENCODE_SEM(exec->sem0.val,
   560							       exec->sem0.index,
   561							       exec->sem0.presync,
   562							       exec->sem0.cmd,
   563							       exec->sem0.flags));
   564		mem->reqs[i].sem_cmd1 = cpu_to_le32(ENCODE_SEM(exec->sem1.val,
   565							       exec->sem1.index,
   566							       exec->sem1.presync,
   567							       exec->sem1.cmd,
   568							       exec->sem1.flags));
   569		mem->reqs[i].sem_cmd2 = cpu_to_le32(ENCODE_SEM(exec->sem2.val,
   570							       exec->sem2.index,
   571							       exec->sem2.presync,
   572							       exec->sem2.cmd,
   573							       exec->sem2.flags));
   574		mem->reqs[i].sem_cmd3 = cpu_to_le32(ENCODE_SEM(exec->sem3.val,
   575							       exec->sem3.index,
   576							       exec->sem3.presync,
   577							       exec->sem3.cmd,
   578							       exec->sem3.flags));
   579	
   580		return 0;
   581	}
   582	
   583	static int commit_execute(struct qaic_device *qdev, struct mem_handle *mem,
   584				  u32 dbc_id)
   585	{
   586		/**
   587		 * Use _relaxed accessors because we don't need barriers.  This is
   588		 * part of our critical path - the faster we can submit work to the
   589		 * device, the faster the device can generate our output. At this point
   590		 * we haven't accessed the device, so we don't need to sync with
   591		 * previous activity, and we just need both values back, but it really
   592		 * doesn't matter which comes back first.
   593		 */
   594		struct dma_bridge_chan *dbc = &qdev->dbc[dbc_id];
 > 595		u32 head = le32_to_cpu(readl_relaxed(dbc->dbc_base + REQHP_OFF));
   596		u32 tail = le32_to_cpu(readl_relaxed(dbc->dbc_base + REQTP_OFF));
   597		u32 avail = head - tail;
   598		struct dbc_req *reqs = mem->reqs;
   599		bool two_copy = tail + mem->nents > dbc->nelem;
   600	
   601		if (head == U32_MAX || tail == U32_MAX)
   602			/* PCI link error */
   603			return -ENODEV;
   604	
   605		if (head <= tail)
   606			avail += dbc->nelem;
   607	
   608		--avail;
   609	
   610		if (avail < mem->nents)
   611			return -EAGAIN;
   612	
   613		if (two_copy) {
   614			avail = dbc->nelem - tail;
   615			avail = min_t(u32, avail, mem->nents);
   616			memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(),
   617			       reqs, sizeof(*reqs) * avail);
   618			reqs += avail;
   619			avail = mem->nents - avail;
   620			if (avail)
   621				memcpy(dbc->req_q_base, reqs, sizeof(*reqs) * avail);
   622		} else {
   623			memcpy(dbc->req_q_base + tail * get_dbc_req_elem_size(),
   624			       reqs, sizeof(*reqs) * mem->nents);
   625		}
   626	
   627		init_completion(&mem->xfer_done);
   628		list_add_tail(&mem->list, &dbc->xfer_list);
   629		tail = (tail + mem->nents) % dbc->nelem;
   630		/**
   631		 * Use _relaxed because this is a critical path and we don't need a
   632		 * barrier.  The data going to the device is in DMA coherent memory,
   633		 * thus the flushing of that data to memory is already handled and
   634		 * we don't need additional wait before we kick the device to process
   635		 * the request we just queued.
   636		 */
 > 637		writel_relaxed(cpu_to_le32(tail), dbc->dbc_base + REQTP_OFF);

638 return 0;
639 }
640

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#4 lore raw

Re: [RFC v2 PATCH 2/8] qaic: Add and init a basic mhi controller

In reply to Jeffrey Hugo
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please…

Hi Jeffrey,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on char-misc/char-misc-testing]
[also build test ERROR on soc/for-next linus/master v5.7-rc6 next-20200519] [cannot apply to linux/master]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jeffrey-Hugo/Qualcomm-Cloud-AI-100-driver/20200519-221821
base:   https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git 57c76221d5af648c8355a55c09b050c5d8d38189
config: x86_64-randconfig-r022-20200519 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project 135b877874fae96b4372c8a3fbfaa8ff44ff86e3)

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> drivers/misc/qaic/mhi_controller.c:457:12: error: no member named 'link_status' in 'struct mhi_controller'

mhi_cntl->link_status = mhi_link_status;
~~~~~~~~ ^
1 error generated.

vim +457 drivers/misc/qaic/mhi_controller.c

430
431 struct mhi_controller *qaic_mhi_register_controller(struct pci_dev *pci_dev,
432 void __iomem *mhi_bar,
433 int mhi_irq)
434 {
435 struct mhi_controller *mhi_cntl;
436 int ret;
437
438 pci_dbg(pci_dev, "%s\n", __func__);
439
440 mhi_cntl = kzalloc(sizeof(*mhi_cntl), GFP_KERNEL);
441 if (!mhi_cntl)
442 return ERR_PTR(-ENOMEM);
443
444 mhi_cntl->cntrl_dev = &pci_dev->dev;
445
446 /*
447 * Covers the entire possible physical ram region. Remote side is
448 * going to calculate a size of this range, so subtract 1 to prevent
449 * rollover.
450 */
451 mhi_cntl->iova_start = 0;
452 mhi_cntl->iova_stop = PHYS_ADDR_MAX - 1;
453
454 mhi_cntl->status_cb = mhi_status_cb;
455 mhi_cntl->runtime_get = mhi_runtime_get;
456 mhi_cntl->runtime_put = mhi_runtime_put;

 > 457		mhi_cntl->link_status = mhi_link_status;

458 mhi_cntl->regs = mhi_bar;
459 mhi_cntl->nr_irqs = 1;
460 mhi_cntl->irq = kmalloc(sizeof(*mhi_cntl->irq), GFP_KERNEL);
461
462 if (!mhi_cntl->irq)
463 return ERR_PTR(-ENOMEM);
464
465 mhi_cntl->irq[0] = mhi_irq;
466
467 mhi_cntl->fw_image = "qcom/aic100/sbl.bin";
468
469 /* use latest configured timeout */
470 aic100_config.timeout_ms = mhi_timeout;
471 ret = mhi_register_controller(mhi_cntl, &aic100_config);
472 if (ret) {
473 pci_err(pci_dev, "register_mhi_controller failed %d\n", ret);
474 kfree(mhi_cntl->irq);
475 kfree(mhi_cntl);
476 return ERR_PTR(ret);
477 }
478
479 ret = mhi_async_power_up(mhi_cntl);
480 if (ret) {
481 pci_err(pci_dev, "mhi_async_power_up failed %d\n", ret);
482 mhi_unregister_controller(mhi_cntl);
483 kfree(mhi_cntl->irq);
484 kfree(mhi_cntl);
485 return ERR_PTR(ret);
486 }
487
488 return mhi_cntl;
489 }
490

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org