Skip to contentSkip to conversation

user space client interface driver

PATCH Superseded Hemant Kumar 7 participants 18 messages Sep 16, 2020 – Oct 1, 2020

Subsystems (3)

v6 of 18 Superseded Latest is v18 →

lore
Share & tools
lore
#1 lore raw

PATCH v6 1/4 bus: mhi: core: Add helper API to return number of free TREs

In reply to Hemant Kumar

Introduce mhi_get_no_free_descriptors() API to return number of TREs available to queue buffer. MHI clients can use this API to know before hand if ring is full without calling queue API.

---
 drivers/bus/mhi/core/main.c | 12 ++++++++++++
 include/linux/mhi.h         |  9 +++++++++
 2 files changed, 21 insertions(+)
drivers/bus/mhi/core/main.c +12 -0
index 2cff5dd..0599e7d 100644
@@ -258,6 +258,18 @@ int mhi_destroy_device(struct device *dev, void *data)
 	return 0;
 }
 
+int mhi_get_no_free_descriptors(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir)
+{
+	struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+	struct mhi_chan *mhi_chan = (dir == DMA_TO_DEVICE) ?
+		mhi_dev->ul_chan : mhi_dev->dl_chan;
+	struct mhi_ring *tre_ring = &mhi_chan->tre_ring;
+
+	return get_nr_avail_ring_elements(mhi_cntrl, tre_ring);
+}
+EXPORT_SYMBOL_GPL(mhi_get_no_free_descriptors);
+
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason)
 {
 	struct mhi_driver *mhi_drv;
include/linux/mhi.h +9 -0
index a35d876..6565528 100644
@@ -600,6 +600,15 @@ void mhi_set_mhi_state(struct mhi_controller *mhi_cntrl,
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason);
 
 /**
+ * mhi_get_no_free_descriptors - Get transfer ring length
+ * Get # of TD available to queue buffers
+ * @mhi_dev: Device associated with the channels
+ * @dir: Direction of the channel
+ */
+int mhi_get_no_free_descriptors(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir);
+
+/**
  * mhi_prepare_for_power_up - Do pre-initialization before power up.
  *                            This is optional, call this before power up if
  *                            the controller does not want bus framework to
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#6 lore raw

PATCH v6 4/4 bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

---
 drivers/bus/mhi/Kconfig  |  13 +
 drivers/bus/mhi/Makefile |   4 +
 drivers/bus/mhi/uci.c    | 657 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 674 insertions(+)
 create mode 100644 drivers/bus/mhi/uci.c
drivers/bus/mhi/Kconfig +13 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
 	 Enable debugfs support for use with the MHI transport. Allows
 	 reading and/or modifying some values within the MHI controller
 	 for debug and test purposes.
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+	 MHI based userspace client interface driver is used for transferring
+	 raw data between host and device using standard file operations from
+	 userspace. Open, read, write, and close operations are supported
+	 by this driver. Please check mhi_uci_match_table for all supported
+	 channels that are exposed to userspace.
+
+	 To compile this driver as a module, choose M here: the module will be
+	 called mhi_uci.
drivers/bus/mhi/Makefile +4 -0
index 19e6443..80feefb 100644
@@ -1,2 +1,6 @@
 # core layer
 obj-y += core/
+
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
drivers/bus/mhi/uci.c new file +657 -0

657 lines not on this page · raw message

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#5 lore raw

PATCH v6 3/4 docs: Add documentation for userspace client interface

In reply to Hemant Kumar

MHI userspace client driver is creating device file node for user application to perform file operations. File operations are handled by MHI core driver. Currently Loopback MHI channel is supported by this driver.

---
 Documentation/mhi/index.rst |  1 +
 Documentation/mhi/uci.rst   | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
 create mode 100644 Documentation/mhi/uci.rst
Documentation/mhi/index.rst +1 -0
index 1d8dec3..c75a371 100644
@@ -9,6 +9,7 @@ MHI
 
    mhi
    topology
+   uci
 
 .. only::  subproject and html
 
Documentation/mhi/uci.rst new file +39 -0
new file mode 100644
index 0000000..5d92939
@@ -0,0 +1,39 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=================================
+Userspace Client Interface (UCI)
+=================================
+
+UCI driver enables userspace clients to communicate to external MHI devices
+like modem and WLAN. It creates standard character device file nodes for user
+space clients to perform open, read, write, poll and close file operations.
+
+Device file node is created with format:-
+
+/dev/mhi_<controller_name>_<mhi_device_name>
+
+controller_name is the name of underlying bus used to transfer data.
+mhi_device_name is the name of the MHI channel being used by MHI client in
+userspace to send or receive data using MHI protocol.
+
+There is a separate character device file node created for each channel specified
+in mhi device id table. MHI channels are statically defined by MHI specification.
+Driver currently supports LOOPBACK channel 0 (Host to device) and 1 (Device to Host).
+
+LOOPBACK Channel
+----------------
+
+Userspace MHI client using LOOPBACK channel opens device file node. As part of
+open operation TREs to transfer ring of LOOPBACK channel 1 gets queued and channel
+doorbell is rung. When userspace MHI client performs write operation on device node,
+data buffer gets queued as a TRE to transfer ring of LOOPBACK channel 0. MHI Core
+driver rings the channel doorbell for MHI device to move data over underlying bus.
+When userspace MHI client driver performs read operation, same data gets looped back
+to MHI host using LOOPBACK channel 1. LOOPBACK channel is used to verify data path
+and data integrity between MHI Host and MHI device.
+
+Other Use Cases
+---------------
+
+Getting MHI device specific diagnostics information to userspace MHI diag client
+using DIAG channel 4 (Host to device) and 5 (Device to Host).
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#3 lore raw

PATCH v6 2/4 bus: mhi: core: Move MHI_MAX_MTU to external header file

In reply to Hemant Kumar

Currently this macro is defined in internal MHI header as a TRE length mask. Moving it to external header allows MHI client drivers to set this upper bound for the transmit buffer size.

---
 drivers/bus/mhi/core/internal.h | 1 -
 include/linux/mhi.h             | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/bus/mhi/core/internal.h +0 -1
index 7989269..4abf0cf 100644
@@ -453,7 +453,6 @@ enum mhi_pm_state {
 #define CMD_EL_PER_RING			128
 #define PRIMARY_CMD_RING		0
 #define MHI_DEV_WAKE_DB			127
-#define MHI_MAX_MTU			0xffff
 #define MHI_RANDOM_U32_NONZERO(bmsk)	(prandom_u32_max(bmsk) + 1)
 
 enum mhi_er_type {
include/linux/mhi.h +3 -0
index 6565528..610f3b0 100644
@@ -16,6 +16,9 @@
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 
+/* MHI client drivers to set this upper bound for tx buffer */
+#define MHI_MAX_MTU 0xffff
+
 #define MHI_MAX_OEM_PK_HASH_SEGMENTS 16
 
 struct mhi_chan;
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#0 lore raw

v6 cover user space client interface driver

V6:
- Moved uci.c to mhi directory.
- Updated Kconfig to add module information.
- Updated Makefile to rename uci object file name as mhi_uci
- Removed kref for open count

V5:
- Removed mhi_uci_drv structure.
- Used idr instead of creating global list of uci devices.
- Used kref instead of local ref counting for uci device and open count.
- Removed unlikely macro.

V4:
- Fix locking to protect proper struct members.
- Updated documentation describing uci client driver use cases.
- Fixed uci ref counting in mhi_uci_open for error case.
- Addressed style related review comments.

V3: Added documentation for MHI UCI driver.

V2: Added mutex lock to prevent multiple readers to access same mhi buffer which can result into use after free.

Hemant Kumar (4):
bus: mhi: core: Add helper API to return number of free TREs
bus: mhi: core: Move MHI_MAX_MTU to external header file
docs: Add documentation for userspace client interface
bus: mhi: Add userspace client interface driver

 Documentation/mhi/index.rst     |   1 +
 Documentation/mhi/uci.rst       |  39 +++
 drivers/bus/mhi/Kconfig         |  13 +
 drivers/bus/mhi/Makefile        |   4 +
 drivers/bus/mhi/core/internal.h |   1 -
 drivers/bus/mhi/core/main.c     |  12 +
 drivers/bus/mhi/uci.c           | 657 ++++++++++++++++++++++++++++++++++++++++
 include/linux/mhi.h             |  12 +
 8 files changed, 738 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/mhi/uci.rst
 create mode 100644 drivers/bus/mhi/uci.c
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
reviewer
#7 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar

drivers/bus/mhi/Kconfig config MHI_BUS_DEBUG · line 22

Quoted 8 lines · Hemant Kumar
drivers/bus/mhi/Kconfig +0 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
 	 Enable debugfs support for use with the MHI transport. Allows
 	 reading and/or modifying some values within the MHI controller
 	 for debug and test purposes.

Hi,
Please indent Kconfig keywords with one tab only, and indent help text with one tab + 2 spaces, as documented in Documentation/process/coding-style.rst:

"""
For all of the Kconfig* configuration files throughout the source tree, the indentation is somewhat different. Lines under a ``config`` definition are indented with one tab, while help text is indented an additional two spaces.
"""

drivers/bus/mhi/Kconfig config MHI_BUS_DEBUG · line 35

Quoted 13 lines
drivers/bus/mhi/Kconfig +13 -0
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+	 MHI based userspace client interface driver is used for transferring
+	 raw data between host and device using standard file operations from
+	 userspace. Open, read, write, and close operations are supported
+	 by this driver. Please check mhi_uci_match_table for all supported
+	 channels that are exposed to userspace.
+
+	 To compile this driver as a module, choose M here: the module will be
+	 called mhi_uci.

thanks.

Signature · 1 line
~Randy
#8 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar
Quoted 19 lines · Hemant Kumar
drivers/bus/mhi/uci.c new file +19 -0
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+	unsigned int minor;
+	struct mhi_device *mhi_dev;
+	const char *chan;
+

drivers/bus/mhi/uci.c line 78

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	/* protects uci_dev struct members */
+	struct mutex lock;
+
+	struct uci_chan ul_chan;
+	struct uci_chan dl_chan;
+	size_t mtu;
+	size_t actual_mtu;
+	bool enabled;
+	struct kref ref_count;
+};

I don't think you actually made the kernel documentation based on these lines.

Or if you did, you ignored the warnings :(

Please test build your patches before sending them out...

thanks,

greg k-h

#10 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar
Quoted 459 lines · Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

---
 drivers/bus/mhi/Kconfig  |  13 +
 drivers/bus/mhi/Makefile |   4 +
 drivers/bus/mhi/uci.c    | 657 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 674 insertions(+)
 create mode 100644 drivers/bus/mhi/uci.c
drivers/bus/mhi/Kconfig +13 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
 	 Enable debugfs support for use with the MHI transport. Allows
 	 reading and/or modifying some values within the MHI controller
 	 for debug and test purposes.
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+	 MHI based userspace client interface driver is used for transferring
+	 raw data between host and device using standard file operations from
+	 userspace. Open, read, write, and close operations are supported
+	 by this driver. Please check mhi_uci_match_table for all supported
+	 channels that are exposed to userspace.
+
+	 To compile this driver as a module, choose M here: the module will be
+	 called mhi_uci.
drivers/bus/mhi/Makefile +4 -0
index 19e6443..80feefb 100644
@@ -1,2 +1,6 @@
 # core layer
 obj-y += core/
+
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
drivers/bus/mhi/uci.c new file +405 -0
new file mode 100644
index 0000000..d6758f2
@@ -0,0 +1,657 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
+
+#include <linux/kernel.h>
+#include <linux/mhi.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#define DEVICE_NAME "mhi"
+#define MHI_UCI_DRIVER_NAME "mhi_uci"
+#define MAX_UCI_MINORS (128)
+
+static DEFINE_IDR(uci_idr);
+static DEFINE_MUTEX(uci_idr_mutex);
+static struct class *uci_dev_class;
+static int uci_dev_major;
+
+/**
+ * struct uci_chan - MHI channel for a uci device
+ * @wq: wait queue for reader/writer
+ * @lock: spin lock
+ * @pending: list of rx buffers userspace is waiting to read
+ * @cur_buf: current buffer userspace is reading
+ * @rx_size: size of the current rx buffer userspace is reading
+ */
+struct uci_chan {
+	wait_queue_head_t wq;
+
+	/* protects pending and cur_buf members in bh context */
+	spinlock_t lock;
+
+	struct list_head pending;
+	struct uci_buf *cur_buf;
+	size_t rx_size;
+};
+
+/**
+ * struct uci_buf - uci buffer
+ * @data: data buffer
+ * @len: length of data buffer
+ * @node: list node of the uci buffer
+ */
+struct uci_buf {
+	void *data;
+	size_t len;
+	struct list_head node;
+};
+
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+	unsigned int minor;
+	struct mhi_device *mhi_dev;
+	const char *chan;
+
+	/* protects uci_dev struct members */
+	struct mutex lock;
+
+	struct uci_chan ul_chan;
+	struct uci_chan dl_chan;
+	size_t mtu;
+	size_t actual_mtu;
+	bool enabled;
+	struct kref ref_count;
+};
+
+static int mhi_queue_inbound(struct uci_dev *udev)
+{
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	size_t mtu = udev->mtu;
+	size_t actual_mtu = udev->actual_mtu;
+	int nr_trbs, i, ret = -EIO;
+	void *buf;
+	struct uci_buf *uci_buf;
+
+	nr_trbs = mhi_get_no_free_descriptors(mhi_dev, DMA_FROM_DEVICE);
+
+	for (i = 0; i < nr_trbs; i++) {
+		buf = kmalloc(mtu, GFP_KERNEL);
+		if (!buf)
+			return -ENOMEM;
+
+		uci_buf = buf + actual_mtu;
+		uci_buf->data = buf;
+
+		dev_dbg(dev, "Allocated buf %d of %d size %ld\n", i, nr_trbs,
+			actual_mtu);
+
+		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, actual_mtu,
+				    MHI_EOT);
+		if (ret) {
+			kfree(buf);
+			dev_err(dev, "Failed to queue buffer %d\n", i);
+			return ret;
+		}
+	}
+
+	return ret;
+}
+
+static void mhi_uci_dev_release(struct kref *ref)
+{
+	struct uci_dev *udev =
+		container_of(ref, struct uci_dev, ref_count);
+
+	mutex_destroy(&udev->lock);
+
+	dev_set_drvdata(&udev->mhi_dev->dev, NULL);
+
+	kfree(udev);
+}
+
+static int mhi_uci_release(struct inode *inode, struct file *file)
+{
+	struct uci_dev *udev = file->private_data;
+	struct uci_buf *itr, *tmp;
+	struct uci_chan *uchan;
+
+	if (kref_read(&udev->ref_count) > 2)
+		goto exit_uci_release;
+
+	if (udev->enabled)
+		mhi_unprepare_from_transfer(udev->mhi_dev);
+
+	/* clean inbound channel */
+	uchan = &udev->dl_chan;
+
+	spin_lock_bh(&uchan->lock);
+	list_for_each_entry_safe(itr, tmp, &uchan->pending, node) {
+		list_del(&itr->node);
+		kfree(itr->data);
+	}
+
+	if (uchan->cur_buf)
+		kfree(uchan->cur_buf->data);
+
+	uchan->cur_buf = NULL;
+	spin_unlock_bh(&uchan->lock);
+
+exit_uci_release:
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+
+	return 0;
+}
+
+static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan;
+	__poll_t mask = 0;
+
+	poll_wait(file, &udev->dl_chan.wq, wait);
+	poll_wait(file, &udev->ul_chan.wq, wait);
+
+	if (!udev->enabled) {
+		mask = EPOLLERR;
+	} else {
+		uchan = &udev->dl_chan;
+		spin_lock_bh(&uchan->lock);
+		if (!list_empty(&uchan->pending) || uchan->cur_buf) {
+			dev_dbg(dev, "Client can read from node\n");
+			mask |= EPOLLIN | EPOLLRDNORM;
+		}
+		spin_unlock_bh(&uchan->lock);
+	}
+
+	if (!udev->enabled) {
+		mask |= EPOLLERR;
+	} else if (mhi_get_no_free_descriptors(mhi_dev, DMA_TO_DEVICE) > 0) {
+		dev_dbg(dev, "Client can write to node\n");
+		mask |= EPOLLOUT | EPOLLWRNORM;
+	}
+
+	dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
+
+	return mask;
+}
+
+static ssize_t mhi_uci_write(struct file *file,
+			     const char __user *buf,
+			     size_t count,
+			     loff_t *offp)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan = &udev->ul_chan;
+	size_t bytes_xfered = 0;
+	int ret, nr_avail = 0;
+
+	if (!buf || !count)
+		return -EINVAL;
+
+	/* confirm channel is active */
+	mutex_lock(&udev->lock);
+	if (!udev->enabled) {
+		ret = -ENODEV;
+		goto err_mtx_unlock;
+	}
+
+	dev_dbg(dev, "%s: to xfer: %lu bytes\n", __func__, count);
+
+	while (count) {
+		size_t xfer_size;
+		void *kbuf;
+		enum mhi_flags flags;
+
+		mutex_unlock(&udev->lock);
+		/* wait for free descriptors */
+		ret = wait_event_interruptible(uchan->wq,
+					       (!udev->enabled) ||
+				(nr_avail = mhi_get_no_free_descriptors(mhi_dev,
+					       DMA_TO_DEVICE)) > 0);
+
+		mutex_lock(&udev->lock);
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Exit signal caught for node\n");
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+
+		xfer_size = min_t(size_t, count, udev->mtu);
+		kbuf = kmalloc(xfer_size, GFP_KERNEL);
+		if (!kbuf) {
+			ret = -ENOMEM;
+			goto err_mtx_unlock;
+		}
+
+		ret = copy_from_user(kbuf, buf, xfer_size);
+		if (ret) {
+			kfree(kbuf);
+			ret = -EFAULT;
+			goto err_mtx_unlock;
+		}
+
+		/* if ring is full after this force EOT */
+		if (nr_avail > 1 && (count - xfer_size))
+			flags = MHI_CHAIN;
+		else
+			flags = MHI_EOT;
+
+		if (udev->enabled)
+			ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf,
+					    xfer_size, flags);
+		else
+			ret = -ENODEV;
+
+		if (ret) {
+			kfree(kbuf);
+			goto err_mtx_unlock;
+		}
+
+		bytes_xfered += xfer_size;
+		count -= xfer_size;
+		buf += xfer_size;
+	}
+
+	mutex_unlock(&udev->lock);
+	dev_dbg(dev, "%s: bytes xferred: %lu\n", __func__, bytes_xfered);
+
+	return bytes_xfered;
+
+err_mtx_unlock:
+	mutex_unlock(&udev->lock);
+
+	return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+			    char __user *buf,
+			    size_t count,
+			    loff_t *ppos)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct uci_chan *uchan = &udev->dl_chan;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_buf *ubuf;
+	char *ptr;
+	size_t to_copy;
+	int ret = 0;
+
+	if (!buf)
+		return -EINVAL;
+
+	mutex_lock(&udev->lock);
+	/* confirm channel is active */
+	if (!udev->enabled) {
+		ret = -ENODEV;
+		goto err_mtx_unlock;
+	}
+
+	spin_lock_bh(&uchan->lock);
+	/* No data available to read, wait */
+	if (!uchan->cur_buf && list_empty(&uchan->pending)) {
+		dev_dbg(dev, "No data available to read waiting\n");
+
+		spin_unlock_bh(&uchan->lock);
+		mutex_unlock(&udev->lock);
+		ret = wait_event_interruptible(uchan->wq,
+					       (!udev->enabled ||
+					      !list_empty(&uchan->pending)));
+
+		mutex_lock(&udev->lock);
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Exit signal caught for node\n");
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+		spin_lock_bh(&uchan->lock);
+	}
+
+	/* new read, get the next descriptor from the list */
+	if (!uchan->cur_buf) {
+		ubuf = list_first_entry_or_null(&uchan->pending,
+						struct uci_buf, node);
+		if (!ubuf) {
+			ret = -EIO;
+			goto err_spin_unlock;
+		}
+
+		list_del(&ubuf->node);
+		uchan->cur_buf = ubuf;
+		uchan->rx_size = ubuf->len;
+		dev_dbg(dev, "Got pkt of size: %zu\n", uchan->rx_size);
+	}
+
+	ubuf = uchan->cur_buf;
+
+	/* Copy the buffer to user space */
+	to_copy = min_t(size_t, count, uchan->rx_size);
+	ptr = ubuf->data + (ubuf->len - uchan->rx_size);
+	spin_unlock_bh(&uchan->lock);
+
+	ret = copy_to_user(buf, ptr, to_copy);
+	if (ret) {
+		ret = -EFAULT;
+		goto err_mtx_unlock;
+	}
+
+	spin_lock_bh(&uchan->lock);
+
+	dev_dbg(dev, "Copied %lu of %lu bytes\n", to_copy, uchan->rx_size);
+	uchan->rx_size -= to_copy;
+
+	/* we finished with this buffer, queue it back to hardware */
+	if (!uchan->rx_size) {
+		uchan->cur_buf = NULL;
+
+		if (udev->enabled)
+			ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE,
+					    ubuf->data,
+					    udev->actual_mtu, MHI_EOT);
+		else
+			ret = -ENODEV;
+
+		if (ret) {
+			dev_err(dev, "Failed to recycle element: %d\n", ret);
+			kfree(ubuf->data);
+			goto err_spin_unlock;
+		}
+	}
+	spin_unlock_bh(&uchan->lock);
+	mutex_unlock(&udev->lock);
+
+	dev_dbg(dev, "%s: Returning %lu bytes\n", __func__, to_copy);
+
+	return to_copy;
+
+err_spin_unlock:
+	spin_unlock_bh(&uchan->lock);
+err_mtx_unlock:
+	mutex_unlock(&udev->lock);
+	return ret;
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+	struct uci_dev *udev = NULL;
+	unsigned int minor = iminor(inode);
+	int ret = -EIO;

drivers/bus/mhi/uci.c line 415

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	struct uci_buf *buf_itr, *tmp;
+	struct uci_chan *dl_chan;
+	struct mhi_device *mhi_dev;
+	struct device *dev;
+
+	mutex_lock(&uci_idr_mutex);
+	udev = idr_find(&uci_idr, minor);
+	mutex_unlock(&uci_idr_mutex);
+	if (!udev) {
+		pr_err("uci dev: minor %d not found\n", minor);

Don't spam the kernel log for things that users can do :(

drivers/bus/mhi/uci.c line 420

Quoted 5 lines
drivers/bus/mhi/uci.c new file +5 -0
+		ret = -ENODEV;
+		goto error_no_dev;
+	}
+
+	kref_get(&udev->ref_count);

Why grab a reference? What does that help with?

drivers/bus/mhi/uci.c line 427

Quoted 7 lines
drivers/bus/mhi/uci.c new file +7 -0
+
+	mhi_dev = udev->mhi_dev;
+	dev = &mhi_dev->dev;
+
+	mutex_lock(&udev->lock);
+	if (kref_read(&udev->ref_count) > 2) {
+		dev_dbg(dev, "Node already opened\n");

Nope, this is NOT doing what you think it is doing.

I told you before, do not try to keep a device node from being opened multiple times, as it will always fail (think about passing file handles around between programs...)

If userspace wants to do this, it will do it. If your driver can't handle that, that's fine, userspace will learn not to do that. But the kernel can not prevent this from happening.

Also note that reading a kref value is a HUGE sign that the code is incorrect, you should never care about the value of a reference. Maybe if it is 0, but that's a special case...

Anyway, given that you ignored my previous review comments here, I'm loath to keep reviewing this patch series. Please get others to review it first before sending it back as I don't like being the only one doing this type of work...

thanks,

greg k-h

#9 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Greg KH

Hi Greg,

Quoted 34 lines · Greg KH
Quoted 19 lines · Hemant Kumar
drivers/bus/mhi/uci.c new file +19 -0
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+	unsigned int minor;
+	struct mhi_device *mhi_dev;
+	const char *chan;
+

drivers/bus/mhi/uci.c line 78

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	/* protects uci_dev struct members */
+	struct mutex lock;
+
+	struct uci_chan ul_chan;
+	struct uci_chan dl_chan;
+	size_t mtu;
+	size_t actual_mtu;
+	bool enabled;
+	struct kref ref_count;
+};

I don't think you actually made the kernel documentation based on these lines.

Quoted 3 lines

Or if you did, you ignored the warnings :(

Please test build your patches before sending them out...

Thanks for pointing out the kernel doc, it was a miss. i will fix it.

Quoted 5 lines

thanks,

greg k-h

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#11 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Greg KH

Hi Greg,

Quoted 1 line · Greg KH

On Wed, Sep 16, 2020 at 12:56:07PM -0700, Hemant Kumar wrote:

...
...

Quoted 10 lines
Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+	struct uci_dev *udev = NULL;
+	unsigned int minor = iminor(inode);
+	int ret = -EIO;
+	struct uci_buf *buf_itr, *tmp;
+	struct uci_chan *dl_chan;
+	struct mhi_device *mhi_dev;
+	struct device *dev;
Quoted 8 lines

drivers/bus/mhi/uci.c line 415

Quoted 6 lines
drivers/bus/mhi/uci.c new file +6 -0
+
+	mutex_lock(&uci_idr_mutex);
+	udev = idr_find(&uci_idr, minor);
+	mutex_unlock(&uci_idr_mutex);
+	if (!udev) {
+		pr_err("uci dev: minor %d not found\n", minor);

Don't spam the kernel log for things that users can do :(

i will change it to a pr_debug, as it helps to debug why open() is failing.

Quoted 8 lines

drivers/bus/mhi/uci.c line 420

Quoted 6 lines
drivers/bus/mhi/uci.c new file +5 -0
+		ret = -ENODEV;
+		goto error_no_dev;
+	}
+
+	kref_get(&udev->ref_count);

Why grab a reference? What does that help with?

In case open() and driver remove() are racing, it helps to prevent use after free of udev in open().

Quoted 11 lines

drivers/bus/mhi/uci.c line 427

Quoted 8 lines
drivers/bus/mhi/uci.c new file +7 -0
+
+	mhi_dev = udev->mhi_dev;
+	dev = &mhi_dev->dev;
+
+	mutex_lock(&udev->lock);
+	if (kref_read(&udev->ref_count) > 2) {
+		dev_dbg(dev, "Node already opened\n");

Nope, this is NOT doing what you think it is doing.

Quoted 7 lines

I told you before, do not try to keep a device node from being opened multiple times, as it will always fail (think about passing file handles around between programs...)

If userspace wants to do this, it will do it. If your driver can't handle that, that's fine, userspace will learn not to do that. But the kernel can not prevent this from happening.

This check is not returning error, instead just setting filp->private_data = udev; and return 0; It is skipping channel prepare and queuing of inbound buffers which was done by first open().

Quoted 4 lines

Also note that reading a kref value is a HUGE sign that the code is incorrect, you should never care about the value of a reference. Maybe if it is 0, but that's a special case...

In previous patch this was done using separate open reference count and after removing that i was relying on udev ref count. MHI channel prepare and buffer allocation for a give channel suppose to happen at open() and only for first open() call.

Quoted 5 lines

Anyway, given that you ignored my previous review comments here, I'm loath to keep reviewing this patch series. Please get others to review it first before sending it back as I don't like being the only one doing this type of work...

Thanks for reviewing my patch series Greg and help making it a better driver!

Quoted 5 lines

thanks,

greg k-h

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#13 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar
Quoted 275 lines · Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

---
  drivers/bus/mhi/Kconfig  |  13 +
  drivers/bus/mhi/Makefile |   4 +
  drivers/bus/mhi/uci.c    | 657 +++++++++++++++++++++++++++++++++++++++++++++++
  3 files changed, 674 insertions(+)

create mode 100644 drivers/bus/mhi/uci.c

drivers/bus/mhi/Kconfig +13 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
  	 Enable debugfs support for use with the MHI transport. Allows
  	 reading and/or modifying some values within the MHI controller
  	 for debug and test purposes.
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+	 MHI based userspace client interface driver is used for transferring
+	 raw data between host and device using standard file operations from
+	 userspace. Open, read, write, and close operations are supported
+	 by this driver. Please check mhi_uci_match_table for all supported
+	 channels that are exposed to userspace.
+
+	 To compile this driver as a module, choose M here: the module will be
+	 called mhi_uci.
drivers/bus/mhi/Makefile +4 -0
index 19e6443..80feefb 100644
@@ -1,2 +1,6 @@
  # core layer
  obj-y += core/
+
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
drivers/bus/mhi/uci.c new file +221 -0
new file mode 100644
index 0000000..d6758f2
@@ -0,0 +1,657 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
+
+#include <linux/kernel.h>
+#include <linux/mhi.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#define DEVICE_NAME "mhi"
+#define MHI_UCI_DRIVER_NAME "mhi_uci"
+#define MAX_UCI_MINORS (128)
+
+static DEFINE_IDR(uci_idr);
+static DEFINE_MUTEX(uci_idr_mutex);
+static struct class *uci_dev_class;
+static int uci_dev_major;
+
+/**
+ * struct uci_chan - MHI channel for a uci device
+ * @wq: wait queue for reader/writer
+ * @lock: spin lock
+ * @pending: list of rx buffers userspace is waiting to read
+ * @cur_buf: current buffer userspace is reading
+ * @rx_size: size of the current rx buffer userspace is reading
+ */
+struct uci_chan {
+	wait_queue_head_t wq;
+
+	/* protects pending and cur_buf members in bh context */
+	spinlock_t lock;
+
+	struct list_head pending;
+	struct uci_buf *cur_buf;
+	size_t rx_size;
+};
+
+/**
+ * struct uci_buf - uci buffer
+ * @data: data buffer
+ * @len: length of data buffer
+ * @node: list node of the uci buffer
+ */
+struct uci_buf {
+	void *data;
+	size_t len;
+	struct list_head node;
+};
+
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+	unsigned int minor;
+	struct mhi_device *mhi_dev;
+	const char *chan;
+
+	/* protects uci_dev struct members */
+	struct mutex lock;
+
+	struct uci_chan ul_chan;
+	struct uci_chan dl_chan;
+	size_t mtu;
+	size_t actual_mtu;
+	bool enabled;
+	struct kref ref_count;
+};
+
+static int mhi_queue_inbound(struct uci_dev *udev)
+{
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	size_t mtu = udev->mtu;
+	size_t actual_mtu = udev->actual_mtu;
+	int nr_trbs, i, ret = -EIO;
+	void *buf;
+	struct uci_buf *uci_buf;
+
+	nr_trbs = mhi_get_no_free_descriptors(mhi_dev, DMA_FROM_DEVICE);
+
+	for (i = 0; i < nr_trbs; i++) {
+		buf = kmalloc(mtu, GFP_KERNEL);
+		if (!buf)
+			return -ENOMEM;
+
+		uci_buf = buf + actual_mtu;
+		uci_buf->data = buf;
+
+		dev_dbg(dev, "Allocated buf %d of %d size %ld\n", i, nr_trbs,
+			actual_mtu);
+
+		ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, actual_mtu,
+				    MHI_EOT);
+		if (ret) {
+			kfree(buf);
+			dev_err(dev, "Failed to queue buffer %d\n", i);
+			return ret;
+		}
+	}
+
+	return ret;
+}
+
+static void mhi_uci_dev_release(struct kref *ref)
+{
+	struct uci_dev *udev =
+		container_of(ref, struct uci_dev, ref_count);
+
+	mutex_destroy(&udev->lock);
+
+	dev_set_drvdata(&udev->mhi_dev->dev, NULL);
+
+	kfree(udev);
+}
+
+static int mhi_uci_release(struct inode *inode, struct file *file)
+{
+	struct uci_dev *udev = file->private_data;
+	struct uci_buf *itr, *tmp;
+	struct uci_chan *uchan;
+
+	if (kref_read(&udev->ref_count) > 2)
+		goto exit_uci_release;
+
+	if (udev->enabled)
+		mhi_unprepare_from_transfer(udev->mhi_dev);
+
+	/* clean inbound channel */
+	uchan = &udev->dl_chan;
+
+	spin_lock_bh(&uchan->lock);
+	list_for_each_entry_safe(itr, tmp, &uchan->pending, node) {
+		list_del(&itr->node);
+		kfree(itr->data);
+	}
+
+	if (uchan->cur_buf)
+		kfree(uchan->cur_buf->data);
+
+	uchan->cur_buf = NULL;
+	spin_unlock_bh(&uchan->lock);
+
+exit_uci_release:
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+
+	return 0;
+}
+
+static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan;
+	__poll_t mask = 0;
+
+	poll_wait(file, &udev->dl_chan.wq, wait);
+	poll_wait(file, &udev->ul_chan.wq, wait);
+
+	if (!udev->enabled) {
+		mask = EPOLLERR;
+	} else {
+		uchan = &udev->dl_chan;
+		spin_lock_bh(&uchan->lock);
+		if (!list_empty(&uchan->pending) || uchan->cur_buf) {
+			dev_dbg(dev, "Client can read from node\n");
+			mask |= EPOLLIN | EPOLLRDNORM;
+		}
+		spin_unlock_bh(&uchan->lock);
+	}
+
+	if (!udev->enabled) {
+		mask |= EPOLLERR;
+	} else if (mhi_get_no_free_descriptors(mhi_dev, DMA_TO_DEVICE) > 0) {
+		dev_dbg(dev, "Client can write to node\n");
+		mask |= EPOLLOUT | EPOLLWRNORM;
+	}
+
+	dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
+
+	return mask;
+}
+
+static ssize_t mhi_uci_write(struct file *file,
+			     const char __user *buf,
+			     size_t count,
+			     loff_t *offp)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_chan *uchan = &udev->ul_chan;
+	size_t bytes_xfered = 0;
+	int ret, nr_avail = 0;
+
+	if (!buf || !count)
+		return -EINVAL;
+
+	/* confirm channel is active */
+	mutex_lock(&udev->lock);
+	if (!udev->enabled) {
+		ret = -ENODEV;
+		goto err_mtx_unlock;
+	}
+
+	dev_dbg(dev, "%s: to xfer: %lu bytes\n", __func__, count);
+
+	while (count) {
+		size_t xfer_size;
+		void *kbuf;

drivers/bus/mhi/uci.c line 231

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+		enum mhi_flags flags;
+
+		mutex_unlock(&udev->lock);
+		/* wait for free descriptors */
+		ret = wait_event_interruptible(uchan->wq,
+					       (!udev->enabled) ||
+				(nr_avail = mhi_get_no_free_descriptors(mhi_dev,
+					       DMA_TO_DEVICE)) > 0);
+
+		mutex_lock(&udev->lock);

It feels like there is a race here. What if we wait because there are no free descriptors, one becomes available, but someone else manages to grab the mutex before we do, and consume it? mhi_queue_buf() will fail later on, and I don't think thats intended.

Quoted 181 lines
drivers/bus/mhi/uci.c new file +181 -0
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Exit signal caught for node\n");
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+
+		xfer_size = min_t(size_t, count, udev->mtu);
+		kbuf = kmalloc(xfer_size, GFP_KERNEL);
+		if (!kbuf) {
+			ret = -ENOMEM;
+			goto err_mtx_unlock;
+		}
+
+		ret = copy_from_user(kbuf, buf, xfer_size);
+		if (ret) {
+			kfree(kbuf);
+			ret = -EFAULT;
+			goto err_mtx_unlock;
+		}
+
+		/* if ring is full after this force EOT */
+		if (nr_avail > 1 && (count - xfer_size))
+			flags = MHI_CHAIN;
+		else
+			flags = MHI_EOT;
+
+		if (udev->enabled)
+			ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf,
+					    xfer_size, flags);
+		else
+			ret = -ENODEV;
+
+		if (ret) {
+			kfree(kbuf);
+			goto err_mtx_unlock;
+		}
+
+		bytes_xfered += xfer_size;
+		count -= xfer_size;
+		buf += xfer_size;
+	}
+
+	mutex_unlock(&udev->lock);
+	dev_dbg(dev, "%s: bytes xferred: %lu\n", __func__, bytes_xfered);
+
+	return bytes_xfered;
+
+err_mtx_unlock:
+	mutex_unlock(&udev->lock);
+
+	return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+			    char __user *buf,
+			    size_t count,
+			    loff_t *ppos)
+{
+	struct uci_dev *udev = file->private_data;
+	struct mhi_device *mhi_dev = udev->mhi_dev;
+	struct uci_chan *uchan = &udev->dl_chan;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_buf *ubuf;
+	char *ptr;
+	size_t to_copy;
+	int ret = 0;
+
+	if (!buf)
+		return -EINVAL;
+
+	mutex_lock(&udev->lock);
+	/* confirm channel is active */
+	if (!udev->enabled) {
+		ret = -ENODEV;
+		goto err_mtx_unlock;
+	}
+
+	spin_lock_bh(&uchan->lock);
+	/* No data available to read, wait */
+	if (!uchan->cur_buf && list_empty(&uchan->pending)) {
+		dev_dbg(dev, "No data available to read waiting\n");
+
+		spin_unlock_bh(&uchan->lock);
+		mutex_unlock(&udev->lock);
+		ret = wait_event_interruptible(uchan->wq,
+					       (!udev->enabled ||
+					      !list_empty(&uchan->pending)));
+
+		mutex_lock(&udev->lock);
+		if (ret == -ERESTARTSYS) {
+			dev_dbg(dev, "Exit signal caught for node\n");
+			goto err_mtx_unlock;
+		}
+
+		if (!udev->enabled) {
+			ret = -ENODEV;
+			goto err_mtx_unlock;
+		}
+		spin_lock_bh(&uchan->lock);
+	}
+
+	/* new read, get the next descriptor from the list */
+	if (!uchan->cur_buf) {
+		ubuf = list_first_entry_or_null(&uchan->pending,
+						struct uci_buf, node);
+		if (!ubuf) {
+			ret = -EIO;
+			goto err_spin_unlock;
+		}
+
+		list_del(&ubuf->node);
+		uchan->cur_buf = ubuf;
+		uchan->rx_size = ubuf->len;
+		dev_dbg(dev, "Got pkt of size: %zu\n", uchan->rx_size);
+	}
+
+	ubuf = uchan->cur_buf;
+
+	/* Copy the buffer to user space */
+	to_copy = min_t(size_t, count, uchan->rx_size);
+	ptr = ubuf->data + (ubuf->len - uchan->rx_size);
+	spin_unlock_bh(&uchan->lock);
+
+	ret = copy_to_user(buf, ptr, to_copy);
+	if (ret) {
+		ret = -EFAULT;
+		goto err_mtx_unlock;
+	}
+
+	spin_lock_bh(&uchan->lock);
+
+	dev_dbg(dev, "Copied %lu of %lu bytes\n", to_copy, uchan->rx_size);
+	uchan->rx_size -= to_copy;
+
+	/* we finished with this buffer, queue it back to hardware */
+	if (!uchan->rx_size) {
+		uchan->cur_buf = NULL;
+
+		if (udev->enabled)
+			ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE,
+					    ubuf->data,
+					    udev->actual_mtu, MHI_EOT);
+		else
+			ret = -ENODEV;
+
+		if (ret) {
+			dev_err(dev, "Failed to recycle element: %d\n", ret);
+			kfree(ubuf->data);
+			goto err_spin_unlock;
+		}
+	}
+	spin_unlock_bh(&uchan->lock);
+	mutex_unlock(&udev->lock);
+
+	dev_dbg(dev, "%s: Returning %lu bytes\n", __func__, to_copy);
+
+	return to_copy;
+
+err_spin_unlock:
+	spin_unlock_bh(&uchan->lock);
+err_mtx_unlock:
+	mutex_unlock(&udev->lock);
+	return ret;
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+	struct uci_dev *udev = NULL;
+	unsigned int minor = iminor(inode);
+	int ret = -EIO;
+	struct uci_buf *buf_itr, *tmp;
+	struct uci_chan *dl_chan;
+	struct mhi_device *mhi_dev;
+	struct device *dev;
+
+	mutex_lock(&uci_idr_mutex);
+	udev = idr_find(&uci_idr, minor);

drivers/bus/mhi/uci.c line 422

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	mutex_unlock(&uci_idr_mutex);
+	if (!udev) {
+		pr_err("uci dev: minor %d not found\n", minor);
+		ret = -ENODEV;
+		goto error_no_dev;
+	}
+
+	kref_get(&udev->ref_count);
+
+	mhi_dev = udev->mhi_dev;

mhi_dev never gets used. You could use it down at the prepare_for_transfer, but you don't. Might as well just remove this from the stack.

Quoted 94 lines
drivers/bus/mhi/uci.c new file +94 -0
+	dev = &mhi_dev->dev;
+
+	mutex_lock(&udev->lock);
+	if (kref_read(&udev->ref_count) > 2) {
+		dev_dbg(dev, "Node already opened\n");
+		goto exit_uci_open;
+	}
+
+	if (!udev->enabled) {
+		dev_info(dev, "Node exists, but is not in active state!\n");
+		goto error_open_chan;
+	}
+
+	dev_dbg(dev, "Starting channel\n");
+	ret = mhi_prepare_for_transfer(udev->mhi_dev);
+	if (ret) {
+		dev_err(dev, "Error starting transfer channels\n");
+		goto error_open_chan;
+	}
+
+	ret = mhi_queue_inbound(udev);
+	if (ret)
+		goto error_rx_queue;
+
+exit_uci_open:
+	filp->private_data = udev;
+	mutex_unlock(&udev->lock);
+
+	return 0;
+
+error_rx_queue:
+	dl_chan = &udev->dl_chan;
+	mhi_unprepare_from_transfer(udev->mhi_dev);
+	list_for_each_entry_safe(buf_itr, tmp, &dl_chan->pending, node) {
+		list_del(&buf_itr->node);
+		kfree(buf_itr->data);
+	}
+error_open_chan:
+	mutex_unlock(&udev->lock);
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+error_no_dev:
+	return ret;
+}
+
+static const struct file_operations mhidev_fops = {
+	.owner = THIS_MODULE,
+	.open = mhi_uci_open,
+	.release = mhi_uci_release,
+	.read = mhi_uci_read,
+	.write = mhi_uci_write,
+	.poll = mhi_uci_poll,
+};
+
+static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
+			   struct mhi_result *mhi_result)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+	struct uci_chan *uchan = &udev->ul_chan;
+	struct device *dev = &mhi_dev->dev;
+
+	dev_dbg(dev, "status: %d xfer_len: %zu\n",
+		mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+	kfree(mhi_result->buf_addr);
+
+	if (!mhi_result->transaction_status)
+		wake_up(&uchan->wq);
+}
+
+static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
+			   struct mhi_result *mhi_result)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+	struct uci_chan *uchan = &udev->dl_chan;
+	struct device *dev = &mhi_dev->dev;
+	struct uci_buf *buf;
+
+	dev_dbg(dev, "status: %d receive_len: %zu\n",
+		mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+	if (mhi_result->transaction_status == -ENOTCONN) {
+		kfree(mhi_result->buf_addr);
+		return;
+	}
+
+	spin_lock_bh(&uchan->lock);
+	buf = mhi_result->buf_addr + udev->actual_mtu;
+	buf->data = mhi_result->buf_addr;
+	buf->len = mhi_result->bytes_xferd;
+	list_add_tail(&buf->node, &uchan->pending);
+	spin_unlock_bh(&uchan->lock);
+
+	wake_up(&uchan->wq);
+}

drivers/bus/mhi/uci.c line 526

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+
+static int mhi_uci_probe(struct mhi_device *mhi_dev,
+			 const struct mhi_device_id *id)
+{
+	struct uci_dev *udev;
+	struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+	struct device *dev;
+	int index, dir;
+
+	udev = kzalloc(sizeof(*udev), GFP_KERNEL);

What if you defer creating the udev until the first open(), and tie the xfer prepare state to the ref count of the udev instance?

Quoted 14 lines
drivers/bus/mhi/uci.c new file +14 -0
+	if (!udev)
+		return -ENOMEM;
+
+	kref_init(&udev->ref_count);
+	mutex_init(&udev->lock);
+	udev->mhi_dev = mhi_dev;
+
+	mutex_lock(&udev->lock);
+
+	mutex_lock(&uci_idr_mutex);
+	index = idr_alloc(&uci_idr, udev, 0, MAX_UCI_MINORS, GFP_KERNEL);
+	mutex_unlock(&uci_idr_mutex);
+	if (index < 0) {
+		mutex_unlock(&udev->lock);

drivers/bus/mhi/uci.c line 550

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+		kfree(udev);
+		return index;
+	}
+
+	udev->minor = index;
+
+	/* create device file node /dev/mhi_<cntrl_dev_name>_<mhi_dev_name> */
+	dev = device_create(uci_dev_class, &mhi_dev->dev,
+			    MKDEV(uci_dev_major, index), udev,
+			    DEVICE_NAME "_%s_%s",

DEVICE_NAME is only used here. Doesn't seem like having a macro is providing much value. I suggest just inlining the value.

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+			    dev_name(mhi_cntrl->cntrl_dev), mhi_dev->name);
+	if (IS_ERR(dev)) {
+		mutex_lock(&uci_idr_mutex);
+		idr_remove(&uci_idr, udev->minor);
+		mutex_unlock(&uci_idr_mutex);
+		mutex_unlock(&udev->lock);
+		kfree(udev);
+		return PTR_ERR(dev);
+	}
+

drivers/bus/mhi/uci.c line 570

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	for (dir = 0; dir < 2; dir++) {
+		struct uci_chan *uchan = (dir) ?
+			&udev->ul_chan : &udev->dl_chan;
+		spin_lock_init(&uchan->lock);
+		init_waitqueue_head(&uchan->wq);
+		INIT_LIST_HEAD(&uchan->pending);
+	}
+
+	udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU);
+	udev->actual_mtu = udev->mtu - sizeof(struct uci_buf);

"mtu" vs "actual_mtu" seems to be very confusing. To start, why not "tx_mtu" and "rx_mtu"? But going from there, why have different mtus for tx and rx? Why not just increase the rx allocations by the required "header" (struct uci_buf) as needed to keep symetry?

Quoted 71 lines
drivers/bus/mhi/uci.c new file +71 -0
+	dev_set_drvdata(&mhi_dev->dev, udev);
+	udev->enabled = true;
+
+	mutex_unlock(&udev->lock);
+
+	dev_info(&mhi_dev->dev, "probed uci dev: minor %d\n", index);
+
+	return 0;
+};
+
+static void mhi_uci_remove(struct mhi_device *mhi_dev)
+{
+	struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+
+	mutex_lock(&udev->lock);
+
+	/* disable the node */
+	udev->enabled = false;
+
+	wake_up(&udev->dl_chan.wq);
+	wake_up(&udev->ul_chan.wq);
+
+	/* delete the node to prevent new opens */
+	device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
+
+	mutex_lock(&uci_idr_mutex);
+	idr_remove(&uci_idr, udev->minor);
+	mutex_unlock(&uci_idr_mutex);
+
+	mutex_unlock(&udev->lock);
+
+	kref_put(&udev->ref_count, mhi_uci_dev_release);
+}
+
+/* .driver_data stores max mtu */
+static const struct mhi_device_id mhi_uci_match_table[] = {
+	{ .chan = "LOOPBACK", .driver_data = 0x1000},
+	{},
+};
+MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
+
+static struct mhi_driver mhi_uci_driver = {
+	.id_table = mhi_uci_match_table,
+	.remove = mhi_uci_remove,
+	.probe = mhi_uci_probe,
+	.ul_xfer_cb = mhi_ul_xfer_cb,
+	.dl_xfer_cb = mhi_dl_xfer_cb,
+	.driver = {
+		.name = MHI_UCI_DRIVER_NAME,
+	},
+};
+
+static int mhi_uci_init(void)
+{
+	int ret;
+
+	ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
+	if (ret < 0)
+		return ret;
+
+	uci_dev_major = ret;
+	uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
+	if (IS_ERR(uci_dev_class)) {
+		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+		return -ENODEV;
+	}
+
+	ret = mhi_driver_register(&mhi_uci_driver);
+	if (ret) {
+		class_destroy(uci_dev_class);
+		unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);

drivers/bus/mhi/uci.c line 651

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+	}
+
+	return ret;
+}
+
+static void __exit mhi_uci_exit(void)
+{
+	mhi_driver_unregister(&mhi_uci_driver);
+	class_destroy(uci_dev_class);
+	unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);

I think you need an idr_destroy() of uci_idr here.

Quoted 7 lines
drivers/bus/mhi/uci.c new file +6 -0
+}
+
+module_init(mhi_uci_init);
+module_exit(mhi_uci_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MHI UCI Driver");
Signature · 3 lines
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
#12 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar
Quoted 33 lines · Hemant Kumar

Hi Greg,

Quoted 1 line · Greg KH

On Wed, Sep 16, 2020 at 12:56:07PM -0700, Hemant Kumar wrote:

...
...

Quoted 10 lines
Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+	struct uci_dev *udev = NULL;
+	unsigned int minor = iminor(inode);
+	int ret = -EIO;
+	struct uci_buf *buf_itr, *tmp;
+	struct uci_chan *dl_chan;
+	struct mhi_device *mhi_dev;
+	struct device *dev;
Quoted 8 lines

drivers/bus/mhi/uci.c line 415

Quoted 6 lines
drivers/bus/mhi/uci.c new file +6 -0
+
+	mutex_lock(&uci_idr_mutex);
+	udev = idr_find(&uci_idr, minor);
+	mutex_unlock(&uci_idr_mutex);
+	if (!udev) {
+		pr_err("uci dev: minor %d not found\n", minor);

Don't spam the kernel log for things that users can do :(

i will change it to a pr_debug, as it helps to debug why open() is failing.

Quoted 8 lines

drivers/bus/mhi/uci.c line 420

Quoted 6 lines
drivers/bus/mhi/uci.c new file +5 -0
+		ret = -ENODEV;
+		goto error_no_dev;
+	}
+
+	kref_get(&udev->ref_count);

Why grab a reference? What does that help with?

Quoted 2 lines

In case open() and driver remove() are racing, it helps to prevent use after free of udev in open().

Are you sure it prevents that? Where is the lock that handles dropping a reference count and incrementing it at the same time?

krefs are not "lock free" entirely, they need to have some type of other control somewhere to prevent foolish things from happening :)

Quoted 14 lines
Quoted 14 lines

drivers/bus/mhi/uci.c line 427

Quoted 7 lines
drivers/bus/mhi/uci.c new file +7 -0
+
+	mhi_dev = udev->mhi_dev;
+	dev = &mhi_dev->dev;
+
+	mutex_lock(&udev->lock);
+	if (kref_read(&udev->ref_count) > 2) {
+		dev_dbg(dev, "Node already opened\n");

Nope, this is NOT doing what you think it is doing.

I told you before, do not try to keep a device node from being opened multiple times, as it will always fail (think about passing file handles around between programs...)

Quoted 6 lines
Quoted 3 lines

If userspace wants to do this, it will do it. If your driver can't handle that, that's fine, userspace will learn not to do that. But the kernel can not prevent this from happening.

This check is not returning error, instead just setting filp->private_data = udev; and return 0; It is skipping channel prepare
and queuing of inbound buffers which was done by first open().

But don't do that by checking a kref value. You should never care about the value of it, that is not how you use it at all, and one reason I hate that function is even present in the kernel...

thanks,

greg k-h

#15 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Hemant Kumar

Hi Hemant,

See comments inline, but globally, the locking and ref counting is more complicated than it should be.

Quoted 276 lines · Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

---
 drivers/bus/mhi/Kconfig  |  13 +
 drivers/bus/mhi/Makefile |   4 +
 drivers/bus/mhi/uci.c    | 657 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 674 insertions(+)
 create mode 100644 drivers/bus/mhi/uci.c
drivers/bus/mhi/Kconfig +13 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
         Enable debugfs support for use with the MHI transport. Allows
         reading and/or modifying some values within the MHI controller
         for debug and test purposes.
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+        MHI based userspace client interface driver is used for transferring
+        raw data between host and device using standard file operations from
+        userspace. Open, read, write, and close operations are supported
+        by this driver. Please check mhi_uci_match_table for all supported
+        channels that are exposed to userspace.
+
+        To compile this driver as a module, choose M here: the module will be
+        called mhi_uci.
drivers/bus/mhi/Makefile +4 -0
index 19e6443..80feefb 100644
@@ -1,2 +1,6 @@
 # core layer
 obj-y += core/
+
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
drivers/bus/mhi/uci.c new file +221 -0
new file mode 100644
index 0000000..d6758f2
@@ -0,0 +1,657 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
+
+#include <linux/kernel.h>
+#include <linux/mhi.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#define DEVICE_NAME "mhi"
+#define MHI_UCI_DRIVER_NAME "mhi_uci"
+#define MAX_UCI_MINORS (128)
+
+static DEFINE_IDR(uci_idr);
+static DEFINE_MUTEX(uci_idr_mutex);
+static struct class *uci_dev_class;
+static int uci_dev_major;
+
+/**
+ * struct uci_chan - MHI channel for a uci device
+ * @wq: wait queue for reader/writer
+ * @lock: spin lock
+ * @pending: list of rx buffers userspace is waiting to read
+ * @cur_buf: current buffer userspace is reading
+ * @rx_size: size of the current rx buffer userspace is reading
+ */
+struct uci_chan {
+       wait_queue_head_t wq;
+
+       /* protects pending and cur_buf members in bh context */
+       spinlock_t lock;
+
+       struct list_head pending;
+       struct uci_buf *cur_buf;
+       size_t rx_size;
+};
+
+/**
+ * struct uci_buf - uci buffer
+ * @data: data buffer
+ * @len: length of data buffer
+ * @node: list node of the uci buffer
+ */
+struct uci_buf {
+       void *data;
+       size_t len;
+       struct list_head node;
+};
+
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+       unsigned int minor;
+       struct mhi_device *mhi_dev;
+       const char *chan;
+
+       /* protects uci_dev struct members */
+       struct mutex lock;
+
+       struct uci_chan ul_chan;
+       struct uci_chan dl_chan;
+       size_t mtu;
+       size_t actual_mtu;
+       bool enabled;
+       struct kref ref_count;
+};
+
+static int mhi_queue_inbound(struct uci_dev *udev)
+{
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       size_t mtu = udev->mtu;
+       size_t actual_mtu = udev->actual_mtu;
+       int nr_trbs, i, ret = -EIO;
+       void *buf;
+       struct uci_buf *uci_buf;
+
+       nr_trbs = mhi_get_no_free_descriptors(mhi_dev, DMA_FROM_DEVICE);
+
+       for (i = 0; i < nr_trbs; i++) {
+               buf = kmalloc(mtu, GFP_KERNEL);
+               if (!buf)
+                       return -ENOMEM;
+
+               uci_buf = buf + actual_mtu;
+               uci_buf->data = buf;
+
+               dev_dbg(dev, "Allocated buf %d of %d size %ld\n", i, nr_trbs,
+                       actual_mtu);
+
+               ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, actual_mtu,
+                                   MHI_EOT);
+               if (ret) {
+                       kfree(buf);
+                       dev_err(dev, "Failed to queue buffer %d\n", i);
+                       return ret;
+               }
+       }
+
+       return ret;
+}
+
+static void mhi_uci_dev_release(struct kref *ref)
+{
+       struct uci_dev *udev =
+               container_of(ref, struct uci_dev, ref_count);
+
+       mutex_destroy(&udev->lock);
+
+       dev_set_drvdata(&udev->mhi_dev->dev, NULL);
+
+       kfree(udev);
+}
+
+static int mhi_uci_release(struct inode *inode, struct file *file)
+{
+       struct uci_dev *udev = file->private_data;
+       struct uci_buf *itr, *tmp;
+       struct uci_chan *uchan;
+
+       if (kref_read(&udev->ref_count) > 2)
+               goto exit_uci_release;
+
+       if (udev->enabled)
+               mhi_unprepare_from_transfer(udev->mhi_dev);
+
+       /* clean inbound channel */
+       uchan = &udev->dl_chan;
+
+       spin_lock_bh(&uchan->lock);
+       list_for_each_entry_safe(itr, tmp, &uchan->pending, node) {
+               list_del(&itr->node);
+               kfree(itr->data);
+       }
+
+       if (uchan->cur_buf)
+               kfree(uchan->cur_buf->data);
+
+       uchan->cur_buf = NULL;
+       spin_unlock_bh(&uchan->lock);
+
+exit_uci_release:
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+
+       return 0;
+}
+
+static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_chan *uchan;
+       __poll_t mask = 0;
+
+       poll_wait(file, &udev->dl_chan.wq, wait);
+       poll_wait(file, &udev->ul_chan.wq, wait);
+
+       if (!udev->enabled) {
+               mask = EPOLLERR;
+       } else {
+               uchan = &udev->dl_chan;
+               spin_lock_bh(&uchan->lock);
+               if (!list_empty(&uchan->pending) || uchan->cur_buf) {
+                       dev_dbg(dev, "Client can read from node\n");
+                       mask |= EPOLLIN | EPOLLRDNORM;
+               }
+               spin_unlock_bh(&uchan->lock);
+       }
+
+       if (!udev->enabled) {
+               mask |= EPOLLERR;
+       } else if (mhi_get_no_free_descriptors(mhi_dev, DMA_TO_DEVICE) > 0) {
+               dev_dbg(dev, "Client can write to node\n");
+               mask |= EPOLLOUT | EPOLLWRNORM;
+       }
+
+       dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
+
+       return mask;
+}
+
+static ssize_t mhi_uci_write(struct file *file,
+                            const char __user *buf,
+                            size_t count,
+                            loff_t *offp)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_chan *uchan = &udev->ul_chan;
+       size_t bytes_xfered = 0;
+       int ret, nr_avail = 0;
+
+       if (!buf || !count)
+               return -EINVAL;
+
+       /* confirm channel is active */
+       mutex_lock(&udev->lock);
+       if (!udev->enabled) {
+               ret = -ENODEV;
+               goto err_mtx_unlock;
+       }
+
+       dev_dbg(dev, "%s: to xfer: %lu bytes\n", __func__, count);
+
+       while (count) {
+               size_t xfer_size;
+               void *kbuf;

drivers/bus/mhi/uci.c line 231

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+               enum mhi_flags flags;
+
+               mutex_unlock(&udev->lock);
+               /* wait for free descriptors */
+               ret = wait_event_interruptible(uchan->wq,
+                                              (!udev->enabled) ||
+                               (nr_avail = mhi_get_no_free_descriptors(mhi_dev,
+                                              DMA_TO_DEVICE)) > 0);
+
+               mutex_lock(&udev->lock);

All this locking unlocking is odd:
- why do you need locking for testing wait_event return code?
- why do you need this udev->enabled?
- The MHI core should be thread safe + mhi_queue_buf should simply fail if removing is ongoing.

Quoted 270 lines
drivers/bus/mhi/uci.c new file +270 -0
+               if (ret == -ERESTARTSYS) {
+                       dev_dbg(dev, "Exit signal caught for node\n");
+                       goto err_mtx_unlock;
+               }
+
+               if (!udev->enabled) {
+                       ret = -ENODEV;
+                       goto err_mtx_unlock;
+               }
+
+               xfer_size = min_t(size_t, count, udev->mtu);
+               kbuf = kmalloc(xfer_size, GFP_KERNEL);
+               if (!kbuf) {
+                       ret = -ENOMEM;
+                       goto err_mtx_unlock;
+               }
+
+               ret = copy_from_user(kbuf, buf, xfer_size);
+               if (ret) {
+                       kfree(kbuf);
+                       ret = -EFAULT;
+                       goto err_mtx_unlock;
+               }
+
+               /* if ring is full after this force EOT */
+               if (nr_avail > 1 && (count - xfer_size))
+                       flags = MHI_CHAIN;
+               else
+                       flags = MHI_EOT;
+
+               if (udev->enabled)
+                       ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf,
+                                           xfer_size, flags);
+               else
+                       ret = -ENODEV;
+
+               if (ret) {
+                       kfree(kbuf);
+                       goto err_mtx_unlock;
+               }
+
+               bytes_xfered += xfer_size;
+               count -= xfer_size;
+               buf += xfer_size;
+       }
+
+       mutex_unlock(&udev->lock);
+       dev_dbg(dev, "%s: bytes xferred: %lu\n", __func__, bytes_xfered);
+
+       return bytes_xfered;
+
+err_mtx_unlock:
+       mutex_unlock(&udev->lock);
+
+       return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+                           char __user *buf,
+                           size_t count,
+                           loff_t *ppos)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct uci_chan *uchan = &udev->dl_chan;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_buf *ubuf;
+       char *ptr;
+       size_t to_copy;
+       int ret = 0;
+
+       if (!buf)
+               return -EINVAL;
+
+       mutex_lock(&udev->lock);
+       /* confirm channel is active */
+       if (!udev->enabled) {
+               ret = -ENODEV;
+               goto err_mtx_unlock;
+       }
+
+       spin_lock_bh(&uchan->lock);
+       /* No data available to read, wait */
+       if (!uchan->cur_buf && list_empty(&uchan->pending)) {
+               dev_dbg(dev, "No data available to read waiting\n");
+
+               spin_unlock_bh(&uchan->lock);
+               mutex_unlock(&udev->lock);
+               ret = wait_event_interruptible(uchan->wq,
+                                              (!udev->enabled ||
+                                             !list_empty(&uchan->pending)));
+
+               mutex_lock(&udev->lock);
+               if (ret == -ERESTARTSYS) {
+                       dev_dbg(dev, "Exit signal caught for node\n");
+                       goto err_mtx_unlock;
+               }
+
+               if (!udev->enabled) {
+                       ret = -ENODEV;
+                       goto err_mtx_unlock;
+               }
+               spin_lock_bh(&uchan->lock);
+       }
+
+       /* new read, get the next descriptor from the list */
+       if (!uchan->cur_buf) {
+               ubuf = list_first_entry_or_null(&uchan->pending,
+                                               struct uci_buf, node);
+               if (!ubuf) {
+                       ret = -EIO;
+                       goto err_spin_unlock;
+               }
+
+               list_del(&ubuf->node);
+               uchan->cur_buf = ubuf;
+               uchan->rx_size = ubuf->len;
+               dev_dbg(dev, "Got pkt of size: %zu\n", uchan->rx_size);
+       }
+
+       ubuf = uchan->cur_buf;
+
+       /* Copy the buffer to user space */
+       to_copy = min_t(size_t, count, uchan->rx_size);
+       ptr = ubuf->data + (ubuf->len - uchan->rx_size);
+       spin_unlock_bh(&uchan->lock);
+
+       ret = copy_to_user(buf, ptr, to_copy);
+       if (ret) {
+               ret = -EFAULT;
+               goto err_mtx_unlock;
+       }
+
+       spin_lock_bh(&uchan->lock);
+
+       dev_dbg(dev, "Copied %lu of %lu bytes\n", to_copy, uchan->rx_size);
+       uchan->rx_size -= to_copy;
+
+       /* we finished with this buffer, queue it back to hardware */
+       if (!uchan->rx_size) {
+               uchan->cur_buf = NULL;
+
+               if (udev->enabled)
+                       ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE,
+                                           ubuf->data,
+                                           udev->actual_mtu, MHI_EOT);
+               else
+                       ret = -ENODEV;
+
+               if (ret) {
+                       dev_err(dev, "Failed to recycle element: %d\n", ret);
+                       kfree(ubuf->data);
+                       goto err_spin_unlock;
+               }
+       }
+       spin_unlock_bh(&uchan->lock);
+       mutex_unlock(&udev->lock);
+
+       dev_dbg(dev, "%s: Returning %lu bytes\n", __func__, to_copy);
+
+       return to_copy;
+
+err_spin_unlock:
+       spin_unlock_bh(&uchan->lock);
+err_mtx_unlock:
+       mutex_unlock(&udev->lock);
+       return ret;
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+       struct uci_dev *udev = NULL;
+       unsigned int minor = iminor(inode);
+       int ret = -EIO;
+       struct uci_buf *buf_itr, *tmp;
+       struct uci_chan *dl_chan;
+       struct mhi_device *mhi_dev;
+       struct device *dev;
+
+       mutex_lock(&uci_idr_mutex);
+       udev = idr_find(&uci_idr, minor);
+       mutex_unlock(&uci_idr_mutex);
+       if (!udev) {
+               pr_err("uci dev: minor %d not found\n", minor);
+               ret = -ENODEV;
+               goto error_no_dev;
+       }
+
+       kref_get(&udev->ref_count);
+
+       mhi_dev = udev->mhi_dev;
+       dev = &mhi_dev->dev;
+
+       mutex_lock(&udev->lock);
+       if (kref_read(&udev->ref_count) > 2) {
+               dev_dbg(dev, "Node already opened\n");
+               goto exit_uci_open;
+       }
+
+       if (!udev->enabled) {
+               dev_info(dev, "Node exists, but is not in active state!\n");
+               goto error_open_chan;
+       }
+
+       dev_dbg(dev, "Starting channel\n");
+       ret = mhi_prepare_for_transfer(udev->mhi_dev);
+       if (ret) {
+               dev_err(dev, "Error starting transfer channels\n");
+               goto error_open_chan;
+       }
+
+       ret = mhi_queue_inbound(udev);
+       if (ret)
+               goto error_rx_queue;
+
+exit_uci_open:
+       filp->private_data = udev;
+       mutex_unlock(&udev->lock);
+
+       return 0;
+
+error_rx_queue:
+       dl_chan = &udev->dl_chan;
+       mhi_unprepare_from_transfer(udev->mhi_dev);
+       list_for_each_entry_safe(buf_itr, tmp, &dl_chan->pending, node) {
+               list_del(&buf_itr->node);
+               kfree(buf_itr->data);
+       }
+error_open_chan:
+       mutex_unlock(&udev->lock);
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+error_no_dev:
+       return ret;
+}
+
+static const struct file_operations mhidev_fops = {
+       .owner = THIS_MODULE,
+       .open = mhi_uci_open,
+       .release = mhi_uci_release,
+       .read = mhi_uci_read,
+       .write = mhi_uci_write,
+       .poll = mhi_uci_poll,
+};
+
+static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
+                          struct mhi_result *mhi_result)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+       struct uci_chan *uchan = &udev->ul_chan;
+       struct device *dev = &mhi_dev->dev;
+
+       dev_dbg(dev, "status: %d xfer_len: %zu\n",
+               mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+       kfree(mhi_result->buf_addr);
+
+       if (!mhi_result->transaction_status)
+               wake_up(&uchan->wq);
+}
+
+static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
+                          struct mhi_result *mhi_result)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+       struct uci_chan *uchan = &udev->dl_chan;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_buf *buf;
+
+       dev_dbg(dev, "status: %d receive_len: %zu\n",
+               mhi_result->transaction_status, mhi_result->bytes_xferd);

drivers/bus/mhi/uci.c line 511

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+
+       if (mhi_result->transaction_status == -ENOTCONN) {
+               kfree(mhi_result->buf_addr);
+               return;
+       }
+
+       spin_lock_bh(&uchan->lock);
+       buf = mhi_result->buf_addr + udev->actual_mtu;
+       buf->data = mhi_result->buf_addr;
+       buf->len = mhi_result->bytes_xferd;

You don't need to protect buf here, only uchan pending list, right? so move lock here.

Quoted 13 lines
drivers/bus/mhi/uci.c new file +13 -0
+       list_add_tail(&buf->node, &uchan->pending);
+       spin_unlock_bh(&uchan->lock);
+
+       wake_up(&uchan->wq);
+}
+
+static int mhi_uci_probe(struct mhi_device *mhi_dev,
+                        const struct mhi_device_id *id)
+{
+       struct uci_dev *udev;
+       struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+       struct device *dev;
+       int index, dir;

drivers/bus/mhi/uci.c line 534

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+
+       udev = kzalloc(sizeof(*udev), GFP_KERNEL);
+       if (!udev)
+               return -ENOMEM;
+
+       kref_init(&udev->ref_count);
+       mutex_init(&udev->lock);
+       udev->mhi_dev = mhi_dev;
+
+       mutex_lock(&udev->lock);

Why locking here? udev has just been created, it cannot be used concurrently.

Quoted 15 lines
drivers/bus/mhi/uci.c new file +15 -0
+
+       mutex_lock(&uci_idr_mutex);
+       index = idr_alloc(&uci_idr, udev, 0, MAX_UCI_MINORS, GFP_KERNEL);
+       mutex_unlock(&uci_idr_mutex);
+       if (index < 0) {
+               mutex_unlock(&udev->lock);
+               kfree(udev);
+               return index;
+       }
+
+       udev->minor = index;
+
+       /* create device file node /dev/mhi_<cntrl_dev_name>_<mhi_dev_name> */
+       dev = device_create(uci_dev_class, &mhi_dev->dev,
+                           MKDEV(uci_dev_major, index), udev,

drivers/bus/mhi/uci.c line 559

Quoted 10 lines
drivers/bus/mhi/uci.c new file +10 -0
+                           DEVICE_NAME "_%s_%s",
+                           dev_name(mhi_cntrl->cntrl_dev), mhi_dev->name);
+       if (IS_ERR(dev)) {
+               mutex_lock(&uci_idr_mutex);
+               idr_remove(&uci_idr, udev->minor);
+               mutex_unlock(&uci_idr_mutex);
+               mutex_unlock(&udev->lock);
+               kfree(udev);
+               return PTR_ERR(dev);
+       }

I would suggest adding the device at the very end of the probe, when everything has been initialized.

Regards,
Loic

Quoted 102 lines
drivers/bus/mhi/uci.c new file +98 -1
+
+       for (dir = 0; dir < 2; dir++) {
+               struct uci_chan *uchan = (dir) ?
+                       &udev->ul_chan : &udev->dl_chan;
+               spin_lock_init(&uchan->lock);
+               init_waitqueue_head(&uchan->wq);
+               INIT_LIST_HEAD(&uchan->pending);
+       }
+
+       udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU);
+       udev->actual_mtu = udev->mtu - sizeof(struct uci_buf);
+       dev_set_drvdata(&mhi_dev->dev, udev);
+       udev->enabled = true;
+
+       mutex_unlock(&udev->lock);
+
+       dev_info(&mhi_dev->dev, "probed uci dev: minor %d\n", index);
+
+       return 0;
+};
+
+static void mhi_uci_remove(struct mhi_device *mhi_dev)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+
+       mutex_lock(&udev->lock);
+
+       /* disable the node */
+       udev->enabled = false;
+
+       wake_up(&udev->dl_chan.wq);
+       wake_up(&udev->ul_chan.wq);
+
+       /* delete the node to prevent new opens */
+       device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
+
+       mutex_lock(&uci_idr_mutex);
+       idr_remove(&uci_idr, udev->minor);
+       mutex_unlock(&uci_idr_mutex);
+
+       mutex_unlock(&udev->lock);
+
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+}
+
+/* .driver_data stores max mtu */
+static const struct mhi_device_id mhi_uci_match_table[] = {
+       { .chan = "LOOPBACK", .driver_data = 0x1000},
+       {},
+};
+MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
+
+static struct mhi_driver mhi_uci_driver = {
+       .id_table = mhi_uci_match_table,
+       .remove = mhi_uci_remove,
+       .probe = mhi_uci_probe,
+       .ul_xfer_cb = mhi_ul_xfer_cb,
+       .dl_xfer_cb = mhi_dl_xfer_cb,
+       .driver = {
+               .name = MHI_UCI_DRIVER_NAME,
+       },
+};
+
+static int mhi_uci_init(void)
+{
+       int ret;
+
+       ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
+       if (ret < 0)
+               return ret;
+
+       uci_dev_major = ret;
+       uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
+       if (IS_ERR(uci_dev_class)) {
+               unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+               return -ENODEV;
+       }
+
+       ret = mhi_driver_register(&mhi_uci_driver);
+       if (ret) {
+               class_destroy(uci_dev_class);
+               unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+       }
+
+       return ret;
+}
+
+static void __exit mhi_uci_exit(void)
+{
+       mhi_driver_unregister(&mhi_uci_driver);
+       class_destroy(uci_dev_class);
+       unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+}
+
+module_init(mhi_uci_init);
+module_exit(mhi_uci_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MHI UCI Driver");
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project

#14 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Jeffrey Hugo

Hi Jeff,

Quoted 10 lines · Jeffrey Hugo
Quoted 9 lines · Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

...
...

Quoted 32 lines
Quoted 32 lines
drivers/bus/mhi/uci.c new file +32 -0
+
+static ssize_t mhi_uci_write(struct file *file,
+                 const char __user *buf,
+                 size_t count,
+                 loff_t *offp)
+{
+    struct uci_dev *udev = file->private_data;
+    struct mhi_device *mhi_dev = udev->mhi_dev;
+    struct device *dev = &mhi_dev->dev;
+    struct uci_chan *uchan = &udev->ul_chan;
+    size_t bytes_xfered = 0;
+    int ret, nr_avail = 0;
+
+    if (!buf || !count)
+        return -EINVAL;
+
+    /* confirm channel is active */
+    mutex_lock(&udev->lock);
+    if (!udev->enabled) {
+        ret = -ENODEV;
+        goto err_mtx_unlock;
+    }
+
+    dev_dbg(dev, "%s: to xfer: %lu bytes\n", __func__, count);
+
+    while (count) {
+        size_t xfer_size;
+        void *kbuf;
+        enum mhi_flags flags;
+
+        mutex_unlock(&udev->lock);
+        /* wait for free descriptors */
Quoted 8 lines
Quoted 6 lines

+        ret = wait_event_interruptible(uchan->wq,
+                           (!udev->enabled) ||
+                (nr_avail = mhi_get_no_free_descriptors(mhi_dev, +                           DMA_TO_DEVICE)) > 0);
+
+        mutex_lock(&udev->lock);

It feels like there is a race here.  What if we wait because there are no free descriptors, one becomes available, but someone else manages to grab the mutex before we do, and consume it?  mhi_queue_buf() will fail later on, and I don't think thats intended.

I agree Jeff, one option i can think of having separate read and write lock so that writer can hold lock if TREs are not available but reader can continue.

Quoted 186 lines
Quoted 186 lines

+        if (ret == -ERESTARTSYS) {
+            dev_dbg(dev, "Exit signal caught for node\n"); +            goto err_mtx_unlock;
+        }

drivers/bus/mhi/uci.c new file +181 -0
+
+        if (!udev->enabled) {
+            ret = -ENODEV;
+            goto err_mtx_unlock;
+        }
+
+        xfer_size = min_t(size_t, count, udev->mtu);
+        kbuf = kmalloc(xfer_size, GFP_KERNEL);
+        if (!kbuf) {
+            ret = -ENOMEM;
+            goto err_mtx_unlock;
+        }
+
+        ret = copy_from_user(kbuf, buf, xfer_size);
+        if (ret) {
+            kfree(kbuf);
+            ret = -EFAULT;
+            goto err_mtx_unlock;
+        }
+
+        /* if ring is full after this force EOT */
+        if (nr_avail > 1 && (count - xfer_size))
+            flags = MHI_CHAIN;
+        else
+            flags = MHI_EOT;
+
+        if (udev->enabled)
+            ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf,
+                        xfer_size, flags);
+        else
+            ret = -ENODEV;
+
+        if (ret) {
+            kfree(kbuf);
+            goto err_mtx_unlock;
+        }
+
+        bytes_xfered += xfer_size;
+        count -= xfer_size;
+        buf += xfer_size;
+    }
+
+    mutex_unlock(&udev->lock);
+    dev_dbg(dev, "%s: bytes xferred: %lu\n", __func__, bytes_xfered);
+
+    return bytes_xfered;
+
+err_mtx_unlock:
+    mutex_unlock(&udev->lock);
+
+    return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+                char __user *buf,
+                size_t count,
+                loff_t *ppos)
+{
+    struct uci_dev *udev = file->private_data;
+    struct mhi_device *mhi_dev = udev->mhi_dev;
+    struct uci_chan *uchan = &udev->dl_chan;
+    struct device *dev = &mhi_dev->dev;
+    struct uci_buf *ubuf;
+    char *ptr;
+    size_t to_copy;
+    int ret = 0;
+
+    if (!buf)
+        return -EINVAL;
+
+    mutex_lock(&udev->lock);
+    /* confirm channel is active */
+    if (!udev->enabled) {
+        ret = -ENODEV;
+        goto err_mtx_unlock;
+    }
+
+    spin_lock_bh(&uchan->lock);
+    /* No data available to read, wait */
+    if (!uchan->cur_buf && list_empty(&uchan->pending)) {
+        dev_dbg(dev, "No data available to read waiting\n");
+
+        spin_unlock_bh(&uchan->lock);
+        mutex_unlock(&udev->lock);
+        ret = wait_event_interruptible(uchan->wq,
+                           (!udev->enabled ||
+                          !list_empty(&uchan->pending)));
+
+        mutex_lock(&udev->lock);
+        if (ret == -ERESTARTSYS) {
+            dev_dbg(dev, "Exit signal caught for node\n");
+            goto err_mtx_unlock;
+        }
+
+        if (!udev->enabled) {
+            ret = -ENODEV;
+            goto err_mtx_unlock;
+        }
+        spin_lock_bh(&uchan->lock);
+    }
+
+    /* new read, get the next descriptor from the list */
+    if (!uchan->cur_buf) {
+        ubuf = list_first_entry_or_null(&uchan->pending,
+                        struct uci_buf, node);
+        if (!ubuf) {
+            ret = -EIO;
+            goto err_spin_unlock;
+        }
+
+        list_del(&ubuf->node);
+        uchan->cur_buf = ubuf;
+        uchan->rx_size = ubuf->len;
+        dev_dbg(dev, "Got pkt of size: %zu\n", uchan->rx_size);
+    }
+
+    ubuf = uchan->cur_buf;
+
+    /* Copy the buffer to user space */
+    to_copy = min_t(size_t, count, uchan->rx_size);
+    ptr = ubuf->data + (ubuf->len - uchan->rx_size);
+    spin_unlock_bh(&uchan->lock);
+
+    ret = copy_to_user(buf, ptr, to_copy);
+    if (ret) {
+        ret = -EFAULT;
+        goto err_mtx_unlock;
+    }
+
+    spin_lock_bh(&uchan->lock);
+
+    dev_dbg(dev, "Copied %lu of %lu bytes\n", to_copy, uchan->rx_size);
+    uchan->rx_size -= to_copy;
+
+    /* we finished with this buffer, queue it back to hardware */
+    if (!uchan->rx_size) {
+        uchan->cur_buf = NULL;
+
+        if (udev->enabled)
+            ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE,
+                        ubuf->data,
+                        udev->actual_mtu, MHI_EOT);
+        else
+            ret = -ENODEV;
+
+        if (ret) {
+            dev_err(dev, "Failed to recycle element: %d\n", ret);
+            kfree(ubuf->data);
+            goto err_spin_unlock;
+        }
+    }
+    spin_unlock_bh(&uchan->lock);
+    mutex_unlock(&udev->lock);
+
+    dev_dbg(dev, "%s: Returning %lu bytes\n", __func__, to_copy);
+
+    return to_copy;
+
+err_spin_unlock:
+    spin_unlock_bh(&uchan->lock);
+err_mtx_unlock:
+    mutex_unlock(&udev->lock);
+    return ret;
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+    struct uci_dev *udev = NULL;
+    unsigned int minor = iminor(inode);
+    int ret = -EIO;
+    struct uci_buf *buf_itr, *tmp;
+    struct uci_chan *dl_chan;
+    struct mhi_device *mhi_dev;
+    struct device *dev;
+
+    mutex_lock(&uci_idr_mutex);
+    udev = idr_find(&uci_idr, minor);
+    mutex_unlock(&uci_idr_mutex);
+    if (!udev) {
+        pr_err("uci dev: minor %d not found\n", minor);
+        ret = -ENODEV;
Quoted 8 lines

drivers/bus/mhi/uci.c line 422

Quoted 6 lines

+        goto error_no_dev;
+    }

drivers/bus/mhi/uci.c new file +4 -0
+
+    kref_get(&udev->ref_count);
+
+    mhi_dev = udev->mhi_dev;

mhi_dev never gets used.  You could use it down at the prepare_for_transfer, but you don't.  Might as well just remove this from the stack.

this was done to pass dev pointer instead of udev->mhi_dev->dev.

Quoted 99 lines
Quoted 99 lines

+    dev = &mhi_dev->dev;

drivers/bus/mhi/uci.c new file +97 -0
+
+    mutex_lock(&udev->lock);
+    if (kref_read(&udev->ref_count) > 2) {
+        dev_dbg(dev, "Node already opened\n");
+        goto exit_uci_open;
+    }
+
+    if (!udev->enabled) {
+        dev_info(dev, "Node exists, but is not in active state!\n");
+        goto error_open_chan;
+    }
+
+    dev_dbg(dev, "Starting channel\n");
+    ret = mhi_prepare_for_transfer(udev->mhi_dev);
+    if (ret) {
+        dev_err(dev, "Error starting transfer channels\n");
+        goto error_open_chan;
+    }
+
+    ret = mhi_queue_inbound(udev);
+    if (ret)
+        goto error_rx_queue;
+
+exit_uci_open:
+    filp->private_data = udev;
+    mutex_unlock(&udev->lock);
+
+    return 0;
+
+error_rx_queue:
+    dl_chan = &udev->dl_chan;
+    mhi_unprepare_from_transfer(udev->mhi_dev);
+    list_for_each_entry_safe(buf_itr, tmp, &dl_chan->pending, node) {
+        list_del(&buf_itr->node);
+        kfree(buf_itr->data);
+    }
+error_open_chan:
+    mutex_unlock(&udev->lock);
+    kref_put(&udev->ref_count, mhi_uci_dev_release);
+error_no_dev:
+    return ret;
+}
+
+static const struct file_operations mhidev_fops = {
+    .owner = THIS_MODULE,
+    .open = mhi_uci_open,
+    .release = mhi_uci_release,
+    .read = mhi_uci_read,
+    .write = mhi_uci_write,
+    .poll = mhi_uci_poll,
+};
+
+static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
+               struct mhi_result *mhi_result)
+{
+    struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+    struct uci_chan *uchan = &udev->ul_chan;
+    struct device *dev = &mhi_dev->dev;
+
+    dev_dbg(dev, "status: %d xfer_len: %zu\n",
+        mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+    kfree(mhi_result->buf_addr);
+
+    if (!mhi_result->transaction_status)
+        wake_up(&uchan->wq);
+}
+
+static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
+               struct mhi_result *mhi_result)
+{
+    struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+    struct uci_chan *uchan = &udev->dl_chan;
+    struct device *dev = &mhi_dev->dev;
+    struct uci_buf *buf;
+
+    dev_dbg(dev, "status: %d receive_len: %zu\n",
+        mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+    if (mhi_result->transaction_status == -ENOTCONN) {
+        kfree(mhi_result->buf_addr);
+        return;
+    }
+
+    spin_lock_bh(&uchan->lock);
+    buf = mhi_result->buf_addr + udev->actual_mtu;
+    buf->data = mhi_result->buf_addr;
+    buf->len = mhi_result->bytes_xferd;
+    list_add_tail(&buf->node, &uchan->pending);
+    spin_unlock_bh(&uchan->lock);
+
+    wake_up(&uchan->wq);
+}
+
+static int mhi_uci_probe(struct mhi_device *mhi_dev,
+             const struct mhi_device_id *id)
+{
Quoted 8 lines
Quoted 6 lines

+    struct uci_dev *udev;
+    struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl; +    struct device *dev;
+    int index, dir;
+
+    udev = kzalloc(sizeof(*udev), GFP_KERNEL);

What if you defer creating the udev until the first open(), and tie the xfer prepare state to the ref count of the udev instance?

With current implementation udev and minor are mapped using idr_alloc. i was thinking about another option of adding kref for uci_chann object to take care of channel and buffer ref count.

Quoted 43 lines
Quoted 22 lines

+    if (!udev)
+        return -ENOMEM;

drivers/bus/mhi/uci.c new file +19 -0
+
+    kref_init(&udev->ref_count);
+    mutex_init(&udev->lock);
+    udev->mhi_dev = mhi_dev;
+
+    mutex_lock(&udev->lock);
+
+    mutex_lock(&uci_idr_mutex);
+    index = idr_alloc(&uci_idr, udev, 0, MAX_UCI_MINORS, GFP_KERNEL);
+    mutex_unlock(&uci_idr_mutex);
+    if (index < 0) {
+        mutex_unlock(&udev->lock);
+        kfree(udev);
+        return index;
+    }
+
+    udev->minor = index;
+
+    /* create device file node 
Quoted 4 lines

/dev/mhi_<cntrl_dev_name>_<mhi_dev_name> */
+    dev = device_create(uci_dev_class, &mhi_dev->dev, +                MKDEV(uci_dev_major, index), udev, +                DEVICE_NAME "_%s_%s",

DEVICE_NAME is only used here.  Doesn't seem like having a macro is providing much value.  I suggest just inlining the value.

Quoted 14 lines

+                dev_name(mhi_cntrl->cntrl_dev), mhi_dev->name); +    if (IS_ERR(dev)) {
+        mutex_lock(&uci_idr_mutex);
+        idr_remove(&uci_idr, udev->minor);
+        mutex_unlock(&uci_idr_mutex);
+        mutex_unlock(&udev->lock);
+        kfree(udev);
+        return PTR_ERR(dev);
+    }
+
+    for (dir = 0; dir < 2; dir++) {
+        struct uci_chan *uchan = (dir) ?
+            &udev->ul_chan : &udev->dl_chan;
+        spin_lock_init(&uchan->lock);

Quoted 8 lines
Quoted 6 lines

+        init_waitqueue_head(&uchan->wq);
+        INIT_LIST_HEAD(&uchan->pending);
+    }
+
+    udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU); +    udev->actual_mtu = udev->mtu - sizeof(struct uci_buf);

"mtu" vs "actual_mtu" seems to be very confusing.  To start, why not "tx_mtu" and "rx_mtu"?  But going from there, why have different mtus for tx and rx?  Why not just increase the rx allocations by the required "header" (struct uci_buf) as needed to keep symetry?

in past we had issues with order of 3 allocation failures for bigger mtu size, so we decided to use memory within the mtu buffer. i can add tx_mtu and rx_mtu if that makes it clear.

Quoted 76 lines
Quoted 76 lines

+    dev_set_drvdata(&mhi_dev->dev, udev);
+    udev->enabled = true;

drivers/bus/mhi/uci.c new file +73 -0
+
+    mutex_unlock(&udev->lock);
+
+    dev_info(&mhi_dev->dev, "probed uci dev: minor %d\n", index);
+
+    return 0;
+};
+
+static void mhi_uci_remove(struct mhi_device *mhi_dev)
+{
+    struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+
+    mutex_lock(&udev->lock);
+
+    /* disable the node */
+    udev->enabled = false;
+
+    wake_up(&udev->dl_chan.wq);
+    wake_up(&udev->ul_chan.wq);
+
+    /* delete the node to prevent new opens */
+    device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
+
+    mutex_lock(&uci_idr_mutex);
+    idr_remove(&uci_idr, udev->minor);
+    mutex_unlock(&uci_idr_mutex);
+
+    mutex_unlock(&udev->lock);
+
+    kref_put(&udev->ref_count, mhi_uci_dev_release);
+}
+
+/* .driver_data stores max mtu */
+static const struct mhi_device_id mhi_uci_match_table[] = {
+    { .chan = "LOOPBACK", .driver_data = 0x1000},
+    {},
+};
+MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
+
+static struct mhi_driver mhi_uci_driver = {
+    .id_table = mhi_uci_match_table,
+    .remove = mhi_uci_remove,
+    .probe = mhi_uci_probe,
+    .ul_xfer_cb = mhi_ul_xfer_cb,
+    .dl_xfer_cb = mhi_dl_xfer_cb,
+    .driver = {
+        .name = MHI_UCI_DRIVER_NAME,
+    },
+};
+
+static int mhi_uci_init(void)
+{
+    int ret;
+
+    ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
+    if (ret < 0)
+        return ret;
+
+    uci_dev_major = ret;
+    uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
+    if (IS_ERR(uci_dev_class)) {
+        unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+        return -ENODEV;
+    }
+
+    ret = mhi_driver_register(&mhi_uci_driver);
+    if (ret) {
+        class_destroy(uci_dev_class);
+        unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+    }
+
+    return ret;
+}
Quoted 8 lines

drivers/bus/mhi/uci.c line 651

Quoted 6 lines
drivers/bus/mhi/uci.c new file +6 -0
+
+static void __exit mhi_uci_exit(void)
+{
+    mhi_driver_unregister(&mhi_uci_driver);
+    class_destroy(uci_dev_class);
+    unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);

I think you need an idr_destroy() of uci_idr here.

Done.

Quoted 9 lines
Quoted 8 lines
drivers/bus/mhi/uci.c new file +6 -0
+}
+
+module_init(mhi_uci_init);
+module_exit(mhi_uci_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MHI UCI Driver");
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#2 lore raw

Re: [PATCH v6 1/4] bus: mhi: core: Add helper API to return number of free TREs

In reply to Hemant Kumar
Quoted 18 lines · Hemant Kumar

Introduce mhi_get_no_free_descriptors() API to return number of TREs available to queue buffer. MHI clients can use this API to know before hand if ring is full without calling queue API.

---
 drivers/bus/mhi/core/main.c | 12 ++++++++++++
 include/linux/mhi.h         |  9 +++++++++
 2 files changed, 21 insertions(+)
drivers/bus/mhi/core/main.c +0 -0
index 2cff5dd..0599e7d 100644
@@ -258,6 +258,18 @@ int mhi_destroy_device(struct device *dev, void *data)
 	return 0;
 }

drivers/bus/mhi/core/main.c int mhi_destroy_device(struct device *dev, void *data) · line 269

Quoted 10 lines
drivers/bus/mhi/core/main.c +9 -0
 
+int mhi_get_no_free_descriptors(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir)
+{
+	struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+	struct mhi_chan *mhi_chan = (dir == DMA_TO_DEVICE) ?
+		mhi_dev->ul_chan : mhi_dev->dl_chan;
+	struct mhi_ring *tre_ring = &mhi_chan->tre_ring;
+
+	return get_nr_avail_ring_elements(mhi_cntrl, tre_ring);

Hmm, so this is essentially a wrapper for get_nr_avail_ring_elements(). Why can't you call this API directly?

Quoted 11 lines
drivers/bus/mhi/core/main.c +3 -0
+}
+EXPORT_SYMBOL_GPL(mhi_get_no_free_descriptors);
+
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason)
 {
 	struct mhi_driver *mhi_drv;
include/linux/mhi.h +0 -0
index a35d876..6565528 100644

include/linux/mhi.h void mhi_set_mhi_state(struct mhi_controller *mhi_cntrl, · line 603

Quoted 4 lines
include/linux/mhi.h +1 -0
@@ -600,6 +600,15 @@ void mhi_set_mhi_state(struct mhi_controller *mhi_cntrl,
 void mhi_notify(struct mhi_device *mhi_dev, enum mhi_callback cb_reason);
 
 /**
+ * mhi_get_no_free_descriptors - Get transfer ring length

mhi_get_nr_free_descriptors?

include/linux/mhi.h void mhi_set_mhi_state(struct mhi_controller *mhi_cntrl, · line 609

Quoted 6 lines
include/linux/mhi.h +6 -0
+ * Get # of TD available to queue buffers
+ * @mhi_dev: Device associated with the channels
+ * @dir: Direction of the channel
+ */
+int mhi_get_no_free_descriptors(struct mhi_device *mhi_dev,
+				enum dma_data_direction dir);

Align enum with start of "("

Thanks,
Mani

Quoted 9 lines
include/linux/mhi.h +2 -0
+
+/**
  * mhi_prepare_for_power_up - Do pre-initialization before power up.
  *                            This is optional, call this before power up if
  *                            the controller does not want bus framework to
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
Reviewed-by on v6 2/4
#4 lore raw

Re: [PATCH v6 2/4] bus: mhi: core: Move MHI_MAX_MTU to external header file

In reply to Hemant Kumar
Quoted 6 lines · Hemant Kumar

Currently this macro is defined in internal MHI header as a TRE length mask. Moving it to external header allows MHI client drivers to set this upper bound for the transmit buffer size.

Thanks,
Mani

Quoted 35 lines
---
 drivers/bus/mhi/core/internal.h | 1 -
 include/linux/mhi.h             | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/bus/mhi/core/internal.h +0 -1
index 7989269..4abf0cf 100644
@@ -453,7 +453,6 @@ enum mhi_pm_state {
 #define CMD_EL_PER_RING			128
 #define PRIMARY_CMD_RING		0
 #define MHI_DEV_WAKE_DB			127
-#define MHI_MAX_MTU			0xffff
 #define MHI_RANDOM_U32_NONZERO(bmsk)	(prandom_u32_max(bmsk) + 1)
 
 enum mhi_er_type {
include/linux/mhi.h +3 -0
index 6565528..610f3b0 100644
@@ -16,6 +16,9 @@
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 
+/* MHI client drivers to set this upper bound for tx buffer */
+#define MHI_MAX_MTU 0xffff
+
 #define MHI_MAX_OEM_PK_HASH_SEGMENTS 16
 
 struct mhi_chan;
Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
#16 lore raw

Re: [PATCH v6 4/4] bus: mhi: Add userspace client interface driver

In reply to Loic Poulain

Hi Loic,

Quoted 288 lines · Loic Poulain

Hi Hemant,

See comments inline, but globally, the locking and ref counting is more complicated than it should be.

Quoted 282 lines · Hemant Kumar

This MHI client driver allows userspace clients to transfer raw data between MHI device and host using standard file operations. Device file node is created with format

/dev/mhi_<controller_name>_<mhi_device_name>

Currently it supports LOOPBACK channel.

---
  drivers/bus/mhi/Kconfig  |  13 +
  drivers/bus/mhi/Makefile |   4 +
  drivers/bus/mhi/uci.c    | 657 +++++++++++++++++++++++++++++++++++++++++++++++
  3 files changed, 674 insertions(+)

create mode 100644 drivers/bus/mhi/uci.c

drivers/bus/mhi/Kconfig +13 -0
index 6a217ff..8aebe8b 100644
@@ -20,3 +20,16 @@ config MHI_BUS_DEBUG
          Enable debugfs support for use with the MHI transport. Allows
          reading and/or modifying some values within the MHI controller
          for debug and test purposes.
+
+config MHI_UCI
+       tristate "MHI UCI"
+       depends on MHI_BUS
+       help
+        MHI based userspace client interface driver is used for transferring
+        raw data between host and device using standard file operations from
+        userspace. Open, read, write, and close operations are supported
+        by this driver. Please check mhi_uci_match_table for all supported
+        channels that are exposed to userspace.
+
+        To compile this driver as a module, choose M here: the module will be
+        called mhi_uci.
drivers/bus/mhi/Makefile +4 -0
index 19e6443..80feefb 100644
@@ -1,2 +1,6 @@
  # core layer
  obj-y += core/
+
+# MHI client
+mhi_uci-y := uci.o
+obj-$(CONFIG_MHI_UCI) += mhi_uci.o
drivers/bus/mhi/uci.c new file +227 -0
new file mode 100644
index 0000000..d6758f2
@@ -0,0 +1,657 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2018-2020, The Linux Foundation. All rights reserved.*/
+
+#include <linux/kernel.h>
+#include <linux/mhi.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/poll.h>
+
+#define DEVICE_NAME "mhi"
+#define MHI_UCI_DRIVER_NAME "mhi_uci"
+#define MAX_UCI_MINORS (128)
+
+static DEFINE_IDR(uci_idr);
+static DEFINE_MUTEX(uci_idr_mutex);
+static struct class *uci_dev_class;
+static int uci_dev_major;
+
+/**
+ * struct uci_chan - MHI channel for a uci device
+ * @wq: wait queue for reader/writer
+ * @lock: spin lock
+ * @pending: list of rx buffers userspace is waiting to read
+ * @cur_buf: current buffer userspace is reading
+ * @rx_size: size of the current rx buffer userspace is reading
+ */
+struct uci_chan {
+       wait_queue_head_t wq;
+
+       /* protects pending and cur_buf members in bh context */
+       spinlock_t lock;
+
+       struct list_head pending;
+       struct uci_buf *cur_buf;
+       size_t rx_size;
+};
+
+/**
+ * struct uci_buf - uci buffer
+ * @data: data buffer
+ * @len: length of data buffer
+ * @node: list node of the uci buffer
+ */
+struct uci_buf {
+       void *data;
+       size_t len;
+       struct list_head node;
+};
+
+/**
+ * struct uci_dev - MHI uci device
+ * @minor: uci device node minor number
+ * @mhi_dev: associated mhi device object
+ * @chan: MHI channel name
+ * @lock: mutex lock
+ * @ul_chan: uplink uci channel object
+ * @dl_chan: downlink uci channel object
+ * @mtu: max tx buffer length
+ * @actual_mtu: maximum size of incoming buffer
+ * @open: open called for device node
+ * @enabled: uci device probed
+ * @ref_count: uci_dev reference count
+ */
+struct uci_dev {
+       unsigned int minor;
+       struct mhi_device *mhi_dev;
+       const char *chan;
+
+       /* protects uci_dev struct members */
+       struct mutex lock;
+
+       struct uci_chan ul_chan;
+       struct uci_chan dl_chan;
+       size_t mtu;
+       size_t actual_mtu;
+       bool enabled;
+       struct kref ref_count;
+};
+
+static int mhi_queue_inbound(struct uci_dev *udev)
+{
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       size_t mtu = udev->mtu;
+       size_t actual_mtu = udev->actual_mtu;
+       int nr_trbs, i, ret = -EIO;
+       void *buf;
+       struct uci_buf *uci_buf;
+
+       nr_trbs = mhi_get_no_free_descriptors(mhi_dev, DMA_FROM_DEVICE);
+
+       for (i = 0; i < nr_trbs; i++) {
+               buf = kmalloc(mtu, GFP_KERNEL);
+               if (!buf)
+                       return -ENOMEM;
+
+               uci_buf = buf + actual_mtu;
+               uci_buf->data = buf;
+
+               dev_dbg(dev, "Allocated buf %d of %d size %ld\n", i, nr_trbs,
+                       actual_mtu);
+
+               ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE, buf, actual_mtu,
+                                   MHI_EOT);
+               if (ret) {
+                       kfree(buf);
+                       dev_err(dev, "Failed to queue buffer %d\n", i);
+                       return ret;
+               }
+       }
+
+       return ret;
+}
+
+static void mhi_uci_dev_release(struct kref *ref)
+{
+       struct uci_dev *udev =
+               container_of(ref, struct uci_dev, ref_count);
+
+       mutex_destroy(&udev->lock);
+
+       dev_set_drvdata(&udev->mhi_dev->dev, NULL);
+
+       kfree(udev);
+}
+
+static int mhi_uci_release(struct inode *inode, struct file *file)
+{
+       struct uci_dev *udev = file->private_data;
+       struct uci_buf *itr, *tmp;
+       struct uci_chan *uchan;
+
+       if (kref_read(&udev->ref_count) > 2)
+               goto exit_uci_release;
+
+       if (udev->enabled)
+               mhi_unprepare_from_transfer(udev->mhi_dev);
+
+       /* clean inbound channel */
+       uchan = &udev->dl_chan;
+
+       spin_lock_bh(&uchan->lock);
+       list_for_each_entry_safe(itr, tmp, &uchan->pending, node) {
+               list_del(&itr->node);
+               kfree(itr->data);
+       }
+
+       if (uchan->cur_buf)
+               kfree(uchan->cur_buf->data);
+
+       uchan->cur_buf = NULL;
+       spin_unlock_bh(&uchan->lock);
+
+exit_uci_release:
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+
+       return 0;
+}
+
+static __poll_t mhi_uci_poll(struct file *file, poll_table *wait)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_chan *uchan;
+       __poll_t mask = 0;
+
+       poll_wait(file, &udev->dl_chan.wq, wait);
+       poll_wait(file, &udev->ul_chan.wq, wait);
+
+       if (!udev->enabled) {
+               mask = EPOLLERR;
+       } else {
+               uchan = &udev->dl_chan;
+               spin_lock_bh(&uchan->lock);
+               if (!list_empty(&uchan->pending) || uchan->cur_buf) {
+                       dev_dbg(dev, "Client can read from node\n");
+                       mask |= EPOLLIN | EPOLLRDNORM;
+               }
+               spin_unlock_bh(&uchan->lock);
+       }
+
+       if (!udev->enabled) {
+               mask |= EPOLLERR;
+       } else if (mhi_get_no_free_descriptors(mhi_dev, DMA_TO_DEVICE) > 0) {
+               dev_dbg(dev, "Client can write to node\n");
+               mask |= EPOLLOUT | EPOLLWRNORM;
+       }
+
+       dev_dbg(dev, "Client attempted to poll, returning mask 0x%x\n", mask);
+
+       return mask;
+}
+
+static ssize_t mhi_uci_write(struct file *file,
+                            const char __user *buf,
+                            size_t count,
+                            loff_t *offp)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_chan *uchan = &udev->ul_chan;
+       size_t bytes_xfered = 0;
+       int ret, nr_avail = 0;
+
+       if (!buf || !count)
+               return -EINVAL;
+
+       /* confirm channel is active */
+       mutex_lock(&udev->lock);
+       if (!udev->enabled) {
+               ret = -ENODEV;
+               goto err_mtx_unlock;
+       }
+
+       dev_dbg(dev, "%s: to xfer: %lu bytes\n", __func__, count);
+
+       while (count) {
+               size_t xfer_size;
+               void *kbuf;
+               enum mhi_flags flags;
+
+               mutex_unlock(&udev->lock);
+               /* wait for free descriptors */
+               ret = wait_event_interruptible(uchan->wq,
+                                              (!udev->enabled) ||
Quoted 8 lines

drivers/bus/mhi/uci.c line 231

Quoted 4 lines
drivers/bus/mhi/uci.c new file +4 -0
+                               (nr_avail = mhi_get_no_free_descriptors(mhi_dev,
+                                              DMA_TO_DEVICE)) > 0);
+
+               mutex_lock(&udev->lock);

All this locking unlocking is odd:
- why do you need locking for testing wait_event return code?
- why do you need this udev->enabled?

udev->enabled used in wait_event_interruptible to unblock in remove. I am going to get rid of udev->enabled check that is added on top of this function and also before calling mhi_queue_buf below.

Quoted 278 lines

- The MHI core should be thread safe + mhi_queue_buf should simply fail if removing is ongoing.

Quoted 275 lines
drivers/bus/mhi/uci.c new file +275 -0
+               if (ret == -ERESTARTSYS) {
+                       dev_dbg(dev, "Exit signal caught for node\n");
+                       goto err_mtx_unlock;
+               }
+
+               if (!udev->enabled) {
+                       ret = -ENODEV;
+                       goto err_mtx_unlock;
+               }
+
+               xfer_size = min_t(size_t, count, udev->mtu);
+               kbuf = kmalloc(xfer_size, GFP_KERNEL);
+               if (!kbuf) {
+                       ret = -ENOMEM;
+                       goto err_mtx_unlock;
+               }
+
+               ret = copy_from_user(kbuf, buf, xfer_size);
+               if (ret) {
+                       kfree(kbuf);
+                       ret = -EFAULT;
+                       goto err_mtx_unlock;
+               }
+
+               /* if ring is full after this force EOT */
+               if (nr_avail > 1 && (count - xfer_size))
+                       flags = MHI_CHAIN;
+               else
+                       flags = MHI_EOT;
+
+               if (udev->enabled)
+                       ret = mhi_queue_buf(mhi_dev, DMA_TO_DEVICE, kbuf,
+                                           xfer_size, flags);
+               else
+                       ret = -ENODEV;
+
+               if (ret) {
+                       kfree(kbuf);
+                       goto err_mtx_unlock;
+               }
+
+               bytes_xfered += xfer_size;
+               count -= xfer_size;
+               buf += xfer_size;
+       }
+
+       mutex_unlock(&udev->lock);
+       dev_dbg(dev, "%s: bytes xferred: %lu\n", __func__, bytes_xfered);
+
+       return bytes_xfered;
+
+err_mtx_unlock:
+       mutex_unlock(&udev->lock);
+
+       return ret;
+}
+
+static ssize_t mhi_uci_read(struct file *file,
+                           char __user *buf,
+                           size_t count,
+                           loff_t *ppos)
+{
+       struct uci_dev *udev = file->private_data;
+       struct mhi_device *mhi_dev = udev->mhi_dev;
+       struct uci_chan *uchan = &udev->dl_chan;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_buf *ubuf;
+       char *ptr;
+       size_t to_copy;
+       int ret = 0;
+
+       if (!buf)
+               return -EINVAL;
+
+       mutex_lock(&udev->lock);
+       /* confirm channel is active */
+       if (!udev->enabled) {
+               ret = -ENODEV;
+               goto err_mtx_unlock;
+       }
+
+       spin_lock_bh(&uchan->lock);
+       /* No data available to read, wait */
+       if (!uchan->cur_buf && list_empty(&uchan->pending)) {
+               dev_dbg(dev, "No data available to read waiting\n");
+
+               spin_unlock_bh(&uchan->lock);
+               mutex_unlock(&udev->lock);
+               ret = wait_event_interruptible(uchan->wq,
+                                              (!udev->enabled ||
+                                             !list_empty(&uchan->pending)));
+
+               mutex_lock(&udev->lock);
+               if (ret == -ERESTARTSYS) {
+                       dev_dbg(dev, "Exit signal caught for node\n");
+                       goto err_mtx_unlock;
+               }
+
+               if (!udev->enabled) {
+                       ret = -ENODEV;
+                       goto err_mtx_unlock;
+               }
+               spin_lock_bh(&uchan->lock);
+       }
+
+       /* new read, get the next descriptor from the list */
+       if (!uchan->cur_buf) {
+               ubuf = list_first_entry_or_null(&uchan->pending,
+                                               struct uci_buf, node);
+               if (!ubuf) {
+                       ret = -EIO;
+                       goto err_spin_unlock;
+               }
+
+               list_del(&ubuf->node);
+               uchan->cur_buf = ubuf;
+               uchan->rx_size = ubuf->len;
+               dev_dbg(dev, "Got pkt of size: %zu\n", uchan->rx_size);
+       }
+
+       ubuf = uchan->cur_buf;
+
+       /* Copy the buffer to user space */
+       to_copy = min_t(size_t, count, uchan->rx_size);
+       ptr = ubuf->data + (ubuf->len - uchan->rx_size);
+       spin_unlock_bh(&uchan->lock);
+
+       ret = copy_to_user(buf, ptr, to_copy);
+       if (ret) {
+               ret = -EFAULT;
+               goto err_mtx_unlock;
+       }
+
+       spin_lock_bh(&uchan->lock);
+
+       dev_dbg(dev, "Copied %lu of %lu bytes\n", to_copy, uchan->rx_size);
+       uchan->rx_size -= to_copy;
+
+       /* we finished with this buffer, queue it back to hardware */
+       if (!uchan->rx_size) {
+               uchan->cur_buf = NULL;
+
+               if (udev->enabled)
+                       ret = mhi_queue_buf(mhi_dev, DMA_FROM_DEVICE,
+                                           ubuf->data,
+                                           udev->actual_mtu, MHI_EOT);
+               else
+                       ret = -ENODEV;
+
+               if (ret) {
+                       dev_err(dev, "Failed to recycle element: %d\n", ret);
+                       kfree(ubuf->data);
+                       goto err_spin_unlock;
+               }
+       }
+       spin_unlock_bh(&uchan->lock);
+       mutex_unlock(&udev->lock);
+
+       dev_dbg(dev, "%s: Returning %lu bytes\n", __func__, to_copy);
+
+       return to_copy;
+
+err_spin_unlock:
+       spin_unlock_bh(&uchan->lock);
+err_mtx_unlock:
+       mutex_unlock(&udev->lock);
+       return ret;
+}
+
+static int mhi_uci_open(struct inode *inode, struct file *filp)
+{
+       struct uci_dev *udev = NULL;
+       unsigned int minor = iminor(inode);
+       int ret = -EIO;
+       struct uci_buf *buf_itr, *tmp;
+       struct uci_chan *dl_chan;
+       struct mhi_device *mhi_dev;
+       struct device *dev;
+
+       mutex_lock(&uci_idr_mutex);
+       udev = idr_find(&uci_idr, minor);
+       mutex_unlock(&uci_idr_mutex);
+       if (!udev) {
+               pr_err("uci dev: minor %d not found\n", minor);
+               ret = -ENODEV;
+               goto error_no_dev;
+       }
+
+       kref_get(&udev->ref_count);
+
+       mhi_dev = udev->mhi_dev;
+       dev = &mhi_dev->dev;
+
+       mutex_lock(&udev->lock);
+       if (kref_read(&udev->ref_count) > 2) {
+               dev_dbg(dev, "Node already opened\n");
+               goto exit_uci_open;
+       }
+
+       if (!udev->enabled) {
+               dev_info(dev, "Node exists, but is not in active state!\n");
+               goto error_open_chan;
+       }
+
+       dev_dbg(dev, "Starting channel\n");
+       ret = mhi_prepare_for_transfer(udev->mhi_dev);
+       if (ret) {
+               dev_err(dev, "Error starting transfer channels\n");
+               goto error_open_chan;
+       }
+
+       ret = mhi_queue_inbound(udev);
+       if (ret)
+               goto error_rx_queue;
+
+exit_uci_open:
+       filp->private_data = udev;
+       mutex_unlock(&udev->lock);
+
+       return 0;
+
+error_rx_queue:
+       dl_chan = &udev->dl_chan;
+       mhi_unprepare_from_transfer(udev->mhi_dev);
+       list_for_each_entry_safe(buf_itr, tmp, &dl_chan->pending, node) {
+               list_del(&buf_itr->node);
+               kfree(buf_itr->data);
+       }
+error_open_chan:
+       mutex_unlock(&udev->lock);
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+error_no_dev:
+       return ret;
+}
+
+static const struct file_operations mhidev_fops = {
+       .owner = THIS_MODULE,
+       .open = mhi_uci_open,
+       .release = mhi_uci_release,
+       .read = mhi_uci_read,
+       .write = mhi_uci_write,
+       .poll = mhi_uci_poll,
+};
+
+static void mhi_ul_xfer_cb(struct mhi_device *mhi_dev,
+                          struct mhi_result *mhi_result)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+       struct uci_chan *uchan = &udev->ul_chan;
+       struct device *dev = &mhi_dev->dev;
+
+       dev_dbg(dev, "status: %d xfer_len: %zu\n",
+               mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+       kfree(mhi_result->buf_addr);
+
+       if (!mhi_result->transaction_status)
+               wake_up(&uchan->wq);
+}
+
+static void mhi_dl_xfer_cb(struct mhi_device *mhi_dev,
+                          struct mhi_result *mhi_result)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+       struct uci_chan *uchan = &udev->dl_chan;
+       struct device *dev = &mhi_dev->dev;
+       struct uci_buf *buf;
+
+       dev_dbg(dev, "status: %d receive_len: %zu\n",
+               mhi_result->transaction_status, mhi_result->bytes_xferd);
+
+       if (mhi_result->transaction_status == -ENOTCONN) {
+               kfree(mhi_result->buf_addr);
+               return;
+       }
Quoted 8 lines

drivers/bus/mhi/uci.c line 511

Quoted 5 lines
drivers/bus/mhi/uci.c new file +5 -0
+
+       spin_lock_bh(&uchan->lock);
+       buf = mhi_result->buf_addr + udev->actual_mtu;
+       buf->data = mhi_result->buf_addr;
+       buf->len = mhi_result->bytes_xferd;

You don't need to protect buf here, only uchan pending list, right? so move lock here.

Done.

Quoted 18 lines
Quoted 18 lines
drivers/bus/mhi/uci.c new file +17 -0
+       list_add_tail(&buf->node, &uchan->pending);
+       spin_unlock_bh(&uchan->lock);
+
+       wake_up(&uchan->wq);
+}
+
+static int mhi_uci_probe(struct mhi_device *mhi_dev,
+                        const struct mhi_device_id *id)
+{
+       struct uci_dev *udev;
+       struct mhi_controller *mhi_cntrl = mhi_dev->mhi_cntrl;
+       struct device *dev;
+       int index, dir;
+
+       udev = kzalloc(sizeof(*udev), GFP_KERNEL);
+       if (!udev)
+               return -ENOMEM;
Quoted 8 lines

drivers/bus/mhi/uci.c line 534

Quoted 6 lines
drivers/bus/mhi/uci.c new file +6 -0
+
+       kref_init(&udev->ref_count);
+       mutex_init(&udev->lock);
+       udev->mhi_dev = mhi_dev;
+
+       mutex_lock(&udev->lock);

Why locking here? udev has just been created, it cannot be used concurrently.

Done, will remove it.

Quoted 21 lines
Quoted 21 lines
drivers/bus/mhi/uci.c new file +20 -0
+
+       mutex_lock(&uci_idr_mutex);
+       index = idr_alloc(&uci_idr, udev, 0, MAX_UCI_MINORS, GFP_KERNEL);
+       mutex_unlock(&uci_idr_mutex);
+       if (index < 0) {
+               mutex_unlock(&udev->lock);
+               kfree(udev);
+               return index;
+       }
+
+       udev->minor = index;
+
+       /* create device file node /dev/mhi_<cntrl_dev_name>_<mhi_dev_name> */
+       dev = device_create(uci_dev_class, &mhi_dev->dev,
+                           MKDEV(uci_dev_major, index), udev,
+                           DEVICE_NAME "_%s_%s",
+                           dev_name(mhi_cntrl->cntrl_dev), mhi_dev->name);
+       if (IS_ERR(dev)) {
+               mutex_lock(&uci_idr_mutex);
+               idr_remove(&uci_idr, udev->minor);
Quoted 8 lines

drivers/bus/mhi/uci.c line 559

Quoted 5 lines
drivers/bus/mhi/uci.c new file +5 -0
+               mutex_unlock(&uci_idr_mutex);
+               mutex_unlock(&udev->lock);
+               kfree(udev);
+               return PTR_ERR(dev);
+       }

I would suggest adding the device at the very end of the probe, when everything has been initialized.

Done, will do that.

Quoted 100 lines

Regards,
Loic

Quoted 94 lines
drivers/bus/mhi/uci.c new file +94 -0
+
+       for (dir = 0; dir < 2; dir++) {
+               struct uci_chan *uchan = (dir) ?
+                       &udev->ul_chan : &udev->dl_chan;
+               spin_lock_init(&uchan->lock);
+               init_waitqueue_head(&uchan->wq);
+               INIT_LIST_HEAD(&uchan->pending);
+       }
+
+       udev->mtu = min_t(size_t, id->driver_data, MHI_MAX_MTU);
+       udev->actual_mtu = udev->mtu - sizeof(struct uci_buf);
+       dev_set_drvdata(&mhi_dev->dev, udev);
+       udev->enabled = true;
+
+       mutex_unlock(&udev->lock);
+
+       dev_info(&mhi_dev->dev, "probed uci dev: minor %d\n", index);
+
+       return 0;
+};
+
+static void mhi_uci_remove(struct mhi_device *mhi_dev)
+{
+       struct uci_dev *udev = dev_get_drvdata(&mhi_dev->dev);
+
+       mutex_lock(&udev->lock);
+
+       /* disable the node */
+       udev->enabled = false;
+
+       wake_up(&udev->dl_chan.wq);
+       wake_up(&udev->ul_chan.wq);
+
+       /* delete the node to prevent new opens */
+       device_destroy(uci_dev_class, MKDEV(uci_dev_major, udev->minor));
+
+       mutex_lock(&uci_idr_mutex);
+       idr_remove(&uci_idr, udev->minor);
+       mutex_unlock(&uci_idr_mutex);
+
+       mutex_unlock(&udev->lock);
+
+       kref_put(&udev->ref_count, mhi_uci_dev_release);
+}
+
+/* .driver_data stores max mtu */
+static const struct mhi_device_id mhi_uci_match_table[] = {
+       { .chan = "LOOPBACK", .driver_data = 0x1000},
+       {},
+};
+MODULE_DEVICE_TABLE(mhi, mhi_uci_match_table);
+
+static struct mhi_driver mhi_uci_driver = {
+       .id_table = mhi_uci_match_table,
+       .remove = mhi_uci_remove,
+       .probe = mhi_uci_probe,
+       .ul_xfer_cb = mhi_ul_xfer_cb,
+       .dl_xfer_cb = mhi_dl_xfer_cb,
+       .driver = {
+               .name = MHI_UCI_DRIVER_NAME,
+       },
+};
+
+static int mhi_uci_init(void)
+{
+       int ret;
+
+       ret = register_chrdev(0, MHI_UCI_DRIVER_NAME, &mhidev_fops);
+       if (ret < 0)
+               return ret;
+
+       uci_dev_major = ret;
+       uci_dev_class = class_create(THIS_MODULE, MHI_UCI_DRIVER_NAME);
+       if (IS_ERR(uci_dev_class)) {
+               unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+               return -ENODEV;
+       }
+
+       ret = mhi_driver_register(&mhi_uci_driver);
+       if (ret) {
+               class_destroy(uci_dev_class);
+               unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+       }
+
+       return ret;
+}
+
+static void __exit mhi_uci_exit(void)
+{
+       mhi_driver_unregister(&mhi_uci_driver);
+       class_destroy(uci_dev_class);
+       unregister_chrdev(uci_dev_major, MHI_UCI_DRIVER_NAME);
+}
+
Quoted 8 lines
Quoted 8 lines
drivers/bus/mhi/uci.c new file +4 -1
+module_init(mhi_uci_init);
+module_exit(mhi_uci_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MHI UCI Driver");
--

The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project

Thanks for reviewing my change Loic, based on concerns from Greg and Jeff, i am working on pushing another patch set. This patch set also address locking related concern that you mentioned. Basically intent for adding udev ref count is to take care of race between driver remove() and file open(). So i am keeping udev ref count in the next patch set but in order to take care of corner cases i need to guard udev ref counting with global mutex. udev needs to ref count the channel so that mhi channel is started when first open is called and channel is stopped only when the last release is called. So we need some kind of ref counting for channel usage as well. Will remove the current way of doing channel ref counting using udev ref count value.

Thanks,
Hemant

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project