Skip to contentSkip to conversation

bus: fsl-mc: enhance Management Complex userspace support

PATCH New Ioana Ciornei 1 participant 5 messages Oct 5, 2018

Subsystems (3)
lore
Share & tools
lore

Ioana Ciornei

author maintainer
#0 lore raw

cover bus: fsl-mc: enhance Management Complex userspace support

This patch set adds userspace support in the fsl-mc bus along with a rescan attribute to the root DPRC container. Also, the patches here depend on 2 other patches submitted but not yet accepted:
- https://lkml.org/lkml/2018/10/2/593
- https://lkml.org/lkml/2018/10/1/747

An earlier discussion on this functionality can be found here:
https://lkml.org/lkml/2018/3/23/941

The FSL_MC_SEND_MC_COMMAND ioctl acts as a direct passthrough to the Management Complex (or MC) by passing fixed-length command/response pairs. Keeping in mind that the MC manages DPAA2 hardware resources and provides and object-based abstraction for sofware drivers, the proposed ioctl interface enables userspace tools to do the following:
- create and destroy DPAA2 objects
- manage the DPRC container that objects live in (by moving objects between containers)
- establish connections between different objects as needed

The ioctl interface is intended for dynamic usecases where DPAA2 objects need to be created on demand or destroyed so that the underlying hardware resources can be further repurposed. In static usecases, depending on the requirements, a firmware configuration file can be used to describe the needed DPAA2 objects, their attributes or any link between them.

Some examples of device drivers that probe on said DPAA2 objects are listed
below:
- https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/soc/fsl/dpio
- https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git/tree/drivers/net/ethernet/freescale/dpaa2
- https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git/tree/drivers/crypto/caam/

Having a low-level ioctl interface where the command is passed through to the MC without any intervention from the fsl-mc bus reduces the complexity added in the kernel while also making the command structure, in turn the ioctl, very stable.

Ioana Ciornei (4):
bus: fsl-mc: move fsl_mc_command struct in a uapi header
bus: fsl-mc: add fsl-mc userspace support
bus: fsl-mc: add root dprc rescan attribute
bus: fsl-mc: add bus rescan attribute

 Documentation/ABI/stable/sysfs-bus-fsl-mc |  19 ++++
 Documentation/ioctl/ioctl-number.txt      |   1 +
 MAINTAINERS                               |   2 +
 drivers/bus/fsl-mc/Kconfig                |   7 ++
 drivers/bus/fsl-mc/Makefile               |   3 +
 drivers/bus/fsl-mc/dprc-driver.c          |  38 ++++++-
 drivers/bus/fsl-mc/fsl-mc-bus.c           |  53 +++++++++
 drivers/bus/fsl-mc/fsl-mc-private.h       |  43 ++++++++
 drivers/bus/fsl-mc/fsl-mc-uapi.c          | 175 ++++++++++++++++++++++++++++++
 include/linux/fsl/mc.h                    |   8 +-
 include/uapi/linux/fsl_mc.h               |  34 ++++++
 11 files changed, 374 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/ABI/stable/sysfs-bus-fsl-mc
 create mode 100644 drivers/bus/fsl-mc/fsl-mc-uapi.c
 create mode 100644 include/uapi/linux/fsl_mc.h
Signature · 1 line
1.9.1
author maintainer
#1 lore raw

PATCH 1/4 bus: fsl-mc: move fsl_mc_command struct in a uapi header

Define "struct fsl_mc_command" as a structure that can cross the user/kernel boundary.

---
Depends on:
https://lkml.org/lkml/2018/10/2/593

 MAINTAINERS                 |  1 +
 include/linux/fsl/mc.h      |  8 +-------
 include/uapi/linux/fsl_mc.h | 25 +++++++++++++++++++++++++
 3 files changed, 27 insertions(+), 7 deletions(-)
 create mode 100644 include/uapi/linux/fsl_mc.h
MAINTAINERS +1 -0
index b22e7fd..0ee7e17 100644
@@ -12022,6 +12022,7 @@ S:	Maintained
 F:	drivers/bus/fsl-mc/
 F:	Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt
 F:	Documentation/networking/dpaa2/overview.rst
+F:	include/uapi/linux/fsl_mc.h
 
 QT1010 MEDIA DRIVER
 M:	Antti Palosaari <[email protected]>
include/linux/fsl/mc.h +1 -7
index 96c54bb..c015bed 100644
@@ -12,6 +12,7 @@
 #include <linux/device.h>
 #include <linux/mod_devicetable.h>
 #include <linux/interrupt.h>
+#include <uapi/linux/fsl_mc.h>
 
 #define FSL_MC_VENDOR_FREESCALE	0x1957
 
@@ -198,8 +199,6 @@ struct fsl_mc_device {
 #define to_fsl_mc_device(_dev) \
 	container_of(_dev, struct fsl_mc_device, dev)
 
-#define MC_CMD_NUM_OF_PARAMS	7
-
 struct mc_cmd_header {
 	u8 src_id;
 	u8 flags_hw;
@@ -209,11 +208,6 @@ struct mc_cmd_header {
 	__le16 cmd_id;
 };
 
-struct fsl_mc_command {
-	__le64 header;
-	__le64 params[MC_CMD_NUM_OF_PARAMS];
-};
-
 enum mc_cmd_status {
 	MC_CMD_STATUS_OK = 0x0, /* Completed successfully */
 	MC_CMD_STATUS_READY = 0x1, /* Ready to be processed */
include/uapi/linux/fsl_mc.h new file +25 -0
new file mode 100644
index 0000000..6c19cfc
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Management Complex (MC) userspace public interface
+ *
+ * Copyright 2018 NXP
+ *
+ */
+#ifndef _UAPI_FSL_MC_H_
+#define _UAPI_FSL_MC_H_
+
+#include <linux/types.h>
+
+#define MC_CMD_NUM_OF_PARAMS	7
+
+/**
+ * struct fsl_mc_command - Management Complex (MC) command structure
+ * @header: MC command header
+ * @params: MC command parameters
+ */
+struct fsl_mc_command {
+	__le64 header;
+	__le64 params[MC_CMD_NUM_OF_PARAMS];
+};
+
+#endif /* _UAPI_FSL_MC_H_ */
Signature · 1 line
1.9.1
author maintainer
#2 lore raw

PATCH 2/4 bus: fsl-mc: add fsl-mc userspace support

In reply to Ioana Ciornei

Adding userspace support for the MC (Management Complex) means exporting an ioctl capable device file representing the root resource container.

This new functionality in the fsl-mc bus driver intends to provide userspace applications an interface to interact with the MC firmware.

Commands that are composed in userspace are sent to the MC firmware through the FSL_MC_SEND_MC_COMMAND ioctl. By default the implicit MC I/O portal is used for this operation, but if the implicit one is busy, a dynamic portal is allocated and then freed upon execution.

---
 Documentation/ioctl/ioctl-number.txt |   1 +
 drivers/bus/fsl-mc/Kconfig           |   7 ++
 drivers/bus/fsl-mc/Makefile          |   3 +
 drivers/bus/fsl-mc/fsl-mc-bus.c      |  12 +++
 drivers/bus/fsl-mc/fsl-mc-private.h  |  41 ++++++++
 drivers/bus/fsl-mc/fsl-mc-uapi.c     | 175 +++++++++++++++++++++++++++++++++++
 include/uapi/linux/fsl_mc.h          |   9 ++
 7 files changed, 248 insertions(+)
 create mode 100644 drivers/bus/fsl-mc/fsl-mc-uapi.c
Documentation/ioctl/ioctl-number.txt +1 -0
index 13a7c99..b29a213 100644
@@ -171,6 +171,7 @@ Code  Seq#(hex)	Include File		Comments
 'R'	00-1F	linux/random.h		conflict!
 'R'	01	linux/rfkill.h		conflict!
 'R'	C0-DF	net/bluetooth/rfcomm.h
+'R'	E0	uapi/linux/fsl_mc.h
 'S'	all	linux/cdrom.h		conflict!
 'S'	80-81	scsi/scsi_ioctl.h	conflict!
 'S'	82-FF	scsi/scsi.h		conflict!
drivers/bus/fsl-mc/Kconfig +7 -0
index c23c77c..cde6f40 100644
@@ -14,3 +14,10 @@ config FSL_MC_BUS
 	  architecture.  The fsl-mc bus driver handles discovery of
 	  DPAA2 objects (which are represented as Linux devices) and
 	  binding objects to drivers.
+
+config FSL_MC_UAPI_SUPPORT
+	bool "Management Complex (MC) userspace support"
+	depends on FSL_MC_BUS
+	help
+	  Provides userspace support for creating/destroying/configuring
+	  DPAA2 objects in the Management Complex.
drivers/bus/fsl-mc/Makefile +3 -0
index 3c518c7..4ae292a 100644
@@ -16,3 +16,6 @@ mc-bus-driver-objs := fsl-mc-bus.o \
 		      fsl-mc-allocator.o \
 		      fsl-mc-msi.o \
 		      dpmcp.o
+
+# MC userspace support
+obj-$(CONFIG_FSL_MC_UAPI_SUPPORT) += fsl-mc-uapi.o
drivers/bus/fsl-mc/fsl-mc-bus.c +12 -0
index 5d8266c..5f862f1 100644
@@ -792,6 +792,7 @@ static int fsl_mc_bus_probe(struct platform_device *pdev)
 	struct fsl_mc *mc;
 	struct fsl_mc_device *mc_bus_dev = NULL;
 	struct fsl_mc_io *mc_io = NULL;
+	struct fsl_mc_bus *mc_bus = NULL;
 	int container_id;
 	phys_addr_t mc_portal_phys_addr;
 	u32 mc_portal_size;
@@ -863,9 +864,18 @@ static int fsl_mc_bus_probe(struct platform_device *pdev)
 	if (error < 0)
 		goto error_cleanup_mc_io;
 
+	mc_bus = to_fsl_mc_bus(mc_bus_dev);
+	error = fsl_mc_uapi_create_device_file(mc_bus);
+	if (error < 0)
+		goto error_cleanup_device;
+
 	mc->root_mc_bus_dev = mc_bus_dev;
+
 	return 0;
 
+error_cleanup_device:
+	fsl_mc_device_remove(mc_bus_dev);
+
 error_cleanup_mc_io:
 	fsl_destroy_mc_io(mc_io);
 	return error;
@@ -878,10 +888,12 @@ static int fsl_mc_bus_probe(struct platform_device *pdev)
 static int fsl_mc_bus_remove(struct platform_device *pdev)
 {
 	struct fsl_mc *mc = platform_get_drvdata(pdev);
+	struct fsl_mc_bus *mc_bus = to_fsl_mc_bus(mc->root_mc_bus_dev);
 
 	if (!fsl_mc_is_root_dprc(&mc->root_mc_bus_dev->dev))
 		return -EINVAL;
 
+	fsl_mc_uapi_remove_device_file(mc_bus);
 	fsl_mc_device_remove(mc->root_mc_bus_dev);
 
 	fsl_destroy_mc_io(mc->root_mc_bus_dev->mc_io);
drivers/bus/fsl-mc/fsl-mc-private.h +41 -0

69 lines not on this page · raw message

drivers/bus/fsl-mc/fsl-mc-uapi.c new file +175 -0

175 lines not on this page · raw message

include/uapi/linux/fsl_mc.h +9 -0
index 6c19cfc..4989c1b 100644
@@ -16,10 +16,19 @@
  * struct fsl_mc_command - Management Complex (MC) command structure
  * @header: MC command header
  * @params: MC command parameters
+ *
+ * Used by FSL_MC_SEND_MC_COMMAND
  */
 struct fsl_mc_command {
 	__le64 header;
 	__le64 params[MC_CMD_NUM_OF_PARAMS];
 };
 
+#define FSL_MC_SEND_CMD_IOCTL_TYPE	'R'
+#define FSL_MC_SEND_CMD_IOCTL_SEQ	0xE0
+
+#define FSL_MC_SEND_MC_COMMAND \
+	_IOWR(FSL_MC_SEND_CMD_IOCTL_TYPE, FSL_MC_SEND_CMD_IOCTL_SEQ, \
+	struct fsl_mc_command)
+
 #endif /* _UAPI_FSL_MC_H_ */
Signature · 1 line
1.9.1
author maintainer
#3 lore raw

PATCH 3/4 bus: fsl-mc: add root dprc rescan attribute

In reply to Ioana Ciornei

Introduce the rescan attribute as a device attribute to synchronize the fsl-mc bus objects and the MC firmware.

To rescan the root dprc only, e.g.
echo 1 > /sys/bus/fsl-mc/devices/dprc.1/rescan

---
 Documentation/ABI/stable/sysfs-bus-fsl-mc |  9 ++++++++
 MAINTAINERS                               |  1 +
 drivers/bus/fsl-mc/dprc-driver.c          | 34 +++++++++++++++++++++++++++++++
 3 files changed, 44 insertions(+)
 create mode 100644 Documentation/ABI/stable/sysfs-bus-fsl-mc
Documentation/ABI/stable/sysfs-bus-fsl-mc new file +9 -0
new file mode 100644
index 0000000..f2f14f4
@@ -0,0 +1,9 @@
+What:		/sys/bus/fsl-mc/devices/dprc.*/rescan
+Date:		October 2018
+KernelVersion:	4.20
+Contact:	Ioana Ciornei <[email protected]>
+Description:	Writing a non-zero value to this attribute will
+		force a rescan of dprc.X container in the system and
+		synchronize the objects under dprc.X and the
+		Management Complex firmware.
+Users:		Userspace drivers and management tools
MAINTAINERS +1 -0
index 0ee7e17..57e1509 100644
@@ -12023,6 +12023,7 @@ F:	drivers/bus/fsl-mc/
 F:	Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt
 F:	Documentation/networking/dpaa2/overview.rst
 F:	include/uapi/linux/fsl_mc.h
+F:	Documentation/ABI/stable/sysfs-bus-fsl-mc
 
 QT1010 MEDIA DRIVER
 M:	Antti Palosaari <[email protected]>
drivers/bus/fsl-mc/dprc-driver.c +34 -0
index 52c7e15..dadc2f3 100644
@@ -354,6 +354,33 @@ static int dprc_scan_container(struct fsl_mc_device *mc_bus_dev)
 	return 0;
 }
 
+static ssize_t rescan_store(struct device *dev,
+			    struct device_attribute *attr,
+			    const char *buf, size_t count)
+{
+	struct fsl_mc_device *root_mc_dev;
+	struct fsl_mc_bus *root_mc_bus;
+	unsigned long val;
+
+	if (!fsl_mc_is_root_dprc(dev))
+		return -EINVAL;
+
+	root_mc_dev = to_fsl_mc_device(dev);
+	root_mc_bus = to_fsl_mc_bus(root_mc_dev);
+
+	if (kstrtoul(buf, 0, &val) < 0)
+		return -EINVAL;
+
+	if (val) {
+		mutex_lock(&root_mc_bus->scan_mutex);
+		dprc_scan_objects(root_mc_dev, NULL);
+		mutex_unlock(&root_mc_bus->scan_mutex);
+	}
+
+	return count;
+}
+static DEVICE_ATTR_WO(rescan);
+
 /**
  * dprc_irq0_handler - Regular ISR for DPRC interrupt 0
  *
@@ -686,6 +713,13 @@ static int dprc_probe(struct fsl_mc_device *mc_dev)
 
 	mutex_init(&mc_bus->scan_mutex);
 
+	error = device_create_file(&mc_dev->dev, &dev_attr_rescan);
+	if (error < 0) {
+		dev_err(&mc_dev->dev, "device_create_file() failed: %d\n",
+			error);
+		goto error_cleanup_open;
+	}
+
 	/*
 	 * Discover MC objects in DPRC object:
 	 */
Signature · 1 line
1.9.1
author maintainer
#4 lore raw

PATCH 4/4 bus: fsl-mc: add bus rescan attribute

In reply to Ioana Ciornei

Introduce the rescan attribute as a bus attribute to synchronize the fsl-mc bus objects and the MC firmware.

To rescan the fsl-mc bus, e.g.,
echo 1 > /sys/bus/fsl-mc/rescan

---
Depends on:
https://lkml.org/lkml/2018/10/1/747

 Documentation/ABI/stable/sysfs-bus-fsl-mc | 10 ++++++++
 drivers/bus/fsl-mc/dprc-driver.c          |  4 +--
 drivers/bus/fsl-mc/fsl-mc-bus.c           | 41 +++++++++++++++++++++++++++++++
 drivers/bus/fsl-mc/fsl-mc-private.h       |  2 ++
 4 files changed, 55 insertions(+), 2 deletions(-)
Documentation/ABI/stable/sysfs-bus-fsl-mc +10 -0
index f2f14f4..97c541a 100644
@@ -7,3 +7,13 @@ Description:	Writing a non-zero value to this attribute will
 		synchronize the objects under dprc.X and the
 		Management Complex firmware.
 Users:		Userspace drivers and management tools
+
+What:		/sys/bus/fsl-mc/rescan
+Date:		October 2018
+KernelVersion:	4.20
+Contact:	Ioana Ciornei <[email protected]>
+Description:	Writing a non-zero value to this attribute will
+		force a rescan of fsl-mc bus in the system and
+		synchronize the objects under fsl-mc bus and the
+		Management Complex firmware.
+Users:		Userspace drivers and management tools
drivers/bus/fsl-mc/dprc-driver.c +2 -2
index dadc2f3..5a5a6f6 100644
@@ -214,8 +214,8 @@ static void dprc_add_new_devices(struct fsl_mc_device *mc_bus_dev,
  * populated before they can get allocation requests from probe callbacks
  * of the device drivers for the non-allocatable devices.
  */
-static int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev,
-			     unsigned int *total_irq_count)
+int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev,
+		      unsigned int *total_irq_count)
 {
 	int num_child_objects;
 	int dprc_get_obj_failures;
drivers/bus/fsl-mc/fsl-mc-bus.c +41 -0

52 lines not on this page · raw message

drivers/bus/fsl-mc/fsl-mc-private.h +2 -0
index 16902f9..ab22347 100644
@@ -545,6 +545,8 @@ int __must_check fsl_mc_device_add(struct fsl_mc_obj_desc *obj_desc,
 
 void dprc_driver_exit(void);
 
+int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev,
+		      unsigned int *total_irq_count);
 int __init fsl_mc_allocator_driver_init(void);
 
 void fsl_mc_allocator_driver_exit(void);
Signature · 1 line
1.9.1