Skip to contentSkip to conversation

drivers/bus: Freescale Management Complex bus driver patch series

PATCH Superseded J. German Rivera 3 participants 10 messages Jan 17, 2015 – Feb 26, 2015

Subsystems (2)

v6 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v6 cover drivers/bus: Freescale Management Complex bus driver patch series

This patch series introduces Linux support for the Freescale Management Complex (fsl-mc) hardware. This patch series is dependent on the patch series "ARM64: Add support for FSL's LS2085A SoC"
(http://thread.gmane.org/gmane.linux.ports.arm.kernel/351829)

The fsl-mc is a hardware resource manager that manages specialized hardware objects used in network-oriented packet processing applications. After the fsl-mc block is enabled, pools of hardware resources are available, such as queues, buffer pools, I/O interfaces. These resources are building blocks that can be used to create functional hardware objects such as network interfaces, crypto accelerator instances, or L2 switches.

All the fsl-mc managed hardware resources/objects are represented in a physical grouping mechanism called a 'container' or DPRC (data path resource container).

Quoted 1 line

From the point of view of an OS, a DPRC functions similar to a plug

and play bus. Using fsl-mc commands software can enumerate the contents of the DPRC discovering the hardware objects present and binding them to drivers. Hardware objects can be created and removed dynamically, providing hot pluggability of the hardware objects.

Software contexts interact with the fsl-mc by sending commands through a memory mapped hardware interface called an "MC portal". Every fsl-mc object type has a command set to manage the objects. Key DPRC commands include:
-create/destroy a DPRC
-enumerate objects and resource pools in the DPRC, including
identifying mappable regions and the number of IRQs an object may have
-IRQ configuration
-move objects/resources between DPRCs
-connecting objects (e.g. connecting a network interface to
an L2 switch port)
-reset

Patch 1 contains a minimal set of low level functions to send an d receive commands to the fsl-mc. It includes support for basic management commands and commands to manipulate DPRC objects.

Patch 2 contains a platform device driver that sets up and registers the basic bus infrastructure including support for adding/removing devices, register/unregister of drivers, and bus match support to bind devices to drivers.

Patch 3 contains an driver that manages DPRC objects (the container that holds the hardware resources). This driver functions as a bus controller and handles enumeration of the objects in the container and hotplug events.

CHANGE HISTORY

Changes in v6:
- Upgraded MC flibs to version 0.5
- Fixed warnings from latest checkpatch

Changes in v5:
- Addressed comments from Alex Graf. Details in each patch.

Changes in v4:
- Addressed changes from ALex Graf. Details in each patch.
- Upgraded MC flibs to version 0.5
- Fixed bug related to setting fsl_mc_bus_type.dev_root too late

Changes in v3:
- Addressed pending issues not resolved in v2:
* Removed Doxygen comments as requested by Greg Kroah-Hartman
* Moved newly added FSL-specific header files from include/linux to include/linux/fsl
* Changed wording of licenses in MC flib source files
* Fixed sparse warning about context imbalance in 'mc_send_command'

- Addressed additional comments from Kim Phillips, Scott Wood and Timur Tabi. Details in each patch.

Issues pending resolution not addressed by v3:
- Checkpatch warnings:
* WARNING: DT compatible string "fsl,qoriq-mc" appears un-documented This warning will go away once the prerequisite patch series is accepted upstream.
* WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? This warning still happens even after adding MAINTAINERS file entries in the same patch where new files were added

Changes in v2:
- Added reference to the patch series this patch is dependent on for device tree and device tree bindings changes. (See above).
- Removed patch 4 (update to the MAINTAINERS file), as this is done now in patch 1
- Addressed comments from Joe Perches, Kim Phillips, Alex Graf and Scott Wood. Details in each patch.

Issues pending resolution not addressed by v2:
- What to do with Doxygen comments in patch 1
- Whether to move or not FSL-specific header files added in include/linux,
by this patch series, to another location
- Whether to change or not the wording of the licenses used in some
source files
- Checkpatch warnings:
* WARNING: DT compatible string "fsl,qoriq-mc" appears un-documented
This warning will go away once the prerequisite patch series is
accepted upstream.
* WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
This warning still happens even after adding MAINTAINERS file entries in
the same patch where new files were added
- Sparse warning:
* drivers/bus/fsl-mc/fsl_mc_sys.c:193:9: warning: context imbalance in 'mc_send_command' - different lock contexts for basic block This warning is caused by conditionally acquiring/releasing a lock in function mc_send_command().
Not clear how to solve this warning, as we need to be able to decide at
run time based on a flag whether to acquire the lock or not.

Changes in v1:
- No changes since RFC v4

Changes in RFC v4:
- Fixed parameter mismatch for device_find_child() call in fsl_mc_dprc.c
- Added back the dma_mask field to struct fsl_mc_device as it is needed by some MC child device drivers. However, the default DMA mask now does not have the 32-bit limitation of the original patch series (v1).

Changes in RFC v3:
Rework to address comments from Arnd Bergmann:
- Removed per-bus list of children, and instead use device_for_each_child()
- Use the same structure (struct fsl_mc_device) to represent both bus devices and their children.
http://www.mail-archive.com/linux-kernel%40vger.kernel.org/msg711301.html

Changes in RFC v2:
Rework to address comments from Arnd Bergmann:
- Removed fsl_mc_bus structure and global variable
- Removed the 'magic' fields from all structs
- Removed NULL initializers
- Replaced all occurrences of EXPORT_SYMBOL() with EXPORT_SYMBOL_GPL()
- Removed function dprc_parse_dt_node(), and replaced its call by a direct call to of_address_to_resource()
- Removed struct fsl_mc_device_region and use standard 'struct resource' instead.
- Removed dma_mask field from 'struct fsl_mc_device' as it is not currently being used.
- Removed redundant 'driver' field from struct fsl_mc_device
- Removed the container field. We can get the parent DPRC of a given dev, from its dev.parent field.
http://www.mail-archive.com/linux-kernel%40vger.kernel.org/msg708858.html

#1 lore raw

PATCH v6 1/3 drivers/bus: Added Freescale Management Complex APIs

APIs to access the Management Complex (MC) hardware module of Freescale LS2 SoCs. This patch includes APIs to check the MC firmware version and to manipulate DPRC objects in the MC.

---
Changes in v6:
- Upgraded MC flibs for MC firmware 0.5.3
- Fixed new checkpatch warnings

Changes in v5:
- Addressed comments from Alex Graf:
* Removed unused variable in mc_send_command()
* Renamed dprc_get_container_id() as dpmng_get_container_id() and moved corresponding code from dprc.[hc] to dpmng.[hc], as "get_container_id" is not really an operation on a DPRC object, but it is a general MC management operation.
* Ensure that strings sent/received to/from the MC fw for DPRC commands are null-terminated.

Changes in v4:
- Addressed comments from Alex Graf:
* Added file description for files that did not have one
* Removed Marshalling/unmarshalling macros invoked in MC command wrapper functions, and instead do the marshalling/ unmarshalling directly in these functions.
* Added type cast u32 in status range-check in mc_status_to_error()
* Moved mc_write_command() and mc_read_response() inline function to mc_sys.c as they are only used in that file
* Renamed u64_enc() as mc_enc() and u64_dec() as mc_dec()
- Upgraded MC flibs for MC firmware 0.5

Changes in v3:
- Addressed comment from Greg Kroah-Hartman:
* Removed doxygen comments

- Addressed comment from Scott Wood:
* Replaced udelay() call with usleep_range() call in polling loop
- Addressed comments from Kim Phillips:
* Fixed license text in all files
* Renamed files:

	drivers/bus/fsl-mc/fsl_dpmng_cmd.h -> drivers/bus/fsl-mc/dpmng-cmd.h
	drivers/bus/fsl-mc/fsl_dprc_cmd.h -> drivers/bus/fsl-mc/dprc-cmd.h
	drivers/bus/fsl-mc/fsl_mc_sys.c -> drivers/bus/fsl-mc/mc-sys.c
	include/linux/fsl_dpmng.h -> include/linux/fsl/dpmng.h
	include/linux/fsl_dprc.h -> include/linux/fsl/dprc.h
	include/linux/fsl_mc_cmd.h -> include/linux/fsl/mc-cmd.h
	include/linux/fsl_mc_sys.h -> include/linux/fsl/mc-sys.h

* Changed dpmng_load_iop() to take the DMA address directly, instead of the image virtual address.
* Removed if and WARN_ON that checks for NULL fsl_destroy_mc_io()
* Removed locking from mc_send_command(). Now the caller of MC flib APIs is responsible for protecting concurrent accesses to the same MC portal.

Changes in v2:
- Addressed comment from Joe Perches:
* Refactored logic to actively fail on err and proceed at the same indent level on success, for all functions in dprc.c and dpmng.c.

- Addressed comments from Kim Phillips:
* Fixed warning in mc_send_command
* Changed serialization logic in mc_send_command() to only use spinlock_irqsave() when both threads and interrupt handlers concurrently access the same portal.
* Changed switch to lookup table in mc_status_to_error()
* Removed macros iowrite64(), ioread64(), ENOTSUP from fsl_mc_sys.h
* Removed #ifdef side for FSL_MC_FIRMWARE in fsl_mc_cmd.h
* Changed non-devm_ API calls to devm_ API calls and refactored fsl_create_mc_io()/fsl_destroy_mc_io() to simplify cleanup logic.

- Addressed comments from Scott Wood:
* Return -ENXIO instead of -EFAULT when ioremap_nocache() fails

- Addressed comments from Alex Graf:
* Added MAINTAINERS file entries for new files
* Added dev param to fsl_create_mc_io(), to enable the use of devm_ APIs in this function and fsl_destroy_mc_io().
* Changed the value of the timeout for MC command completion to be a function of HZ instead of a hard-coded jiffies value.

 MAINTAINERS                    |   8 +
 drivers/bus/fsl-mc/dpmng-cmd.h |  47 +++
 drivers/bus/fsl-mc/dpmng.c     |  78 ++++
 drivers/bus/fsl-mc/dprc-cmd.h  |  84 ++++
 drivers/bus/fsl-mc/dprc.c      | 913 +++++++++++++++++++++++++++++++++++++++++
 drivers/bus/fsl-mc/mc-sys.c    | 283 +++++++++++++
 include/linux/fsl/dpmng.h      |  80 ++++
 include/linux/fsl/dprc.h       | 801 ++++++++++++++++++++++++++++++++++++
 include/linux/fsl/mc-cmd.h     | 113 +++++
 include/linux/fsl/mc-sys.h     |  70 ++++
 10 files changed, 2477 insertions(+)
 create mode 100644 drivers/bus/fsl-mc/dpmng-cmd.h
 create mode 100644 drivers/bus/fsl-mc/dpmng.c
 create mode 100644 drivers/bus/fsl-mc/dprc-cmd.h
 create mode 100644 drivers/bus/fsl-mc/dprc.c
 create mode 100644 drivers/bus/fsl-mc/mc-sys.c
 create mode 100644 include/linux/fsl/dpmng.h
 create mode 100644 include/linux/fsl/dprc.h
 create mode 100644 include/linux/fsl/mc-cmd.h
 create mode 100644 include/linux/fsl/mc-sys.h
MAINTAINERS +8 -0
index 8bed045..81701af 100644
@@ -4075,6 +4075,14 @@ F:	sound/soc/fsl/fsl*
 F:	sound/soc/fsl/imx*
 F:	sound/soc/fsl/mpc8610_hpcd.c
 
+FREESCALE QORIQ MANAGEMENT COMPLEX DRIVER
+M:	J. German Rivera <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	include/linux/fsl_mc*
+F:	include/linux/fsl_dp*
+F:	drivers/bus/fsl-mc/*
+
 FREEVXFS FILESYSTEM
 M:	Christoph Hellwig <[email protected]>
 W:	ftp://ftp.openlinux.org/pub/people/hch/vxfs
drivers/bus/fsl-mc/dpmng-cmd.h new file +47 -0

47 lines not on this page · raw message

drivers/bus/fsl-mc/dpmng.c new file +78 -0

78 lines not on this page · raw message

drivers/bus/fsl-mc/dprc-cmd.h new file +84 -0

84 lines not on this page · raw message

drivers/bus/fsl-mc/dprc.c new file +913 -0

913 lines not on this page · raw message

drivers/bus/fsl-mc/mc-sys.c new file +283 -0

283 lines not on this page · raw message

include/linux/fsl/dpmng.h new file +80 -0

80 lines not on this page · raw message

include/linux/fsl/dprc.h new file +801 -0

801 lines not on this page · raw message

include/linux/fsl/mc-cmd.h new file +113 -0

113 lines not on this page · raw message

include/linux/fsl/mc-sys.h new file +70 -0

70 lines not on this page · raw message

--
2.2.2

#2 lore raw

PATCH v6 2/3 drivers/bus: Freescale Management Complex (fsl-mc) bus driver

In reply to J. German Rivera

Platform device driver that sets up the basic bus infrastructure for the fsl-mc bus type, including support for adding/removing fsl-mc devices, register/unregister of fsl-mc drivers, and bus match support to bind devices to drivers.

---
Changes addressed in v6:
- Fixed new checkpatch warnings

Changes addressed in v5:
- Addressed comments from Alex Graf:
* Renamed dprc_get_container_id() call as dpmng_get_container_id().
* Added TODO comment to use the 'ranges' property of the fsl-mc DT node.

Changes addressed in v4:
- Addressed comments from Alex Graf:
* Added missing scope limitations and more descriptive help text for the FSL_MC_BUS config option.
- Fixed bug related to setting fsl_mc_bus_type.dev_root too late.

Changes in v3:
- Addressed changes from Kim Phillips:
* Renamed files:

	drivers/bus/fsl-mc/fsl_mc_bus.c -> drivers/bus/fsl-mc/mc-bus.c
	include/linux/fsl_mc.h -> include/linux/fsl/mc.h
	include/linux/fsl_mc_private.h -> include/linux/fsl/mc-private.h

- Addressed comments from Timur Tabi:
* Changed all functions that had goto out/error when no common cleanup was done, to just have multiple return points.
* Replaced error cleanup boolean flags with multiple exit points.

Changes in v2:
- Addressed comment from Joe Perches:
* Changed pr_debug to dev_dbg in fsl_mc_bus_match

- Addressed comments from Kim Phillips and Alex Graf:
* Changed version check to allow the driver to run with MC firmware that has major version number greater than or equal to the driver's major version number.
* Removed minor version check

- Removed unused variable parent_dev in fsl_mc_device_remove

 drivers/bus/Kconfig            |   3 +
 drivers/bus/Makefile           |   3 +
 drivers/bus/fsl-mc/Kconfig     |  24 ++
 drivers/bus/fsl-mc/Makefile    |  14 +
 drivers/bus/fsl-mc/mc-bus.c    | 585 +++++++++++++++++++++++++++++++++++++++++
 include/linux/fsl/mc-private.h |  33 +++
 include/linux/fsl/mc.h         | 137 ++++++++++
 7 files changed, 799 insertions(+)
 create mode 100644 drivers/bus/fsl-mc/Kconfig
 create mode 100644 drivers/bus/fsl-mc/Makefile
 create mode 100644 drivers/bus/fsl-mc/mc-bus.c
 create mode 100644 include/linux/fsl/mc-private.h
 create mode 100644 include/linux/fsl/mc.h
drivers/bus/Kconfig +3 -0
index b99729e..dde3ec2 100644
@@ -67,4 +67,7 @@ config VEXPRESS_CONFIG
 	help
 	  Platform configuration infrastructure for the ARM Ltd.
 	  Versatile Express.
+
+source "drivers/bus/fsl-mc/Kconfig"
+
 endmenu
drivers/bus/Makefile +3 -0
index 2973c18..6abcab1 100644
@@ -15,3 +15,6 @@ obj-$(CONFIG_ARM_CCI)		+= arm-cci.o
 obj-$(CONFIG_ARM_CCN)		+= arm-ccn.o
 
 obj-$(CONFIG_VEXPRESS_CONFIG)	+= vexpress-config.o
+
+# Freescale Management Complex (MC) bus drivers
+obj-$(CONFIG_FSL_MC_BUS)	+= fsl-mc/
drivers/bus/fsl-mc/Kconfig new file +24 -0
new file mode 100644
index 0000000..0d779d9
@@ -0,0 +1,24 @@
+#
+# Freescale Management Complex (MC) bus drivers
+#
+# Copyright (C) 2014 Freescale Semiconductor, Inc.
+#
+# This file is released under the GPLv2
+#
+
+config FSL_MC_BUS
+	tristate "Freescale Management Complex (MC) bus driver"
+	depends on OF && ARM64
+	help
+	  Driver to enable the bus infrastructure for the Freescale
+          QorIQ Management Complex (fsl-mc). The fsl-mc is a hardware
+	  module of the QorIQ LS2 SoCs, that does resource management
+	  for hardware building-blocks in the SoC that can be used
+	  to dynamically create networking hardware objects such as
+	  network interfaces (NICs), crypto accelerator instances,
+	  or L2 switches.
+
+	  Only enable this option when building the kernel for
+	  Freescale QorQIQ LS2xxxx SoCs.
+
+
drivers/bus/fsl-mc/Makefile new file +14 -0
new file mode 100644
index 0000000..decd339
@@ -0,0 +1,14 @@
+#
+# Freescale Management Complex (MC) bus drivers
+#
+# Copyright (C) 2014 Freescale Semiconductor, Inc.
+#
+# This file is released under the GPLv2
+#
+obj-$(CONFIG_FSL_MC_BUS) += mc-bus-driver.o
+
+mc-bus-driver-objs := mc-bus.o \
+		      mc-sys.o \
+		      dprc.o \
+		      dpmng.o
+
drivers/bus/fsl-mc/mc-bus.c new file +585 -0

585 lines not on this page · raw message

include/linux/fsl/mc-private.h new file +33 -0
new file mode 100644
index 0000000..604832a
@@ -0,0 +1,33 @@
+/*
+ * Freescale Management Complex (MC) bus private declarations
+ *
+ * Copyright (C) 2014 Freescale Semiconductor, Inc.
+ * Author: German Rivera <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+#ifndef _FSL_MC_PRIVATE_H_
+#define _FSL_MC_PRIVATE_H_
+
+#include <linux/fsl/mc.h>
+#include <linux/mutex.h>
+#include <linux/stringify.h>
+
+#define FSL_MC_DEVICE_MATCH(_mc_dev, _obj_desc) \
+	(strcmp((_mc_dev)->obj_desc.type, (_obj_desc)->type) == 0 && \
+	 (_mc_dev)->obj_desc.id == (_obj_desc)->id)
+
+#define FSL_MC_IS_ALLOCATABLE(_obj_type) \
+	(strcmp(_obj_type, "dpbp") == 0 || \
+	 strcmp(_obj_type, "dpcon") == 0)
+
+int __must_check fsl_mc_device_add(struct dprc_obj_desc *obj_desc,
+				   struct fsl_mc_io *mc_io,
+				   struct device *parent_dev,
+				   struct fsl_mc_device **new_mc_dev);
+
+void fsl_mc_device_remove(struct fsl_mc_device *mc_dev);
+
+#endif /* _FSL_MC_PRIVATE_H_ */
include/linux/fsl/mc.h new file +137 -0

137 lines not on this page · raw message

--
2.2.2

#3 lore raw

PATCH v6 3/3 drivers/bus: Device driver for FSL-MC DPRC devices

In reply to J. German Rivera

A DPRC (Data Path Resource Container) is an isolation device that contains a set of DPAA networking devices to be assigned to an isolation domain (e.g., a virtual machine).

---
Changes in v6:
- Fixed new checkpatch warnings

Changes in v5:
None

Changes in v4:
- Addressed comments from Alex Graf:
* Fixed typo in comment

Changes in v3:
- Addressed comments from Kim Phillips:
* Renamed files:

    drivers/bus/fsl-mc/fsl_mc_dprc.c -> drivers/bus/fsl-mc/dprc-driver.c

- Addressed comments from Timur Tabi:
* Changed dprc_scan_container() to just return dprc_scan_objects()
* Changed all functions that had goto out/error when no common cleanup was done, to just have multiple return points.
* Replaced error cleanup boolean flags with multiple exit points.
* REmoved __must_chewck from dprc_scan_*() functions

Changes in v2:
- Addressed comments from Kim Phillips:
* Fix warning in drivers/bus/fsl-mc/fsl_mc_dprc.c:173
* Fixed linker errors when MC bus driver built as module

 drivers/bus/fsl-mc/Makefile      |   3 +-
 drivers/bus/fsl-mc/dprc-driver.c | 383 +++++++++++++++++++++++++++++++++++++++
 drivers/bus/fsl-mc/mc-bus.c      |   8 +
 include/linux/fsl/mc-private.h   |  10 +
 4 files changed, 403 insertions(+), 1 deletion(-)
 create mode 100644 drivers/bus/fsl-mc/dprc-driver.c
drivers/bus/fsl-mc/Makefile +2 -1
index decd339..424e58e 100644
@@ -10,5 +10,6 @@ obj-$(CONFIG_FSL_MC_BUS) += mc-bus-driver.o
 mc-bus-driver-objs := mc-bus.o \
 		      mc-sys.o \
 		      dprc.o \
-		      dpmng.o
+		      dpmng.o \
+		      dprc-driver.o
 
drivers/bus/fsl-mc/dprc-driver.c new file +383 -0

383 lines not on this page · raw message

drivers/bus/fsl-mc/mc-bus.c +8 -0
index e6f3d62..1967807 100644
@@ -555,8 +555,15 @@ static int __init fsl_mc_bus_driver_init(void)
 		goto error_cleanup_bus;
 	}
 
+	error = dprc_driver_init();
+	if (error < 0)
+		goto error_cleanup_driver;
+
 	return 0;
 
+error_cleanup_driver:
+	platform_driver_unregister(&fsl_mc_bus_driver);
+
 error_cleanup_bus:
 	bus_unregister(&fsl_mc_bus_type);
 
@@ -572,6 +579,7 @@ static void __exit fsl_mc_bus_driver_exit(void)
 	if (WARN_ON(!mc_dev_cache))
 		return;
 
+	dprc_driver_exit();
 	platform_driver_unregister(&fsl_mc_bus_driver);
 	bus_unregister(&fsl_mc_bus_type);
 	kmem_cache_destroy(mc_dev_cache);
include/linux/fsl/mc-private.h +10 -0
index 604832a..431a0b1 100644
@@ -15,6 +15,8 @@
 #include <linux/mutex.h>
 #include <linux/stringify.h>
 
+#define FSL_MC_DPRC_DRIVER_NAME    "fsl_mc_dprc"
+
 #define FSL_MC_DEVICE_MATCH(_mc_dev, _obj_desc) \
 	(strcmp((_mc_dev)->obj_desc.type, (_obj_desc)->type) == 0 && \
 	 (_mc_dev)->obj_desc.id == (_obj_desc)->id)
@@ -30,4 +32,12 @@ int __must_check fsl_mc_device_add(struct dprc_obj_desc *obj_desc,
 
 void fsl_mc_device_remove(struct fsl_mc_device *mc_dev);
 
+int dprc_scan_container(struct fsl_mc_device *mc_bus_dev);
+
+int dprc_scan_objects(struct fsl_mc_device *mc_bus_dev);
+
+int __init dprc_driver_init(void);
+
+void __exit dprc_driver_exit(void);
+
 #endif /* _FSL_MC_PRIVATE_H_ */

--
2.2.2

#4 lore raw
In reply to J. German Rivera

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Thanks,
Stuart

Quoted 155 lines

-----Original Message-----
From: J. German Rivera [mailto:[email protected]]
Sent: Friday, January 16, 2015 7:01 PM
To: [email protected]; [email protected]; [email protected]
Cc: Yoder Stuart-B08248; Phillips Kim-R1AAHA; Wood Scott-B07421; [email protected]; Hamciuc Bogdan-BHAMCIU1; Marginean Alexandru-R89243; Thorpe Geoff-R01361; Sharma Bhupesh-B45370; Erez Nir-RM30794; Schmitt Richard- B43082
Subject: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

This patch series introduces Linux support for the Freescale Management Complex (fsl-mc) hardware. This patch series is dependent on the patch series "ARM64: Add support for FSL's LS2085A SoC"
(http://thread.gmane.org/gmane.linux.ports.arm.kernel/351829)

The fsl-mc is a hardware resource manager that manages specialized hardware objects used in network-oriented packet processing applications. After the fsl-mc block is enabled, pools of hardware resources are available, such as queues, buffer pools, I/O interfaces. These resources are building blocks that can be used to create functional hardware objects such as network interfaces, crypto accelerator instances, or L2 switches.

All the fsl-mc managed hardware resources/objects are represented in a physical grouping mechanism called a 'container' or DPRC (data path resource container).

From the point of view of an OS, a DPRC functions similar to a plug and play bus. Using fsl-mc commands software can enumerate the contents of the DPRC discovering the hardware objects present and binding them to drivers. Hardware objects can be created and removed dynamically, providing hot pluggability of the hardware objects.

Software contexts interact with the fsl-mc by sending commands through a memory mapped hardware interface called an "MC portal". Every fsl-mc object type has a command set to manage the objects. Key DPRC commands include:
-create/destroy a DPRC
-enumerate objects and resource pools in the DPRC, including
identifying mappable regions and the number of IRQs an object may have
-IRQ configuration
-move objects/resources between DPRCs
-connecting objects (e.g. connecting a network interface to
an L2 switch port)
-reset

Patch 1 contains a minimal set of low level functions to send an d receive commands to the fsl-mc. It includes support for basic management commands and commands to manipulate DPRC objects.

Patch 2 contains a platform device driver that sets up and registers the basic bus infrastructure including support for adding/removing devices, register/unregister of drivers, and bus match support to bind devices to drivers.

Patch 3 contains an driver that manages DPRC objects (the container that holds the hardware resources). This driver functions as a bus controller and handles enumeration of the objects in the container and hotplug events.

CHANGE HISTORY

Changes in v6:
- Upgraded MC flibs to version 0.5
- Fixed warnings from latest checkpatch

Changes in v5:
- Addressed comments from Alex Graf. Details in each patch.

Changes in v4:
- Addressed changes from ALex Graf. Details in each patch.
- Upgraded MC flibs to version 0.5
- Fixed bug related to setting fsl_mc_bus_type.dev_root too late

Changes in v3:
- Addressed pending issues not resolved in v2:
* Removed Doxygen comments as requested by Greg Kroah-Hartman
* Moved newly added FSL-specific header files from include/linux to include/linux/fsl
* Changed wording of licenses in MC flib source files
* Fixed sparse warning about context imbalance in 'mc_send_command'

- Addressed additional comments from Kim Phillips, Scott Wood and Timur Tabi. Details in each patch.

Issues pending resolution not addressed by v3:
- Checkpatch warnings:
* WARNING: DT compatible string "fsl,qoriq-mc" appears un-documented This warning will go away once the prerequisite patch series is accepted upstream.
* WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? This warning still happens even after adding MAINTAINERS file entries in the same patch where new files were added

Changes in v2:
- Added reference to the patch series this patch is dependent on for device tree and device tree bindings changes. (See above).
- Removed patch 4 (update to the MAINTAINERS file), as this is done now in patch 1
- Addressed comments from Joe Perches, Kim Phillips, Alex Graf and Scott Wood. Details in each patch.

Issues pending resolution not addressed by v2:
- What to do with Doxygen comments in patch 1
- Whether to move or not FSL-specific header files added in include/linux,
by this patch series, to another location
- Whether to change or not the wording of the licenses used in some
source files
- Checkpatch warnings:
* WARNING: DT compatible string "fsl,qoriq-mc" appears un-documented
This warning will go away once the prerequisite patch series is
accepted upstream.
* WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
This warning still happens even after adding MAINTAINERS file entries in
the same patch where new files were added
- Sparse warning:
* drivers/bus/fsl-mc/fsl_mc_sys.c:193:9: warning: context imbalance in 'mc_send_command' - different lock
contexts for basic block
This warning is caused by conditionally acquiring/releasing a lock in function mc_send_command(). Not clear how to solve this warning, as we need to be able to decide at
run time based on a flag whether to acquire the lock or not.

Changes in v1:
- No changes since RFC v4

Changes in RFC v4:
- Fixed parameter mismatch for device_find_child() call in fsl_mc_dprc.c
- Added back the dma_mask field to struct fsl_mc_device as it is needed by some MC child device drivers. However, the default DMA mask now does not have the 32-bit limitation of the original patch series (v1).

Changes in RFC v3:
Rework to address comments from Arnd Bergmann:
- Removed per-bus list of children, and instead use device_for_each_child()
- Use the same structure (struct fsl_mc_device) to represent both bus devices and their children.
http://www.mail-archive.com/linux-kernel%40vger.kernel.org/msg711301.html

Changes in RFC v2:
Rework to address comments from Arnd Bergmann:
- Removed fsl_mc_bus structure and global variable
- Removed the 'magic' fields from all structs
- Removed NULL initializers
- Replaced all occurrences of EXPORT_SYMBOL() with EXPORT_SYMBOL_GPL()
- Removed function dprc_parse_dt_node(), and replaced its call by a direct call to of_address_to_resource()
- Removed struct fsl_mc_device_region and use standard 'struct resource' instead.
- Removed dma_mask field from 'struct fsl_mc_device' as it is not currently being used.
- Removed redundant 'driver' field from struct fsl_mc_device
- Removed the container field. We can get the parent DPRC of a given dev, from its dev.parent field.
http://www.mail-archive.com/linux-kernel%40vger.kernel.org/msg708858.html

#5 lore raw
Quoted 5 lines · Stuart Yoder

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

Quoted 8 lines

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Ok, so if I grasp this correctly the idea is that we have a driver attaching to an individual device on the fsl-mc bus. That driver then goes and allocates / blocks more devices from that bus as it initializes.

Is that model always possible? Which device would a NIC bind to for example? I merely want to make sure we're not running ourselves into a bad corner ;).

Alex

#6 lore raw
Quoted 23 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 8:33 AM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 11 lines · Stuart Yoder

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

Quoted 5 lines
Quoted 2 lines

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Ok, so if I grasp this correctly the idea is that we have a driver attaching to an individual device on the fsl-mc bus.

Yes.

Quoted 2 lines

That driver then
goes and allocates / blocks more devices from that bus as it initializes.

Yes, there are certain devices/objects on the bus that by themselves are not standalone, functional devices. An example is a "buffer pool". Network interface drivers, crypto driver, decompression driver, etc need one or more hardware buffer pools. There is a buffer depletion interrupt associated with the device.

The buffer pools itself binds to a resource allocation driver in the kernel, which then can hand out buffer pools as required by other drivers.

Quoted 1 line

Is that model always possible?

Yes, why would it not be?

Quoted 2 lines

Which device would a NIC bind to for
example?

Network interface / Ethernet driver requires some number of buffer pools, plus a management complex portal device (DPMCP) used for sending commands to manage the hardware.

Quoted 2 lines

I merely want to make sure we're not running ourselves into a bad corner ;).

If we are, I would like to understand it. :)

Thanks,
Stuart

#7 lore raw
Quoted 40 lines · Stuart Yoder
Quoted 23 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 8:33 AM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 11 lines · Stuart Yoder

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

Quoted 5 lines
Quoted 2 lines

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Ok, so if I grasp this correctly the idea is that we have a driver attaching to an individual device on the fsl-mc bus.

Yes.

Quoted 2 lines

That driver then
goes and allocates / blocks more devices from that bus as it initializes.

Yes, there are certain devices/objects on the bus that by themselves are not standalone, functional devices. An example is a "buffer pool". Network interface drivers, crypto driver, decompression driver, etc need one or more hardware buffer pools. There is a buffer depletion interrupt associated with the device.

Quoted 3 lines

The buffer pools itself binds to a resource allocation driver in the kernel, which then can hand out buffer pools as required by other drivers.

Ok, so there are 2 things on the bus

* devices
* resources

Someone really needs to sit down and write some nice ASCII art about all of this and include all the abbreviations in it as well, so that anyone not deeply involved in the architecture has the chance to grasp what this is about.

Quoted 10 lines
Quoted 1 line

Is that model always possible?

Yes, why would it not be?

Quoted 2 lines

Which device would a NIC bind to for
example?

Network interface / Ethernet driver requires some number of buffer pools, plus a management complex portal device (DPMCP) used for sending commands to manage the hardware.

Ok, so there is always one object that basically "owns" a particular device. And then there is a cloud of resources that drivers grab as they go.

I think I got it by now and the concept makes a lot of sense. I'm not sure whether there's any particular benefit or downside of having resources be devices, but looking at the resource manager code it probably doesn't hurt.

Alex

#8 lore raw
Quoted 52 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 3:38 PM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 40 lines · Stuart Yoder
Quoted 23 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 8:33 AM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 11 lines · Stuart Yoder

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

Quoted 5 lines
Quoted 2 lines

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Ok, so if I grasp this correctly the idea is that we have a driver attaching to an individual device on the fsl-mc bus.

Yes.

Quoted 2 lines

That driver then
goes and allocates / blocks more devices from that bus as it initializes.

Yes, there are certain devices/objects on the bus that by themselves are not standalone, functional devices. An example is a "buffer pool". Network interface drivers, crypto driver, decompression driver, etc need one or more hardware buffer pools. There is a buffer depletion interrupt associated with the device.

Quoted 8 lines
Quoted 3 lines

The buffer pools itself binds to a resource allocation driver in the kernel, which then can hand out buffer pools as required by other drivers.

Ok, so there are 2 things on the bus

* devices
* resources

In the general sense, yes. To be picky about terminology we call all these things on the bus "objects". Some are more resource-like, in that they are handed out by an allocator to the functional drivers.

I don't want to call them 'resources' because that term actually means something slightly different in the hardware architecture that is not actually visible to Linux.

Quoted 4 lines

Someone really needs to sit down and write some nice ASCII art about all of this and include all the abbreviations in it as well, so that anyone not deeply involved in the architecture has the chance to grasp what this is about.

The cover letter for the patch series is a starting point, but yes we need something for ./Documentation.

Quoted 11 lines
Quoted 10 lines
Quoted 1 line

Is that model always possible?

Yes, why would it not be?

Quoted 2 lines

Which device would a NIC bind to for
example?

Network interface / Ethernet driver requires some number of buffer pools, plus a management complex portal device (DPMCP) used for sending commands to manage the hardware.

Quoted 7 lines

Ok, so there is always one object that basically "owns" a particular device. And then there is a cloud of resources that drivers grab as they go.

I think I got it by now and the concept makes a lot of sense. I'm not sure whether there's any particular benefit or downside of having resources be devices, but looking at the resource manager code it probably doesn't hurt.

They need to be real Linux devices. The reason is that when we bind a DPRC and the objects in it to VFIO, VFIO expects everything to be a device. VFIO exposes 'devices' to user space, and so for example a buffer pool's IRQ needs to be exposed via standard VFIO mechanisms.

Thanks,
Stuart

#9 lore raw
Quoted 96 lines · Stuart Yoder
Quoted 52 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 3:38 PM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 40 lines · Stuart Yoder
Quoted 23 lines

-----Original Message-----
From: Alexander Graf [mailto:[email protected]]
Sent: Thursday, February 26, 2015 8:33 AM
To: Yoder Stuart-B08248; [email protected]
Cc: Rivera Jose-B46482; [email protected]; [email protected]
Subject: Re: [PATCH 0/3 v6] drivers/bus: Freescale Management Complex bus driver patch series

Quoted 11 lines · Stuart Yoder

Hi Arnd/Alex,

German has posted an example driver for the fsl-mc bus in his RFC "[RFC PATCH 1/1] drivers/bus: fsl-mc object allocator driver".

In addition I have made available the skeleton for a driver for one of the objects/devices (crypto) that will be discovered on the bus:
https://github.com/stuyoder/linux
branch: fsl-ms-bus

Quoted 5 lines
Quoted 2 lines

...it is not functional yet, but shows how a driver registers with the bus, get's probed, performs initialization.

Ok, so if I grasp this correctly the idea is that we have a driver attaching to an individual device on the fsl-mc bus.

Yes.

Quoted 2 lines

That driver then
goes and allocates / blocks more devices from that bus as it initializes.

Yes, there are certain devices/objects on the bus that by themselves are not standalone, functional devices. An example is a "buffer pool". Network interface drivers, crypto driver, decompression driver, etc need one or more hardware buffer pools. There is a buffer depletion interrupt associated with the device.

Quoted 8 lines
Quoted 3 lines

The buffer pools itself binds to a resource allocation driver in the kernel, which then can hand out buffer pools as required by other drivers.

Ok, so there are 2 things on the bus

* devices
* resources

In the general sense, yes. To be picky about terminology we call all these things on the bus "objects". Some are more resource-like, in that they are handed out by an allocator to the functional drivers.

I don't want to call them 'resources' because that term actually means something slightly different in the hardware architecture that is not actually visible to Linux.

Quoted 4 lines

Someone really needs to sit down and write some nice ASCII art about all of this and include all the abbreviations in it as well, so that anyone not deeply involved in the architecture has the chance to grasp what this is about.

The cover letter for the patch series is a starting point, but yes we need something for ./Documentation.

Quoted 11 lines
Quoted 10 lines
Quoted 1 line

Is that model always possible?

Yes, why would it not be?

Quoted 2 lines

Which device would a NIC bind to for
example?

Network interface / Ethernet driver requires some number of buffer pools, plus a management complex portal device (DPMCP) used for sending commands to manage the hardware.

Quoted 7 lines

Ok, so there is always one object that basically "owns" a particular device. And then there is a cloud of resources that drivers grab as they go.

I think I got it by now and the concept makes a lot of sense. I'm not sure whether there's any particular benefit or downside of having resources be devices, but looking at the resource manager code it probably doesn't hurt.

Quoted 5 lines

They need to be real Linux devices. The reason is that when we bind a DPRC and the objects in it to VFIO, VFIO expects everything to be a device. VFIO exposes 'devices' to user space, and so for example a buffer pool's IRQ needs to be exposed via standard VFIO mechanisms.

Ah, I see. Yeah, certainly works well for me :).

Alex